Over the past years, Jupyter Notebook gained much interest from the programming community and is considered one of the most useful and popular tools you can currently find. We’d like to share with you our opinion of Jupyter Notebook and how exactly we used it in our development process.
What is Jupyter Notebook?
Jupyter Notebook (earlier known as IPython Notebook) is an interactive computing environment which allows you to work on notebook documents that can contain live code, explanatory text, equations and multimedia resources like audio and video. Jupyter can and very often is used for scientific purposes as well as data transformation, numerical simulations, and machine learning. It supports over 40 programming languages, especially ones used in data science like Python and R.
For us, Jupyter Notebook is a really great tool. Code cells and easy way of creating data visualizations inside notebook are really helpful.
Live Access to Django APP through Jupyter Notebook
Django is one of our favorite web frameworks. Some of you probably know this, but there is a good library called django-extensions which adds a couple of useful elements. One of those extensions is django shell_plus which can be run as a notebook. This gives us some new options for working with Django, all of them based on Jupyter Notebook characteristics and features. For example, we can easily extract django orm objects, apply any operations we need and save them back to database.
This became especially useful during development of one of our data analysis platforms. We had an API deployed in production server. Sometimes we had to work on complex computations off-schedule which were normally automated and ran during nights or weekends. Without Jupyter we had to go through Rancher (awesome docker orchestration platform we use), run console on app container, initiate the shell_plus and only then we had access to Django objects. Moreover, if we had to do the same computation again we had to repeat all the steps again, previous work was lost in terminal history. Not very effective.
What we did to overcome this issue was as simple as running
manage.py shell_plus --notebook
on app container and exposing Jupyter port so that we could connect to it remotely (no worries, we secured it too).
We could “store” all of the necessary scripts in our notebook, run them and add new ones. And all of it had to be written just once. Additionally, Jupyter’s code cell architecture gave us an opportunity to just run a single, secluded script without running the whole set.
Possible future implementation
Theoretically, we could make a set of different notebooks divided thematically; for example one for operations on projects, one responsible for portfolios etc. We also consider creating a Docker image which would, by default, add the option to work through Jupyter Notebook to each Django project.