Deploying Jupyter in Ubuntu with Nginx and Supervisor

Image Credit: Google

The IPython Notebook or Jupyter Notebook is a convenient and interactive web application for running codes in Python (and R, Ruby, Octave and others) in the web browser.

Installing it on a local (Ubuntu) machine is easy, but to install it on the server and running it as SaaS (Software as a Service) takes quite a bit of effort. This tutorial will serve as the simplest way to Deploy Jupyter in a Ubuntu Server, using the Nginx Web server and the Supervisor system. I’ll try to keep this tutorial short😉 and cover most of the topics, so just sit back tight, and take a dive in this tutorial!

Installing Jupyter

Here, we assume that you’ll be using Python 3.x as the default kernel. We can install Jupyter by following command, which would all install the dependencies:

sudo pip3 install --upgrade pip
sudo pip3 install jupyter

Installing additional Kernels (Optional)

Jupyter supports a large number of kernels such as R, Scala and Octave (Isn’t that cool😎😍). For example, the octave kernel can be installed by:

sudo pip3 install octave_kernel
sudo python3 -m octave_kernel.install

Configuring Jupyter

Once installed, we can run the Jupyter notebook server using the following command:

jupyter notebook

This command lets you run the notebook in the local machine. We will be able to access it on loaclhost:8888. However, this is not what we want. We want to install and deploy Jupyter on the server (e.g. an instance of AWS) and access it from wherever we want. We want it to automatically get restarted in case it has stopped for some reasons because we don’t want it to be done manually😪😌. In this tutorial we’ll be accessing the notebook using ports (like server-ip:<port-no>) and original ports will be hidden behind Nginx Reverse Proxy. Now its time to configure our notebook.

jupyter notebook --generate-config

By default, the configuration file, jupyter_notebook_config.py will be generated under the .jupyter folder under your home directory. The file is well commented and should be self-explanatory. If you wanna play with the settings😎👍, you are free to do so! Before going forward, we need to make some changes in jupyter_notebook_config.py so, head over to the text editor such as vim (recommended) or nano and start making these changes:

c.NotebookApp.allow_remote_access = True
c.NotebookApp.allow_root = False
c.NotebookApp.notebook_dir = '/path/to/notebook_directory'
c.NotebookApp.port = 8888
c.NotebookApp.port_retries = 50

As for the port option (c.NotebookApp.port), you can set it to whichever port you prefer (just remember⚠️, it shouldn’t be a reserved port). Now, we will configure nginx as a reverse proxy to redirect requests to your notebook server. Here, we assume that you have set the port number to 8888 as above.

Managing firewalls

Before we proceed, we’ve reached the stage where we’ll manage firewalls so that we can access our notebook which was deployed on the server. Enable the firewall by entering sudo ufw enable.

⚠️Beware, we must allow port 22 on firewall so that we won’t lose ssh connection to the server. To do this, sudo ufw allow 22/tcp is enough.

Don’t forget to restart the firewall (sudo ufw reload), whenever you allow or denya new port so that new changes can be loaded.

Now its time to test our notebook, but before that, we must allow the port 8888 connection from the firewall and this can be done by:

sudo ufw allow 8888/tcp
sudo ufw reload

Now start the notebook by jupyter notebook head over to your-server-ip:8888 (here, we are assuming the port is 8888) and with this, let’s move to the next part😎.

With this, we have reached our first checkpoint. If you wanted to take a break or drink a cup of coffee, go ahead. But do remember😜 I’ll be waiting for you😉

Setting up a password to Jupyter

It’s now time to set a password to your notebook. You can follow the instructions in Jupyter documentation to obtain password string.

In [1]: from notebook.auth import passwd
In [2]: passwd()
Enter password:
Verify password:
Out[2]: 'sha1:67c9e60bb8b6:9ffede0825894254b2e042ea597d771089e11aed'

An image below shows the final screen while setting up a password.

Set-up password

Setting up Nginx Reverse Proxy

Nginx is an event-driven based HTTP and reverse proxy server. In this guide, we will install Nginx as a Web server listening for HTTP requests on a custom port (other than 8888, let say 1222), and will redirect requests to port 8888 if they are for the Jupyter Server. We are doing this so as to hide the original port (as exposing the original port is not recommended due to security reasons).

Firstly, let’s install Nginx using the command below:

sudo apt update
sudo apt install nginx

Start the service by sudo service nginx restartand check its status by sudo service nginx status .

Once installed, we now have to configure nginx. For this we’ll have to create a new file named jupyter.conf and make changes to it:

cd /etc/nginx/conf.d
sudo touch jupyter.conf

Now head over to your favourite text editor and add the following block in jupyter.conf :

Jupyter Nginx Config. File

NOTE: The port on proxy_pass should match Notebook’s (c.NotebookApp.port as configured earliar) port otherwise we won’t be able to connect.

In the above configuration, we had requested nginx to redirect all request of port 1222 to port 8888. (Don’t forget to allow port 1222in firewall). Once done, restart the nginx server by :

sudo service nginx restart

If you want to use any port other than 1222 you are free to do so, just make sure that you have made essential changes in the configuration file.

Whoof! that’s a hell lot of typing😥 But don’t worry, with this, we have reached the final checkpoint. If you feel like tired, pause it here, take a walk in the fresh air and come back soon. I’ll be waiting for you😉 and then we’ll march together till the end of the tutorial!😜

Supervisor — For Monitoring the Service

Finally, we will be able to execute Jupyter notebook on port 8888 and access it through port 1222. Isn’t that cool?😎🤗🤩 We can, of course, use the command jupyter notebook to execute Jupyter, but our task is to run it more like a service in the background and restart it whenever something wrong happens and it is killed by the OS🤗.

For this purpose, we will use Supervisor, which is a process control system for monitoring and controlling processes in UNIX-like OS. To install supervisor, you can run sudo apt install supervisor and that just check if the supervisor is running:

If we are able to see the prompt (as shown in image above), that means the supervisor has been installed and running successfully. Otherwise, you can start the service by yourself:

sudo service supervisor start

To ask the supervisor to monitor and control our Jupyter notebook, we need to first create a configuration file and place it in the /etc/supervisor/conf.d. Now create a new file by sudo touch jupyter.conf (in the conf.d directory) and add the following block:

After saving the configuration file to the directory /etc/supervisor/conf.d, we can ask the supervisor to read the configuration file and start the program:

sudo supervisorctl reread
sudo supervisorctl update

If the service is successfully configured and started, you can see something like the following when you execute the supervisorctl command without any parameters:

We should now be able to access your Jupyter notebook at the URL http://www.yourdomain.com:<port-no>

With this, we had completed this tutorial🥂🎉🥳.

What if I got stuck? — Debugging begins!!

What to do if we face any problem🤔😕, don’t worry, I’ll be covering the most common problem with their solutions

1. Unable to set password to Jupyter Notebook.

Solution: To set the password for the first time, you must know the token number. It’ll be either shown in terminal (before setting up supervisor) or in log file jupyter.err.log under /var/log/ directory.

2. Status: FATAL while running Jupyter from supervisor.

Solution: Check the log file jupyter.err.log under/var/log/ directory and you’ll be able to find out the error.

Stuck anywhere😨😢 or having a query? Feel free to drop me a 📧 Mail or ping me up on Telegram!