Installing Jupyter on a Cloud Machine

Python is a great language for scientific computing and machine learning. The tools that are readily available for you are extremely useful for scientific computing. Powered with Jupyter, you could use Python in an interactive way, writing and evaluating code in the same place called Jupyter notebook. See the figure below for an example of using numpy and matplotlib in a Jupyter notebook to plot the sine function.

Using numpy and matplotlib in a Jupyter notebook to plot the sin function.

Jupyter can be installed easily with pip (among multiple other methods):

pip install jupyterlab

Once installed locally, you could run Jupyter notebook with the following command, which will run Jupyter server on a local port and open Jupyter notebook in the browser:

jupyter notebook

Jupyter notebook homepage

You can take it from there, create notes, etc. However, I personally find it inconvenient to have Jupyter notebook running off my machine, for two main reasons:

1. I have to start it every time I want to use it. I also need to keep a terminal window open for it, which can be a little bit distracting.
2. If I am doing scientific computing, executions will likely consume CPU power, which could slow down the machine and reduce battery life.

As such, I prefer to run it on a could machine and leave it running so I could use it anytime I want, just like I use any 3rd party website. I wrote a simple Bash script do this, which runs Jupyter as a background process and leave it running until it is manually terminated. I thought I would share this script with others since it made my life easier:

This script checks whether Jupyter notebook is already running. If it is, print a message and do nothing. Otherwise, it creates a Jupyter notebook as a background process. Since this is a background process, the script redirects stderr and stdout to files so they can be inspected when needed. Three notes about the script:

1. The grep in the second line works for my machine only; you need to change it to wherever your Jupyter notebook lives. In my case, I am using pyenv for managing Python installations, so you can see that it lives under .pyenv folder. To find out where your Jupyter notebook lives, the easiest way is to run it and then execute ps aux | grep jupyter to find out the path.
2. The scripts automatically creates a sub-folder called notes and CDs into it before starting Jupyter notebook. This way, every note you create will be stored in that folder.
3. The script redirects stdout and stderr to files in the current directory, i.e. the directory where you were before executing the script above. As such, I created a folder called jupyter and put this script in it to isolate it from other stuff on my machine. This is how the jupyter folder looks like on my machine:

Now that you have a script on your could machine, you need to redirect a port on your local machine to the remote Jupyter notebook. This seems to be necessary for some reason, as I tried to try and open the remote Jupyter notebook from the browser directly, i.e. http://<serveraddress>:<jupyter port> but it didn’t work; it seems that Jupyter notebook only runs when the address is specifically http://localhost:<port>. To work around this, you can execute this command locally (note that I am using port 9001 instead of the default 8888 port used by Jupyter notebook if you don’t specify the --port argument):

ssh -N -f -L localhost:9001:localhost:9001 <serveraddress>

Now this overall might sound a little bit complicated. First, you ssh into a remote machine to run Jupyter notebook. Then you execute the command above. To make it easier, I created the following script locally, which executes the script above via ssh followed by the command above for binding the local port to the remote Jupyter notebook:

Notice that this scripts automatically open the browser for you to save you a few clicks and key strokes ;-) Finally, because I like to have my commands easily accessible anywhere in bash, without having to CD into where the script actually lives, and to reduce the number of characters I have to type :P, I created the following alias in my .bash_profile :

alias jup="~/BashTools/jupyter.sh"

That’s it, I now just open bash and type jup and it does all the above for me. It also stays in the background, so even if I restart my machine, I don’t have to write this command again. I only need to write it again if for some reason the Jupyter server on my cloud machine dies.

That’s it, go ahead and enjoy your Jupyter notebook :-)