AWS (Amazon Web Services) has become a common name in the industry these days and if you are working with Python, chances are that you are already familiar with Jupyter (previously iPython) Notebooks. Although, if you are not familiar with Jupyter and you work with Python, then you are definitely missing on a very handy tool.
The reason for deploying Jupyter on AWS is to access and work with it from almost anywhere, just using a browser. So, let’s begin…
2. Create a new EC2 Instance with Ubuntu. If you are not familiar with how to create an EC2 instance, you can check out this video for reference.
3. The important thing to remember while creating the instance is to assign the security group settings as shown below
4. Launch your instance and ssh into it to perform the following operations
- First of all we are going to use Anaconda Python Distribution for installing all the required Python libraries. This is a free distribution and we are going to use the Linux version of it. Remember to verify the latest version of the distribution from the site.
$ wget https://repo.anaconda.com/archive/Anaconda3-2018.12-Linux-x86_64.sh
- Next we will bash to run this .sh file. You need to accept the license terms and set the installation directory for the anaconda distribution. I use the default one, which is “/home/ubuntu/anaconda3/”. Also, it asks you to add the default path of anaconda python to your .bashrc profile. You can accept it or add it manually later.
$ bash Anaconda3-2018.12-Linux-x86_64.sh
- Now you need to check, which python version you are using, just to confirm if Anaconda Python was installed and set correctly or not. You can do this by using
$ which python
- This will list the python your system is currently using. If it does not mentions the one from “…/anaconda3/…” folder, then you can use the following command to re-load your .bashrc, so as to set the correct python
$ source .bashrc
- Open the iPython terminal to get an encrypted password so as to use it for logging into our iPython Notebook Server. Remember to copy and save the output of this command, which will be an encrypted password, something like “sha1…”
$ In :from IPython.lib import passwd
$ In :passwd()
- and exit out of ipython terminal.
- Now we’re going to create the configuration profile for our Jupyter Notebook server
$ jupyter notebook --generate-config
- (Optional step) You could also create a self-signed certificate for accessing the Notebooks over HTTPS
$ mkdir certs
$ cd certs
$ sudo openssl req -x509 -nodes -days 365 -newkey rsa:1024 -keyout mycert.key -out mycert.pem
- You will be prompted with some questions, please answer them to the best of your knowledge as some of these are required to successfully create the certificate.
- It’s time to change the config settings of Jupyter
$ cd ~/.jupyter/
$ nano jupyter_notebook_config.py
- You will see a long list of configuration settings. You can go through each one of them and uncomment them as required. For simplicity, it’s easier to just add the following settings to the top of the file and leave the rest commented as it is.
c = get_config()
c.IPKernelApp.pylab = 'inline' # for consistent plotting support
c.NotebookApp.certfile = u'/home/ubuntu/certs.pem' # location of your certificate file
c.NotebookApp.keyfile = u'/home/ubuntu/certs/mycert.key # location of your certificate key
c.NotebookApp.ip = '*'
c.NotebookApp.open_browser = False # so that jupyter does not opens a browser by default
c.NotebookApp.password = u'sha1:68c123ab4523...' # the encrypted passoword generated above
c.NotebookApp.port = 8888 # always a good idea to use a known fixed port
- Almost done with all the setup. Now it’s time to start our Jupyter notebook server. For this, let’s first create a new folder to store all the notebooks
$ cd ~
$ mkdir Notebooks
$ cd Notebooks
- and now START the notebook server
$ jupyter notebook
5. And that’s all folks... Now you can access your Notebooks — navigate to the DNS name, or Public IP of your instance, along with the port number. (By default, the browser adds “http” to the url. Please remember to update it to “https” if you’re using the self-signed certificate)
You will be asked by your browser to trust the certificate. Refer to screenshots below:
6. Login using the password you specified on the iPython terminal and you are good to go.
7. Note that if you close ssh access to your instance, your notebook server will not work. To keep it working, even when you close the ssh access to the server, you can use the following command
$ nohup jupyter notebook
This will put your server process as no-hangup (nohup) and will keep it running even if you close the ssh access to the instance
8. Later, if you decide you want to stop this process, you have to find the process ID (PID) of this process. You can do so by navigating to your notebooks folder and using the command
$ lsof nohup.out
which will list the PID of the nohup process which is running(if any). Then you can use the kill command to kill this process and stop your Jupyter notebook server.
$ kill -9 <PID>
replace <PID> with the ID of the process you get from the lsof command.
That’s it!! Now you have your Jupyter Notebooks on AWS EC2 instance and can work and use them from almost anywhere around the World.
Originally published at blog.impiyush.com on February 16, 2015.