JupyterLab on AWS EC2
Detailed steps to set up and run JupyterLab on AWS EC2 machine.
This tutorial documents the steps required to serve JupyterLab from AWS EC2. This is particularly useful for running programs that require continuous network access, like web-scraping. I personally use this setup very often for testing and debugging intraday trading algorithms.
JupyterLab is a web-based Python IDE (Interactive Development Environment). EC2 is a service offered by Amazon that provides Virtual Machines on the cloud.
Note:- For sake of completeness and to keep it short, common practices like SSH key setup, virtual environment creation etc. are not covered in this tutorial.
Spin Up an EC2 instance and Connect
- Create an AWS account (or login to an existing one) from here.
- Go to EC2 under Services then click on Instances.
3. Click on Launch instances.
4. Choose Ubuntu Server 20.04 (64-bit). (or any other Linux distribution)
5. Choose t2.micro (or pick a different instance type based on project requirements) and Go to Security Group Configuration.
6. Add an Inbound rule with the following parameters for serving JupyterLab.
╔═════════════════╦══════════╦════════════╦═══════════════════════╗
║ Type ║ Protocol ║ Port Range ║ Source ║
╠═════════════════╬══════════╬════════════╬═══════════════════════╣
║ Custom TCP Rule ║ TCP ║ 8888 ║ Custom 0.0.0.0/0,::/0 ║
╚═════════════════╩══════════╩════════════╩═══════════════════════╝Click on Review and Launch.
7. Select Proceed without a key pair then click on Launch Instances.
The instance is now getting ready.
8. Go to the running instances section and connect to it when it is up and running.
Install pip and JupyterLab
Ubuntu Server 20.04 LTS includes Python 3. Run the following commands to install pip3. pip is a package manager written in python. (At this point one can choose to install conda and create an environment)
sudo apt update
sudo apt install python3-pipInstall JupyterLab using pip3.
pip3 install jupyterlabNote:- It is recommended to close and reconnect to the server again after installing JupyterLab, because certain commands may not work unless you restart the terminal.
Protect JupyterLab with password
Since the IDE will be accessible publicly, it is better to secure it with a password. Prepare password hash using IPython.
ipythonfrom IPython.lib import passwd
passwd()quit()
To avoid sending the password unencrypted, let’s use SSL. Create a self-signed certificate and make the current user the file owner.
mkdir certs
cd certssudo openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout mycert.pem -out mycert.pemsudo chown $USER:$USER mycert.pemcd ..
Generate a jupyter server configuration file and update it with the password hash and certificate path.
jupyter server --generate-config
nano .jupyter/jupyter_server_config.pyUpdate the config file as below.
c = get_config()
c.ServerApp.certfile = u'/home/ubuntu/certs/mycert.pem'
c.ServerApp.ip = '*'
c.ServerApp.password = u'sha1:e261d4617d68:3d1a307b7bcc279a6ac72b4dd5fb5a067c52b3c8'
c.ServerApp.port = 8888
Replace the password with your password hash and update the SSL certificate path if different.
Press ctrl-xPress yPress Enter
Run JupyterLab
To keep the jupyter server process running even when the SSH connection drops, the process should be initiated inside a terminal multiplexer. For this purpose let’s use a simple multiplexer GNU Screen. Most of the Linux distros ships with GNU Screen preinstalled.
Simply type screen to start a new session.
screenNow start the jupyter process in no-browser mode.
jupyter-lab --no-browser
The IDE will be accessible at https://<public-ip>:8888. Instance’s public IP address can be found on the instance summary page. The browser might raise a warning about the self-signed certificate, which can be ignored.
Press ctrl-a ctrl-d to detach from the screen session
Run screen -r to reattach to the session.
Leave a comment if I made any mistakes. Hope this helps!
