Setting up AWS EC2 for Running Jupyter Notebook on GPU in the Cloud

I set up an Amazon Elastic Compute Cloud (EC2) instance on AWS to run Jupyter Notebook on GPU as part of my study with Udacity Deep Learning Nano Degree. I’m sharing with you my notes to help with your setup in case you decide to choose AWS for running Jupyter Notebook on GPU in the cloud. Take a look at my post about Floydhub here and next up I will write about setup on Google Cloud.

Image for post
Image for post

High level summary

  1. Log in to Amazon AWS, then navigate to the EC2 console
  • Change region to US East (N. Virginia)
  • Increase service limits for p2.xlarge instance type (by default it’s 0)

2. Create an instance and configure instance image and type

3. Set up security group to configure port 8888 in order to access the Jupyter notebook

4. Create key pair for authentication

5. Launch instance from EC2console, and then connect to instance via command line, with the .pem file and instance ip address

7. Configure Jupyter Notebook settings

8. Download source code and install any necessary tools such as Keras, Tensorflow etc

9. Start Jupyter Notebook, do your work, then remember to stop the instance when done.

Detailed step by step

Log in to AWS

  • Change region to US East (N. Virginia)
  • View EC2 Service Limits — click on EC2/EC2 Dashboard/Limits or directly go to this link
  • Find p2.xlarge instance type and click on “Request limit increase” and set it to 1 or a number higher than 1.
Image for post
Image for post
  • Wait for approval and you will get an email notification on this.

Configure instance — select image

  • Click on the dropdown next to the “Launch Instance” button
  • Select “Launch instance”. (Note even though it says “Launch instance”, you can’t really launch it until you set the instance image and instance type etc. and finish all the configuration steps below. )
Image for post
Image for post
  • Click on AWS Marketplace
  • Search for Deep Learning AMI with Source Code (CUDA 8, Ubuntu) and click on select.

Configure instance — select instance type

  • Filter by “GPU compute”
  • Select type = p2.xlarge
Image for post
Image for post
  • Click on “Review and Launch” then you will jump to step 7. Review

Next we need to set up a security group.

Configure instance — set up Security Group

So add a new rule as follows:

  • Type = Custom TCP
  • Protocol=TCP
  • Port Range=8888
  • Source=Anywhere
Image for post
Image for post

Then click on Review and Launch which will take you to “step 7. Review” again

Launch instance-create key pair for authentication

  • You will be prompted to select or create a new key pair
  • Select “create new key pair” if you don’t have one already, and give it a name
Image for post
Image for post
  • Click on Download Key Pair button then a .pem file will be downloaded
  • Click on “Launch Instance” and now your instance is actually finally launching!
  • (Optional) click on billing alerts to set up email alerts

Login to the Instance

ssh -i [key-name].pem ubuntu@[]

If you get error like this:

Permissions 0644 for ‘key-name.pem’ are too open.
It is required that your private key files are NOT accessible by others.
This private key will be ignored.
Load key “[key-name].pem”: bad permissions
ubuntu@ Permission denied (publickey)

Then update the permission of the .pem as follows:

chmod 400 key-name.pem

Try connect again:

ssh -i [key-name].pem ubuntu@[]

And hopefully now you are connected to your AWS EC2 instance.

Configure Jupyter notebook settings

While connected to your ASW instance, at terminal type

jupyter notebook --generate-config

Then you will see -

Writing default config to: /home/ubuntu/.jupyter/

Use the instance

  • Install and or upgrade any required tools: Numpy, Keras & tensorflow etc.
  • Start Jupyter Notebook from command line
jupyter notebook --ip= --no-browser 
  • Copy/pate the url to browser but replace the before :8888 with your instance ip address (Ipv4 Public IP)

Stop the instance

Click on Actions/Instance State/Stop to stop the instance.

Image for post
Image for post

Note: if you don’t need the instance at all in the future, remember to terminate it in order to avoid being charged for the storage (EBS volume).

Start the instance again

Log in to the AWS EC2 console, click on instances, select your instance then

  • Click on Actions/Instance State/Start
  • Wait for the state to go from initialize to “2/2 checks”, then note the instance ip address
  • At terminal use command to login ssh -i [key-name].pem ubuntu@[]
  • Navigate to your source code
  • Launch jupyter notebook-
jupyter notebook --ip= --no-browser
  • Copy/pate the url to browser but replace the before :8888 with your instance ip address (Ipv4 Public IP)


Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch

Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore

Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store