How to Set Up an AWS EC2 Instance For GPU Work

No, this is not a guide on how to set one up for mining (I would be very surprised if Amazon doesn’t shut you down if they hint your mining crypto). But there are definitely many more uses for a GPU — such as NLP, AI Art, and benchmarking. In this guide I’ll walk through how to set up an EC-2 Instance to run an NVIDIA GPU. I’d recommend your read through this guide before getting started, as you’ll have to reach out to AWS at some point.

Sorry if I sound salty in this article. Currently dealing with massive burnout + an epic migraine, so I’m not operating at my most optimal. But, without further ado, let’s get started!

Get an AWS Account

This step is probably a “well duh.” But do it if you haven’t done it already.

Unfortunately, AWS still does want your ID to sell to shady 3rd party companies —Er — I mean…in order to keep your account safe, so there’s no way around it. Just give them the info. It’ll be easier for you if you do and don’t try anything privacy-conscious— I mean…shady, like provide an alias. Once that’s done, and your account is active, you’re ready to get rolling on setting up an EC-2 instance!

Get Started with an EC-2 Instance

In the AWS panel, click on “Services,” then look for EC2. If you’ve used it recently it will obviously be listed under Recently Visited.

You’ll see a screen like below:

If this is your first time, click on “Launch instance,” and “Launch instance.”

On this next page you’ll want to pay careful attention to a few settings.

First is the AMI. I’d suggest using Ubuntu, unless you’re more familiar with the other operating systems. Ensure that the AMI you selected has GPU in the name.

Next, notice Instance type. IT’S IMPORTANT YOU GET THIS PART RIGHT! Take a look at this page and look at the chart under the section “Available Drivers by instance type.”

Instance type must start with one of these (notice how I have p2.xlarge selected). If you select something else (like m2.xlarge ) your GPU will technically exist on the system, but won’t be exposed. Also notice the number of vCPUs (in this case 4).

Oh, one more thing — notice your Region in the top right. Make note of it for later.

So, to recap, make note of:

• The type of EC2 type (either P or G)
• Your region (e.g. “Central”)
• The number of vCPUs you need (e.g. 4)

With that squared away, let’s continue.

Create a security key. Don’t mess with security group (unless you want to spend 5 hours understanding why your box isn’t connected to the internet). And you’ll probably want to increase the storage size (something like 500GiB ought to be enough, but scale according to you needs).

Finally, hit Launch Instance. And this will fail with

Instance launch failed
You have requested more vCPU capacity than your current vCPU limit of 0 allows for the instance bucket that the specified instance type belongs to. Please visit http://aws.amazon.com/contact-us/ec2-request to request an adjustment to this limit.

In this case, if you didn’t get a failure and your instance started, you probably did something wrong (or you already requested a vCPU increase). Double-check your ec2 type (make sure it starts with “P” or “G”) and AMI (make sure it contains a “GPU” description.)

At this point we’ll need to reach out to Amazon support.

Reach out to Amazon Support to Have your vCPU Increased

See why I suggested you read the entire article beforehand?

So reach out using the URL suggested (http://aws.amazon.com/contact-us/ec2-request).

And don’t worry if you have the free tier…asking for a vCPU increase is available for free users.

Also note it’s totally fine to ask for more than you need (of course, don’t go crazy and ask for 9,000).

A message like this would suffice:

Hey, AWS,
I’m working on a GPU project for [reason], and I’d like to increase my vCPU count to [count] for region [region].
Please increase it as soon as you can.
Thank you.

It will take a couple of days for them to work on it, and you may need to be more specific about why the GPU EC2 is needed.

If you’re really on top of things, I’d suggest you reach out to AWS right after you create your account so you don’t encounter the vCPU error. But ensure you have all the right information.

Now, start your EC2 Instance Again

Once your vCPU limit has increased, follow the steps again to create an EC2 instance. Pay attention to the region. it needs to the same on you requested to have increased.

Once in instance is launched, click on the link it gives you (should look something like i-29da23c34). You’ll then be taken to your EC2 instances.

Click on your running instance. Wait until Status Check doesn’t say initialized anymore. Then click Connect.

You’ll then be presented with various ways to connect. I’d suggest SSH, because these next steps will take place in the terminal.

Use the “SSH” command mentioned. It should look something like this:

$ ssh -i key.pem ubuntu@some-host.aws.com

If you run the nvidia-smi command you’ll find that it will fail with the following error:

NVIDIA-SMI has failed because it couldn't communicate with the NVIDIA driver. Make sure that the latest NVIDIA driver is installed and running.

Installing the NVIDIA Driver

This is probably the easiest part of this article, and is mostly straightforward.

You know that page I mentioned earlier listing the ec2 instance types? Go to Install options > Install Public Drivers. Take a look at the chart.

Make note of the instance you used and the Product Series you need to download. In my example (P2), I need to download the K-Series.

Head on over to NVIDIA’s download page and download the right driver. Follow the instructions for installing the driver in Ubuntu.

And, That’s It!

There you have it! Bob’s your uncle! The cat has come home!

However you put it, that’s the guide on how to get a GPU EC-2 instance up on AWS.

Hopefully you found this helpful and that it would save hours of headaches. Soon you’ll be on your way to GPU-ing in no time!

📢 Comment below: What’s been your experience spinning up EC-2? Was it easy?

👉‍‍ Share this article with 3 of your friends or colleagues. On Twitter, LinkedIn or Mastodon. Be sure to tag me in the post. Helps me know if my content is still relevant.

💓 Subscribe to DamnGoodTech on Ko-Fi for as little as $7/mo. Get articles 3 days early and get a shout-out on each article! That’s like hiring a team lead for your software organization for way less than minimum wage. 🙏🏻 Special thanks to James N, Lucy R, and Steve O for your support.

💼 Hire me as a Tech Creation Lead on your team. I have over 10 years development experience and would love to help your team reach their full potential. Head on over to https://damngood.tech/pages/schedule.html to schedule a free consultation.