Deep Learning Setup: ECS GPU Task On Ubuntu (Part 1)

Michael Loewenstein
Jan 3 · 4 min read

In order to run a GPU based task in ECS, we need to create our own EC2 instance as Fargate still doesn’t support GPUs. That shouldn’t be too hard with the ECS GPU optimized AMIs.

However, sometimes that cannot be done as teams are already using their favorite AMI setups such as an Ubuntu CIS optimized AMI or any other flavor. This means they need to install and configure the setup from scratch.

In this set of 4 articles, we’ll review the installation and configuration process of an ECS task with GPU required resources over an Ubuntu 18.04 OS.

Part 1: The NVIDIA driver

Part 2: The ECS agent

Part 3: The NVIDIA-Docker run time

Part 4: GPU configuration on ECS Agent

NVIDIA Driver & ToolKit

Although installing and configuring the NVIDIA tool kit & drivers is very well documented, I had a few gotchas so I found it valuable to document the required steps on my own. However, I recommend going over the formal documentation which is much more detailed, and also the steps might change according to one's needs & individual requirements/preference.


Verify You Have a CUDA-Capable GPU To verify that your GPU is CUDA-capable:

$ lspci | grep -i nvidiaIf your graphics card is from NVIDIA and it is listed in, your GPU is CUDA-capable.

Verify You Have a Supported Version of Linux

$ uname -m && cat /etc/*releaseYou should see output similar to the following, modified for your particular system: x86_64 Red Hat Enterprise Linux Workstation release 6.0 (Santiago)

Verify the System Has GCC Installed

$ gcc --version

I didn’t have GCC installed and got the following error: “Command ‘GCC’ not found, but can be installed with sudo apt install gcc”, which is easily solved by running:

sudo apt install gcc

Verify the System has the Correct Kernel Headers and Development Packages Installed

$ uname -rThis is the version of the kernel headers and development packages that must be installed prior to installing the CUDA Drivers.

The kernel headers and development packages for the currently running kernel can be installed with:

$ sudo apt-get install linux-headers-$(uname -r)This is for Ubuntu only, for a different OS please see the formal NVIDIA documentation.

Toolkit Installation

We can either install the NVIDIA drivers only or install the NVIDIA toolkit, which is a set of tools plus the drivers. I prefer to install the toolkit since it will help us verify the installation succeed later on.

Download the NVIDIA CUDA Toolkit

The NVIDIA CUDA Toolkit is available at Select the operation system, distribution, and version; then select your preferred installation type, I selected here deb(local):

And run the commands, for Linux Ubuntu 18.04 x86_64:

$ wget
$ sudo mv /etc/apt/preferences.d/cuda-repository-pin-600
$ wget
$ sudo dpkg -i cuda-repo-ubuntu1804-11-2-local_11.2.0-460.27.04-1_amd64.deb
$ sudo apt-key add /var/cuda-repo-ubuntu1804-11-2-local/
$ sudo apt-get update
$ sudo apt-get -y install cuda

Post Installation

Environment Setup — Add Path:

$ export PATH=/usr/local/cuda-11.2/bin${PATH:+:${PATH}}

Setup the Persistence Daemon:

$ sudo /usr/bin/nvidia-persistenced --verbose

Verifying the installation

Now that everything is in place, let’s verify the drivers are installed successfully:

$ cat /proc/driver/nvidia/versionThe output should be similar to the following:
NVRM version: NVIDIA UNIX x86_64 Kernel Module 460.27.04 Fri Dec 11 23:35:05 UTC 2020
GCC version: gcc version 7.5.0 (Ubuntu 7.5.0-3ubuntu1~18.04)

And Verify The Toolkit Installation:

$ nvcc -VThe output should be similar to the following:nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2020 NVIDIA Corporation
Built on Mon_Nov_30_19:08:53_PST_2020
Cuda compilation tools, release 11.2, V11.2.67
Build cuda_11.2.r11.2/compiler.29373293_0

And As A Final Test, We’ll Run A Sample GPU Program; Notice This Might Take A Long Time To Execute.

mkdir tmp && tmp && cd NVIDIA_CUDA-11.2_Samples && make && cd 1_Utilities/deviceQuery && ./deviceQueryThe output should be similar to the following image:

Now, we have our NVIDIA driver installed and functioning. In the next article, we’ll install and configure the ECS agent and adjust it to work with the driver we just installed.


Everything connected with Tech & Code

Medium is an open platform where 170 million readers come to find insightful and dynamic thinking. Here, expert and undiscovered voices alike dive into the heart of any topic and bring new ideas to the surface. Learn more

Follow the writers, publications, and topics that matter to you, and you’ll see them on your homepage and in your inbox. Explore

If you have a story to tell, knowledge to share, or a perspective to offer — welcome home. It’s easy and free to post your thinking on any topic. Write on Medium

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store