Nanopore GPU basecalling using GUPPY on UBUNTU 18.04 and nvidia docker v2 with a RTX 2080

Oxford nanopore released the GPU version of the guppy basecaller. Coincidentally, we acquired a small footprint gaming rig with the newest RTX 2080 on Ubuntu 18.04. The only snag is that guppy is currently compatible with Ubuntu 16. Lucky for us, we have the latest release of nvidia docker to the rescue. Nvidia docker allows us to create containers that uses our GPU.

Our strategy is to setup nvidia docker on the rig, pull a docker image with cuda and CNN installed on Ubuntu 16, followed by installation of guppy via its deb file and benchmark GPU basecalling.

First we start with installing the drivers for the new RTX 2080 which we did using the instructions in this article: https://medium.com/@avinchintha/how-to-install-nvidia-drivers-and-cuda-10-0-for-rtx-2080-ti-gpu-on-ubuntu-16-04-18-04-ce32e4edf1c0

Once we verified that driver and GPU are on board using the last steps in the above article,

Running nvidia-smi to verify that we have the 410 drivers installed and running

We followed up with these excellent steps to install nvidia-docker 2.0 on the rig:

https://medium.com/@sh.tsang/docker-tutorial-5-nvidia-docker-2-0-installation-in-ubuntu-18-04-cb80f17cac65

Now, we have a platform to load docker containers with the required Ubuntu 16 that uses our local RTX 2080. To ensure compatibility, we proceed to pull the image from nvidia/cuda.

We picked CUDA 9.0 with cudnn7 image for our benchmarking

docker pull nvidia/cuda:9.0-cudnn7-devel-ubuntu16.04 

To be quick, we fire up the container to install guppy. In the future, we can update the dockerfile and build the image. For now, we will load the container and commit the changes after installation of guppy.

docker run --runtime=nvidia --name <container_name> -i -t -v <local_dir_of_fast5:/media/>  nvidia/cuda:9.0-cudnn7-devel-ubuntu16.04 /bin/bash

Next, we wget for ourselves the guppy deb file:

wget -q https://mirror.oxfordnanoportal.com/software/analysis/ont_guppy_2.2.2-1~xenial_amd64.deb

followed by installation, ignoring the nvidia drivers and cuda libraries :

dpkg -i --ignore-depends=nvidia-384,libcuda1-384 ont_guppy_2.2.2-2~xenial_amd64.deb

After installation, time to take our guppy out for a joyride. When we first fired up guppy, it started basecalling using CPUs.

guppy_basecaller -i <fast5_dir> -o <output_folder> -c dna_r9.4.1_450bps

By default, CPU mode is ON.

To kickstart GPU basecalling we had to add -x flag specifying cuda:0

guppy_basecaller -i <fast5_dir> -o <output_folder> -c dna_r9.4.1_450bps -x "cuda:0"

And here we see out finished basecalling test run output

So we achieved 1308.81 kev/s or in other units: ~1308,000 events called per sec as indicated in the screenshot above. To put it on a more relevant scale, we manage to basecall 2.6 gigabases in real time of ~100 mins. In rrwick article, the 1070 GTX is going at ~700 kb/sec , so in the similar units, the RTX is giving us ~1308kb/sec. Overall, things are looking speedy. Time to clean up the bench i.e committing changes we have made to the container for future use.

docker container ls

Get the container ID from the output, which we then proceed to commit changes:

docker commit <container_ID> <Your_repo_name>:cuda_9.0-cudnn7-devel-ubuntu16.04

Time to get down to doing the biology. Enjoy!