How to install CUDA & cuDNN

Setting up CUDA & cuDNN for Machine Learning can be an overwhelming process. In this guide, I will walk you through the steps to install CUDA and cuDNN on your system, ensuring your machine is correctly set up for machine learning tasks.

Whether you’re an experienced developer or a beginner in the world of machine learning, this tutorial will help you get started with ease.

System Configuration:

• Operating System: Ubuntu 22.4
• GPU: GeForce RTX 3090
• ML Framework: PyTorch

Install NVIDIA drivers

Update & Upgrade

Firstly, check for updates on your OS.

sudo apt update && sudo apt upgrade

Remove previous NVIDIA installation

Uninstall the previous NVIDIA and CUDA installation to avoid messing up the new installation.

sudo apt-get remove --purge -y '*nvidia*' '*cuda*' 'libcudnn*' 'libnccl*' '*cudnn*' '*nccl*'

sudo apt-get autoremove --purge -y

sudo apt-get clean

Check if there are any remaining packages or files.

dpkg -l | grep -E 'nvidia|cuda|cudnn|nccl'

Detecting and Managing Drivers on Ubuntu

ubuntu-drivers devices

We will install the NVIDIA driver tagged recommended — Which indicates which drivers are recommended for each piece of hardware based on compatibility and performance.

Install Ubuntu drivers

sudo ubuntu-drivers autoinstall

Install NVIDIA drivers

My recommended version is 555, change “XYZ” in the following command to your recommended driver.

sudo apt install nvidia-driver-XZY

Reboot the system for these changes to take effect.

reboot

Check Installation

After reboot verify that the following command works:

nvidia-smi

Install CUDA drivers

Update & Upgrade

Again, check for updates on your OS.

sudo apt update && sudo apt upgrade

Install CUDA Toolkit

At the moment of writing this text, the newest CUDA version supported by Pytorch is 12.1.

You can find older versions in the CUDA Toolkit Archive. In my case, I will be continuing with CUDA Toolkit 12.1.1 (April 2023).

You will need to select your operating system (Linux in my case). Afterwards, you will be prompted to select the Architecture. If you are not sure what is the Architecture of your PC you can use the command below (in my case the Architecture is x86_64).

uname -m

Next, we need to select the distribution and version of our operating system, in my case Ubuntu 22.04. Lastly, I am using the deb (local) installer type.

These are the commands for installing CUDA Toolkit 12.1:

wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/x86_64/cuda-ubuntu2204.pin
sudo mv cuda-ubuntu2204.pin /etc/apt/preferences.d/cuda-repository-pin-600
wget https://developer.download.nvidia.com/compute/cuda/12.1.1/local_installers/cuda-repo-ubuntu2204-12-1-local_12.1.1-530.30.02-1_amd64.deb
sudo dpkg -i cuda-repo-ubuntu2204-12-1-local_12.1.1-530.30.02-1_amd64.deb
sudo cp /var/cuda-repo-ubuntu2204-12-1-local/cuda-*-keyring.gpg /usr/share/keyrings/
sudo apt-get update
sudo apt-get -y install cuda-12-1

⚠️ NOTE: The last command differs from the one on the CUDA installation page. I added “-12–1” to specify the CUDA version to install.

Check CUDA install

nvcc --version

If you are not getting the CUDA version as output, do the following:

• Ensure that CUDA 12.1 is installed in the correct directory, typically /usr/local/cuda-12.1.
• Create a symlink to the CUDA directory to make it easier to reference.

sudo ln -s /usr/local/cuda-12.1 /usr/local/cuda

• Add the CUDA paths to your .bashrc file to ensure they are set up every time you open a terminal.

echo 'export PATH=/usr/local/cuda-12.1/bin:$PATH' >> ~/.bashrc

• Apply the changes made to the .bashrc

source ~/.bashrc

Install cuDNN

Download the cuDNN .deb file

Firstly, to match appropriate cuDNN with our CUDA and Driver versions we can use the table provided by NVIDIA. In my case, the newest 9.2 cuDNN version is appropriate.

You can download the cuDNN file here, for which you will need an Nvidia account. Same as for CUDA, you will have to select OS, architecture, distribution, version, and installer type.

In my case, to Download Installer for Linux Ubuntu 22.04 x86_64 instructions are as follows:

wget https://developer.download.nvidia.com/compute/cudnn/9.2.0/local_installers/cudnn-local-repo-ubuntu2204-9.2.0_1.0-1_amd64.deb
sudo dpkg -i cudnn-local-repo-ubuntu2204-9.2.0_1.0-1_amd64.deb
sudo cp /var/cudnn-local-repo-ubuntu2204-9.2.0/cudnn-*-keyring.gpg /usr/share/keyrings/
sudo apt-get update
sudo apt-get -y install cudnn-cuda-12

Test CUDA with PyTorch

I assume that you already have Python installed on your machine, so this tutorial would not be covering that part.

⚠️ NOTE: The latest PyTorch requires Python 3.8 or later.

Create a Directory

mkdir test_cuda
cd test_cuda

Create and Activate a Virtual Environment

python -m venv .venv

source .venv/bin/activate

Install PyTorch

pip3 install torch torchvision torchaudio

Execute Python test script

import torch


# Check PyTorch version
print(f"PyTorch version: {torch.__version__}")
# Check if CUDA device is recognised
print(f"CUDA device is available: {torch.cuda.is_available()}")
assert torch.cuda.is_available(), ValueError("CUDA device is not recognised.")

device = "cuda"
a = torch.rand(3, 1).to(device)
b = torch.rand(1, 3).to(device)

c = a @ b

print(f"Variable shape: {c.shape}")
print(f"Variable devie: {c.device}")

Conclusion

If you carefully followed these instructions, you have successfully installed CUDA and cuDNN on your Ubuntu 22.4 system. Your NVIDIA GPU is now ready for deep learning tasks with PyTorch.

For more details, visit my GitHub repository linked below. If this guide helped you give it a 👏, share it, and give it a ⭐️ on GitHub.

GitHub - milistu/cuda-cudnn-install.md: How to install CUDA & cuDNN.