How to install CUDA & cuDNN
Setting up CUDA & cuDNN for Machine Learning can be an overwhelming process. In this guide, I will walk you through the steps to install CUDA and cuDNN on your system, ensuring your machine is correctly set up for machine learning tasks.
Whether you’re an experienced developer or a beginner in the world of machine learning, this tutorial will help you get started with ease.
System Configuration:
- Operating System: Ubuntu 22.4
- GPU: GeForce RTX 3090
- ML Framework: PyTorch
Install NVIDIA drivers
Update & Upgrade
Firstly, check for updates on your OS.
sudo apt update && sudo apt upgrade
Remove previous NVIDIA installation
Uninstall the previous NVIDIA and CUDA installation to avoid messing up the new installation.
sudo apt-get remove --purge -y '*nvidia*' '*cuda*' 'libcudnn*' 'libnccl*' '*cudnn*' '*nccl*'
sudo apt-get autoremove --purge -y
sudo apt-get clean
Check if there are any remaining packages or files.
dpkg -l | grep -E 'nvidia|cuda|cudnn|nccl'
Detecting and Managing Drivers on Ubuntu
ubuntu-drivers devices
We will install the NVIDIA driver tagged recommended — Which indicates which drivers are recommended for each piece of hardware based on compatibility and performance.
Install Ubuntu drivers
sudo ubuntu-drivers autoinstall
Install NVIDIA drivers
My recommended version is 555, change “XYZ” in the following command to your recommended driver.
sudo apt install nvidia-driver-XZY
Reboot the system for these changes to take effect.
reboot
Check Installation
After reboot verify that the following command works:
nvidia-smi
Install CUDA drivers
Update & Upgrade
Again, check for updates on your OS.
sudo apt update && sudo apt upgrade
Install CUDA Toolkit
At the moment of writing this text, the newest CUDA version supported by Pytorch is 12.1.
You can find older versions in the CUDA Toolkit Archive. In my case, I will be continuing with CUDA Toolkit 12.1.1 (April 2023).
You will need to select your operating system (Linux in my case). Afterwards, you will be prompted to select the Architecture. If you are not sure what is the Architecture of your PC you can use the command below (in my case the Architecture is x86_64).
uname -m
Next, we need to select the distribution and version of our operating system, in my case Ubuntu 22.04. Lastly, I am using the deb (local) installer type.
These are the commands for installing CUDA Toolkit 12.1:
wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/x86_64/cuda-ubuntu2204.pin
sudo mv cuda-ubuntu2204.pin /etc/apt/preferences.d/cuda-repository-pin-600
wget https://developer.download.nvidia.com/compute/cuda/12.1.1/local_installers/cuda-repo-ubuntu2204-12-1-local_12.1.1-530.30.02-1_amd64.deb
sudo dpkg -i cuda-repo-ubuntu2204-12-1-local_12.1.1-530.30.02-1_amd64.deb
sudo cp /var/cuda-repo-ubuntu2204-12-1-local/cuda-*-keyring.gpg /usr/share/keyrings/
sudo apt-get update
sudo apt-get -y install cuda-12-1
⚠️ NOTE: The last command differs from the one on the CUDA installation page. I added “-12–1” to specify the CUDA version to install.
Check CUDA install
nvcc --version
If you are not getting the CUDA version as output, do the following:
- Ensure that CUDA 12.1 is installed in the correct directory, typically
/usr/local/cuda-12.1
. - Create a symlink to the CUDA directory to make it easier to reference.
sudo ln -s /usr/local/cuda-12.1 /usr/local/cuda
- Add the CUDA paths to your
.bashrc
file to ensure they are set up every time you open a terminal.
echo 'export PATH=/usr/local/cuda-12.1/bin:$PATH' >> ~/.bashrc
- Apply the changes made to the
.bashrc
source ~/.bashrc
Install cuDNN
Download the cuDNN .deb file
Firstly, to match appropriate cuDNN with our CUDA and Driver versions we can use the table provided by NVIDIA. In my case, the newest 9.2 cuDNN version is appropriate.
You can download the cuDNN file here, for which you will need an Nvidia account. Same as for CUDA, you will have to select OS, architecture, distribution, version, and installer type.
In my case, to Download Installer for Linux Ubuntu 22.04 x86_64 instructions are as follows:
wget https://developer.download.nvidia.com/compute/cudnn/9.2.0/local_installers/cudnn-local-repo-ubuntu2204-9.2.0_1.0-1_amd64.deb
sudo dpkg -i cudnn-local-repo-ubuntu2204-9.2.0_1.0-1_amd64.deb
sudo cp /var/cudnn-local-repo-ubuntu2204-9.2.0/cudnn-*-keyring.gpg /usr/share/keyrings/
sudo apt-get update
sudo apt-get -y install cudnn-cuda-12
Test CUDA with PyTorch
I assume that you already have Python installed on your machine, so this tutorial would not be covering that part.
⚠️ NOTE: The latest PyTorch requires Python 3.8 or later.
Create a Directory
mkdir test_cuda
cd test_cuda
Create and Activate a Virtual Environment
python -m venv .venv
source .venv/bin/activate
Install PyTorch
pip3 install torch torchvision torchaudio
Execute Python test script
import torch
# Check PyTorch version
print(f"PyTorch version: {torch.__version__}")
# Check if CUDA device is recognised
print(f"CUDA device is available: {torch.cuda.is_available()}")
assert torch.cuda.is_available(), ValueError("CUDA device is not recognised.")
device = "cuda"
a = torch.rand(3, 1).to(device)
b = torch.rand(1, 3).to(device)
c = a @ b
print(f"Variable shape: {c.shape}")
print(f"Variable devie: {c.device}")
Conclusion
If you carefully followed these instructions, you have successfully installed CUDA and cuDNN on your Ubuntu 22.4 system. Your NVIDIA GPU is now ready for deep learning tasks with PyTorch.
For more details, visit my GitHub repository linked below. If this guide helped you give it a 👏, share it, and give it a ⭐️ on GitHub.