Using TensorFlow on Windows 10 with Nvidia RTX 3000 series GPUs
A step by step guide to building CUDA/cuDNN from source to use for GPU accelerated deep learning
Overview
Installing TensorFlow/CUDA/cuDNN for use with accelerating hardware like a GPU can be non-trivial, especially for novice users on a windows machine. The below describes how to build the CUDA/cuDNN packages from source so that TensorFlow tasks can be accelerated with a Nvidia RTX 30XX GPU.
This article assumes that
- you are using an IDE like IntelliJ, PyCharm, or Spyder/Anaconda
- you have used pip/conda to install packages
- you have already physically installed your GPU
We will cover
- briefly, checking system/hardware requirements
- installing TensorFlow using pip
- in depth, building CUDA/cuDNN from source so that TensorFlow can be used with GPU support
Before beginning it is worthwhile to note that TensorFlow and associated dependencies require very specific driver/package versions, as such it is recommended to use the exact versions specified in this article.
System and Hardware Requirements
Before starting please ensure that python version > 3.8 is downloaded correctly and installed. Check python version by typing the below into the command line (open terminal by typing “cmd” in windows search bar + enter).
C:\Users\...> python --version
Python 3.8.xPlease also ensure that the GPU is correctly installed and Nvidia drivers are updated. You can view the driver version by opening Nvidia Control Panel (either from desktop or programs list) and looking at the version number above the GPU name. Driver version must be > 455.
If using an IDE like pycharm or spyder it would be useful (but not necessary) to create a virtual environment where TensorFlow will be used. Some resources for setting up a virtual environment in IntelliJ can be found here, and in Spyder/Anaconda here
Installing TensorFlow
With the recent TensorFlow v. 2.4.0 release a simple pip install (or conda install if you are in a conda environment) command in your IDE terminal (as opposed to the cmd line, in which case you would need to do an system wide install with “pip3” instead of pip) will install an out of the box version of TensorFlow with GPU support capability; to utilize the full functionality of the GPU enabled TensorFlow however you must install the dependencies. You must specify tensorflow==2.4.0 ; previous versions of TensorFlow do not support the Ampere architecture found in 3000 series GPUs. TensorFlow v < 2.4.0 are built for the Turing architecture found on 2000 series and lower GPUs.
C:\...\generic_project_folder>pip install tensorflow==2.4.0Installing Dependencies
TensorFlow requires the below dependencies before being installed. The installation process is described in the subsequent sections.
Microsoft Visual Studio C++ compiler
The CUDA toolkit requires the MS Visual Studio C++ compiler. The easiest way to install this is by installing Microsoft Visual Studio Code (an entire IDE) which can be downloaded here. Follow onscreen prompts.
CUDA toolkit 11.0
Follow the link. Select Windows > x86_64 Architecture > Version 10 > exe [network]. Using the local installer type has all of the components embedded into it and is useful if you need to do multiple installs however it can be a lengthy process depending on connection/hardware, for this reason it is recommended to install via the network executable.
Right click on “Download” and select “open link in new tab” or double click to initiate the download.
Once downloaded extract the folder to a Temp location.
Follow the on-screen prompts, accept all agreements, install, and finish.
Once the installer is finished you can exit the window.
cuDNN 8.0.4
Follow the link and select the appropriate download library (shown below). This will take you to the Nvidia Developer page. If you are not already a member of the Nvidia Developer program you will need to join and accept terms, after which you may download the file. Be sure to download cuDNN v8.0.4 for CUDA 11.0.
Once downloaded, unzip the contents of the folder to a file storage location that you have easy access to like your desktop or downloads folder.
You now have all of the components to build the CUDA/cuDNN package from source so that TensorFlow can be used with GPU acceleration.
Building CUDA/cuDNN
This next task requires the use of two main directories, you’ll be transferring 3 sets of files from the downloaded cuDNN library to the NVIDIA GPU Computing Toolkit program files. The roots of these directories can be found at the below, it should be noted that the cuDNN root is wherever you unzipped it in the last step.
# cuDNN root
C:\Users\...\Downloads\cudnn-11.0-windows-x64-v8.0.4.30.zip# NVIDIA GPU Computing Toolkit root
C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.0
Set 1
Navigate to the below directories and copy “cudnn.lib” from 1 to 2.
# 1. cuDNN
\...\cudnn-11.0-windows-x64-v8.0.4.30.zip\cuda\lib\x64# 2. NVIDIA GPU Computing Toolkit
\...\NVIDIA GPU Computing Toolkit\CUDA\v11.0\lib\x64
Set 2
Navigate to the below directories and copy “cudnn.h” from 1 to 2.
# 1. cuDNN
\...\cudnn-11.0-windows-x64-v8.0.4.30.zip\cuda\include# 2. NVIDIA GPU Computing Toolkit
\...\NVIDIA GPU Computing Toolkit\CUDA\v11.0\include
Set 3
Navigate to the below directories and copy all files from 1 to 2.
# 1. cuDNN
\...\cudnn-11.0-windows-x64-v8.0.4.30.zip\cuda\bin# 2. NVIDIA GPU Computing Toolkit
\...\NVIDIA GPU Computing Toolkit\CUDA\v11.0\include
Note: If you copy just the cudnn64_8.dll file from 1 to 2 you will be able to compile your models but they will crash on training.
To ensure all the packages are properly built first restart your computer.
After restarting, search “edit the system and environment variables” in the windows bar and click enter. Click “Environment Variables…”
Double click on the “Path” System Variable. Ensure the below are displayed. Click OK to exit all windows.
Checking Configuration
Now that TensorFlow and all it’s dependencies have been installed and built you need to check that TensorFlow and python can “see” the GPU.
- Open your IDE in whatever project you installed TensorFlow previously
- to check if the GPU is available type the following into your IDE terminal or script and run; your output should look something like the below
>>>import tensorflow as tf
>>>tf.config.list_physical_devices("GPU")
[PhysicalDevice(name='/physical_device:GPU:0', device_type='GPU')]4. Test your GPU with a convolutional neural network! Use the below script to ensure that TensorFlow is being used to train. The performance of the below model will not be good, it is just meant to ensure that the TensorFlow model can compile and train.
from tensorflow.keras import datasets, layers, models
import matplotlib.pyplot as plt
(train_images, train_labels), (test_images, test_labels) = datasets.cifar10.load_data()
train_images, test_images = train_images / 255.0, test_images / 255.0
model = models.Sequential()
model.add(layers.Conv2D(32, (3, 3), activation='relu', input_shape=(32, 32, 3)))
model.add(layers.MaxPooling2D((2, 2)))
model.add(layers.Flatten())
model.add(layers.Dense(64, activation='relu'))
model.add(layers.Dense(10))
model.compile(optimizer='Adam', loss=tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True))
history = model.fit(train_images, train_labels, batch_size=10, epochs=100)If all is well your IDE should output something like the below.
Congratulations on installing TensorFlow, building CUDA/cuDNN, and utilizing your GPU hardware to accelerate deep learning performance!
