Setting Up TensorFlow 2 GPU on Windows in 2021

Machine Learning is complicated — but shouldn’t installation be the easiest part?

Photo by Nana Dua from Pexels

I have a kind of love-hate relationship with TensorFlow but I would like to think over the years this relationship has been growing and improving with each new release. TensorFlow is an insanely powerful platform for machine learning which is why I really do love what they are doing. The hate part of my relationship came in when I tried to install my local TensorFlow V1 environment on Windows with GPU support for the first time and it took me over a day to get it somewhat working. I understand, machine learning is complicated — but shouldn’t installation be the easiest part?

In this guide, I’m going to try to make this process as painless as possible so you can have a loving relationship with TensorFlow right from the start!

TensorFlow 2 (CPU Only)
∘ Anaconda for Python Installation
∘ TensorFlow 2 in Anaconda Installation
TensorFlow 2 with GPU Support
∘ Check Compatibility
∘ Nvidia CUDA Toolkit Installation
∘ Nvidia cuDNN Setup
∘ Nvidia GPU Driver Update
∘ Environment Variables Setup
∘ Verify Full TensorFlow Installation
What’s Next?

TensorFlow 2 (CPU Only)

Anaconda for Python Installation:

I highly recommend installing Anaconda if you have not already installed it as it's really great at managing packages and creating virtual environments which are both important for the next step for installing Tensorflow.

1. Download Anaconda Individual Edition
2. Run the executable installer and navigate through steps ← defaults are often good enough. You can “Add Anaconda3 to my PATH environment variables” if you do not already have conflicting distributions of python already installed on your PC.

TensorFlow 2 in Anaconda Installation:

1. Open an Anaconda Prompt (Anaconda3) Terminal ← search “Anaconda Prompt” in the Start menu and window will pop up and read (base) C:\Users\name>
2. Run: conda create -n tf pip python=3.9 ← tells Anaconda to make a new virtual environment called "tf" that will be using Python version 3.9
3. Run: activate tf ← tells Anaconda to run inside virtual environment “tf” and will read (tf) C:\Users\name>
4. Run: pip install --ignore-installed --upgrade tensorflow==2.5.0 ← installs TensorFlow version 2.5.0 package which includes GPU support as well now (check for newest stable versions)

TensorFlow 2 with GPU Support

This guide, up until this point, has been very similar to others found across the internet and has been pretty straightforward. We now have a Tensorflow environment install which is good enough to use if you only want to train your machine learning models with your CPU. For some, this may be okay, but machine learning is known to work best when training using GPUs as it is much faster than just using your CPU.

Due to this, we will be going through the process to enable GPU Support for TensorFlow 2 in the section below.

Check Compatibility:

TensorFlow GPU Compatibility List ← TensorFlow has specific software compatibility requirements so it is important to make sure you are downloading the correct versions in the next steps.

For TF 2.5.0, CUDA 11.2 and cuDNN 8.1 have been verified by TensorFlow.

Nvidia CUDA Toolkit Installation:

1. Download CUDA Toolkit 11.2 ← if you need a different version of CUDA for a different TensorFlow version, find it in the CUDA Toolkit Archive
2. Run the executable installer and navigate through steps ← defaults are often good enough

Nvidia cuDNN Setup:

1. Create an account for the Nvidia Developer Progam Membership ← needed in order to download cuDNN
2. Sign in and download cuDNN 8.1 for Windows ← if you need a different version of cuDNN for a different TensorFlow version, find it in the cuDNN Archive
3. Extract C:\Users\name\Downloads\cudnn-11.2-windows-x64-v8.1.0.77.zip\cuda folder into C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.2 folder

Nvidia GPU Driver Update:

1. Open Nvidia’s Geforce Experience ← you can just type in “Geforce Experience” in the Start Search Bar and it will pop up as an app
2. Click on the “Drivers” Tab
3. Under “Available” a driver might be ready to install, if not, click on “Check For Updates” and see if there are any driver updates ready after that

Environment Variables Setup:

1. Open “System Properties” ← search “Edit system environment variables” from the Start menu
2. Click on “Environment Variables…” ← under the “Advanced” tab at the bottom right corner
3. Find and edit Path variable ← under the “System variables” section
4. Add the following five new Path values:

C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.2\bin
C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.2\libnvvp
C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.2\include
C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.2\cuda\bin
C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.2\extras\CUPTI\lib64

Verify Full TensorFlow Installation:

1. Open a new Anaconda Prompt (Anaconda3) Terminal
2. Run: python -c “import tensorflow as tf;print(tf.reduce_sum(tf.random.normal([1000, 1000])))”
3. Correct output should have “Successfully opened dynamic library” and “Created TensorFlow device” flags and will look like this:

(tf) C:\Users\name>python -c "import tensorflow as tf;print(tf.reduce_sum(tf.random.normal([1000, 1000])))"
2021-07-02 20:19:44.622280: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library cudart64_110.dll
2021-07-02 20:19:48.037812: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library nvcuda.dll
2021-07-02 20:19:48.158677: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1733] Found device 0 with properties:
pciBusID: 0000:2d:00.0 name: NVIDIA GeForce RTX 2070 SUPER computeCapability: 7.5
coreClock: 1.785GHz coreCount: 40 deviceMemorySize: 8.00GiB deviceMemoryBandwidth: 417.29GiB/s
2021-07-02 20:19:48.158984: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library cudart64_110.dll
2021-07-02 20:19:48.632718: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library cublas64_11.dll
2021-07-02 20:19:48.632851: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library cublasLt64_11.dll
2021-07-02 20:19:48.939031: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library cufft64_10.dll
2021-07-02 20:19:48.971598: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library curand64_10.dll
2021-07-02 20:19:49.202579: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library cusolver64_11.dll
2021-07-02 20:19:49.416235: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library cusparse64_11.dll
2021-07-02 20:19:50.335902: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library cudnn64_8.dll
2021-07-02 20:19:50.336366: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1871] Adding visible gpu devices: 0
2021-07-02 20:19:50.338899: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX AVX2
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags. 
2021-07-02 20:19:50.341514: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1733] Found device 0 with properties:
pciBusID: 0000:2d:00.0 name: NVIDIA GeForce RTX 2070 SUPER computeCapability: 7.5
coreClock: 1.785GHz coreCount: 40 deviceMemorySize: 8.00GiB deviceMemoryBandwidth: 417.29GiB/s
2021-07-02 20:19:50.341677: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1871] Adding visible gpu devices: 0
2021-07-02 20:19:51.176159: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1258] Device interconnect StreamExecutor with strength 1 edge matrix:
2021-07-02 20:19:51.176333: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1264]      0
2021-07-02 20:19:51.177492: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1277] 0:   N
2021-07-02 20:19:51.179734: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1418] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 5973 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2070 SUPER, pci bus id: 0000:2d:00.0, compute capability: 7.5)
tf.Tensor(1009.2107, shape=(), dtype=float32)

… and we’re DONE! That is all that you need to do to set up TensorFlow 2! Whenever you want to use this environment, just open a new Anaconda Prompt (Anaconda3) Terminal and run: activate tf

What’s Next?

In my next articles, I will be going through the installation and setup for creating your own custom object detection models using TensorFlow’s Object Detection API. I will be going through all the code fixes and tweaks I needed to figure out along the way so you don't have to!

I highly recommend looking at TensorFlow 2 Object Detection API Tutorials as it has some more in-depth explanations and is one of the best resources I’ve found so far. In this series of articles, I will be going through similar steps to their tutorial, but helping to fix a few key pieces of code along the way.