Installing TensorFlow with GPU on Windows 10

In an earlier article I showed how to test your Linux system to see if you have a GPU that supports TensorFlow, with the promise that I’d next do Windows and MacOS. So, in this article, I will show how you can test a Windows system for a supported GPU, and if you have one, how you can then install and configure the required drivers, before getting a TensorFlow nightly build and ensuring that it works.

But before you go any further, you’ll also need Python and pip, which don’t typically come with Windows machines.

Getting Python and pip

There’s quite a few ways to install Python and pip for Windows, but I found the easiest to be to go to the python site and download the executible installer: https://www.python.org/downloads/windows/

Once you’ve downloaded it, launch it and be sure to select the custom option for install. This takes you into a configuration dialog where you can select pip as an option to install.

I’d recommend you just choose them all.

Once you’ve finished installing, you can open a command prompt and type ‘python’ to see which version you’re using. As you can see from the above I downloaded 3.6.4, and my python command matches this.

You can exit the Python interpreter with

exit()

And then test your pip install with

pip -V

You should see something like this:

You’re now good to go with the Python runtime and pip installer for Windows. You’ll use these to install TensorFlow, but first, let’s check on your GPU to see if it is supported.

Checking your Windows GPU

From your command prompt you can launch Device Manager with this command:

control /name Microsoft.DeviceManager

Then look for the Display Adapter setting, open it and read the name of your adapter. You should see something like this:

As you can see, my system has a GTX 980 Ti. Take note of yours, and visit NVidia’s site of GPUs that support CUDA here. If your card is on the list (my GTX 980 Ti is), then you know that TensorFlow can support GPU operations on your machine, but before you can install and run TensorFlow, you’ll need to install the CUDA drivers for your machine and the CUDNN updates for it.

Installing the CUDA drivers

At the time of writing, the TensorFlow nightly builds support CUDA 9.0, and the release builds support 8.0. If you visit the cuda downloads site here, you’ll notice that it will take you to the latest version (right now it’s 9.1), so be sure to download the right version of the driver by selecting the Legacy link at the bottom of the downloads screen:

Of course as CUDA gets updated, and as TensorFlow gets updated, these versions will change. If you have the incorrect version, TensorFlow will warn you (I’ll show you where, later in this article), and you’ll have to go back to this site to get the correct version. Download the drivers for your system and install them using defaults. It takes a little while, and a couple of reboots, so while you’re waiting, check out a couple of episodes of Coffee With A Googler :)

You’ll also need a version of CuDNN that matches your version of CUDA before you can run TensorFlow, but, I’ve found it easier not to install it yet — you want to ensure that you have the right CUDA first, so, while it might seem counterintuitive, you should install TensorFlow first, and try to get it to run. One of two things will happen:

• It will fail on the CUDA drivers, so you’ll need to repeat this step
• It will fail on the CuDNN drivers, so you know your CUDA drivers are good, and you will get the version of CuDNN that you need.

So let’s install TensorFlow next:

Installing TensorFlow nightly build

I’m assuming here you’re using TensorFlow with GPU, so, to install it, from a command prompt, simply type:

pip install tf-nightly-gpu

(Replace with tf-nightly if you don’t want the GPU version)

Then, once it’s completed, in your command prompt window type

python

to open the python editor, and within it type

import tensorflow as tf

If you are using the GPU version you should get an error. That’s good.

If you have the wrong CUDA drivers, it will likely show that cudart64_XX.dll fails, where ‘XX’ is a version number. At time of writing it is cudart_90.dll, which tells you that you need version 9.0 of the CUDA drivers. Check this version carefully, and repeat the previous step (downloading and installing the drivers) until this error goes away. Be sure to close the command prompt window, and open a new one to test the new installation so your paths are up to date.

When you have the right CUDA drivers, but the wrong or missing CuDNN drivers, you’ll see an error saying something like cudnn64_X.dll is missing, where X is a version number. Take note of that, and you’ll use it in the next step. Exit python and close the command prompt window before continuing. In my case it told me that cudnn64_7.dll was missing, so I know I need version 7.

Installing the CuDNN libraries

The CuDNN libraries are an update to CUDA for Deep Neural Nets, and used by TensorFlow to accelerate deep learning on NVidia GPUs. You can download them from here: https://developer.nvidia.com/cudnn. You’ll have to sign up for an NVidia developer account first, but it’s pretty quick and costs nothing. Once you’ve signed in you’ll see a variety of CUDNN downloads. Here’s where you’ll have to match to the version of CUDA that you downloaded in the previous section. So, for example, I used CUDA 9.0, so made sure I used a CuDNN that matches both this and the required version you saw in the last step (in my case version 7), so I chose cuDNN v7.0.5 for CUDA 9.0

This will download a ZIP file with several folders, each containing the CuDNN files (one DLL, one header and one library). Find your CUDA installation (should be at something like: C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v9.0)

You’ll see the directories from the ZIP file are also in this directory — i.e. there’s a bin, and include, a lib etc. Copy the files from the zip to the relevant directory. So, for example, drag cudnn64_7.dll to C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v9.0\bin, and do the same for the others. You can see more here: http://docs.nvidia.com/deeplearning/sdk/cudnn-install/index.html#install-windows

Once you’re done, re-open a command prompt window, and test TensorFlow again!

Testing TensorFlow

If all goes to plan, you can now open a command prompt window, and type ‘python’ to open the Python interpreter. Then type:

import tensorflow as tf

And if you have the right versions of the libraries, you should see no error, like this:

If you see errors in cudart64_xx.dll or cudnn64_x.dll, go back to the previous steps and make sure you’ve downloaded and installed the correct versions of the drivers that cause these errors. It takes a bit of trial and error I’m afraid :(

Then, you can check your TensorFlow version by typing

print(tf.__version__)

Note that there are 2 underscores before and after ‘version’. TensorFlow will output it’s version #, as shown here:

You now have a full TensorFlow runtime environment with GPU support on Windows! Congratulations. Time to do some deep learning…