Introduction to TensorFlow — CPU vs GPU

Dear reader,

This article has been republished at Educaora and has also been open sourced. Unfortunately TensorFlow 2.0 changed the API so it is broken for later versions. Any help to make the tutorials up to date are greatly appreciated. I also recommend you looking into PyTorch.

In this tutorial we will do simple simple matrix multiplication in TensorFlow and compare the speed of the GPU to the CPU, the basis for why Deep Learning has become state-of-the art in recent years.

What is TensorFlow?

Graphs and Tensors



You see that the GPU (a GTX 1080 in my case) is much faster than the CPU (Intel i7). Back-propagation is almost exclusively used today when training neural networks, and it can be stated as a number of matrix multiplications (backward and forward pass). That’s why using GPU:s are so important for quickly training deep-learning models.

CPU time in green and GPU time in blue. The initial GPU delay at the first iteration is perhaps due to TensorFlow setting starting up stuff.

Next step

Studied Engineering Physics and in Machine Learning at Royal Institute of Technology in Stockholm. Also been living in Taiwan 學習中文 Interested in Deep Learning.