What is the Tensor Processing Unit (TPU)?
The Future of Hardware that will power Deep Learning
Before we get to TPUs, let us first talk about the GPUs.
One could say it’s the best thing Sony ever thought of because it is practically the most important thing for gamers, deep learning engineers, video producers, etc. You could say, The Graphical Processing Unit or GPU is a special type of computer chip that does rapid mathematical calculations, fundamentally for rendering images. The applications have diversified since then.
A Core Processing Unit (CPU) is the part of a computer that performs calculations, actions, and runs programs. With time, the complexity of the calculations performed increased and rather than normal computer calculations, the task of rendering became more desired and prominent. The GPU aids the computer in performing visual and graphical tasks. As of today, it will be really difficult for the common computer user to name a computer task that doesn’t involve visuals! We’ll now be seeing how the GPU and CPU are quite different!
How is the GPU fundamentally different
- While the CPU focuses on multiple tasks at once the GPU is mainly used for a single specific task. One could say the CPU is the brain while the GPU is the muscle.
- A CPU is designed with a lesser number of cores optimized for sequential serial processing whereas the GPU is made up of thousands of smaller and more efficient cores for parallel processing.
- For tasks such as generating a single password hash or for traditional kernel operations such as writing files to a disk, the CPU will actually perform better. GPUs aren’t designed for quick simple tasks. They are designed with a larger bandwidth in mind.
- In short, one can say a CPU is more like a swiss army knife which can be used to overcome daily problems but a GPU is more like a surgical knife which is only used in special cases!
TPU — Introduction
In 2015, Google came out with technology which made the GPU look bad. Enter TPU, the Tensor Processing Unit which actually reduces a month’s training time to a matter of days. TPU is an AI accelerator application-specific integrated circuit which was made specifically for neural nets. With the TPU, it is possible for the normal person to work on the state of the art as well.
The TPU is only (and only) used for neural network calculations and they do it at amazing speeds and small physical footprints. Now we will be heading into its details and application.
TPU vs GPU
- The first main difference is that TPU works for neural networks whereas GPUs are meant for graphics and image rendering. You can use a GPU to run your PUBG at 4k but a TPU sticks on to neural networks.
- While you can buy GPUs with the system you buy, TPUs are only accessible in the cloud (for now)!
- Interestingly, TPUs are a lot cheaper in comparison to GPUs.
- There is a major difference in speed, the tasks which are viable with TPUs makes the GPU look completely useless.
- One major factor to consider is TPU works as long as there are no custom TensorFlow models applied which is a major constraint not applied to GPUs.
Advantages of TPUs
- Complex neural networks are always time-consuming. TPUs can greatly speed up that process by minimizing the time to accuracy.
- While GPUs focus on matrix computations, TPUs can also work on linear algebra computations.
- Since it’s hosted on the cloud, anyone can access it all over the world. Recently Google also developed the architecture to run on the edge.
- Google also has preset models where the model is already adjusted to the TPU. They are also completely adjusted with Google Cloud Platform which can greatly help with analytics.
How to use it
Here is a link to a colab notebook that contains a model trained with TPUs and how exactly one can access them from the cloud. This notebook will be a little tough to interpret at first but a good 10-minute read will not only help you how to access TPUs but also give insight on how to actually structure code while executing machine learning techniques.
Conclusion
While it comes to real speed, quantum computing makes all this sound very hilarious but then again that is a completely different story to tackle. What one should take away from the article is that as of now we need ALL three processors to get work done. The variety of tasks one has cannot be done by just one. TPUs are definitely a huge boost to the community as neural networks are literally everywhere. To see a detailed analysis of the comparison of the three please refer to this paper. If anyone really wants to get into this field we highly recommend going through an intermediate course on computer architecture to really get a feel of how the TPU actually works.
TPU Tutorials for further learning and experimentation:
Editorial Note-
This article was conceptualized by Aditya Vivek Thota and written by Soumya Kundu of The Research Nest.
Stay tuned for more diverse research trends and insights from across the world in science and technology, with a prime focus on artificial intelligence!
