As Google announced their new Pixel phone, the Pixel 6, this month, a very interesting announcement was seen. Google actually announced that these new phones would be launching with a TPU chip on board. Now, this may be quite confusing for most people since very few really know what a TPU is and hence, what this hype is about.
So in this article, I will answer the following questions;
- What is a TPU?
- Where is it used?
- What does this mean for the mobile industry?
What is a Tensor Processing Unit (TPU)?
We can answer that by answering a simpler question first.
What is a tensor?
We know what a CPU or GPU is, we use them every day for normal tasks i.e. office work, gaming, or even Machine Learning practice. But what is this tensor that needs processing?
Simply speaking, a tensor is a data container. It can hold data values in n-dimensions.
As the illustration above suggests, a tensor is just an array of numbers that represent valuable data.
**For Machine Learning Engineers, it is noteworthy that a tensor is actually much complicated than a simple array. It stores much more valuable information such as dot product values and descriptions of transformations between the data points**
This data structure was developed by Google mainly for AI training.
With that out of the way let's straight-up jump to Tensor Processing Units. As I have already mentioned, tensors are majorly used in AI training since — as any Machine Learning engineer here would know — a major chunk of them consist of matrix addition and multiplications (Dot product). All the processing units (CPU, GPU, TPU)can do these operations easily but the time complexity of these operations is where the three devices differ. The best analogy is explained by folks at Google itself.
“A CPU is like printing out letters whereas a GPU is printing out entire sentences, but TPUs are printing entire pages”
TPUs do these matrix calculations so fast because, unlike GPUs and CPUs which have to constantly access memory to pick out data or store it, TPUs have dedicated systolic arrays to perform scalar, vector, and matrix operations. Since in AI algorithms all the mathematics is pre-defined, the TPUs are able to load huge chunks of data into these arrays and perform huge calculations simultaneously which means they have to access the memory much less often hence preventing something called the Von Neumann Bottleneck, a problem CPUs have seen since forever.
Hence this dedicated hardware outperforms GPUs and the removal of several useless components means these are built on a much smaller form factor and consume much MUCH less power than a GPU.
So what’s a TPU doing on a phone?
Now back to the main question, what does it have to do with cell phones?.
Well my friends you do not realize how much artificial intelligence your phone uses. When you read “ AI camera ” on your phone back, it actually means something.
When your camera detects the scene you are capturing and tweaks the colors accordingly or when you speak to your virtual assistant and it recognizes your words, your phone is utilizing artificial intelligence.
Before now, these calculations were being done on the CPU of the phone. As we’ve discussed above, a CPU has its limitations in terms of calculations. Placing a TPU on board means that pixel 6 can utilize it to perform much faster machine learning calculations and the AI models can be run much faster.
According to Google, this will greatly impact the camera quality of the phone. Google phones already take amazing pictures all thanks to the amazing AI algorithms they have built but one domain where they lacked was the video quality which is considered average as compared to what its competitors have to offer.
This inclusion of the TPU means that the same algorithms can now be utilized, frame-by-frame, on the video the phone captures and improve its quality. Another advantage that Google highlighted in their teaser video was that now many hefty machine learning models can run offline without having to connect to Google's data center. This also means that you have increased data security.
The Future?
Many are coining this move by Google as a “ game-changer”. For now, these Tensors will only be found on Google phones, primarily because the TPU itself was architected by Google back in 2015 but it is only a matter of time before other manufactures hop on the bandwagon and start manufacturing such Tensor chips, or perhaps Google starts supplying to other manufacturers and we’ll be seeing other manufacturers releasing a phone with TPUs.
