Deep Learning on a Free TPU
Deep learning is an artificial intelligence technology that is surprisingly successful.
It’s so successful that it’s now dominating in many fields, including computer vision, speech analysis, natural language processing, and robotics. Its success is surprising, because it’s basically a rebranding of an old technology called “Neural Network”. Neural networks were invented in the early 1940s. For decades, it was just a beautiful idea — no serious applications. Deep neural nets, that is, networks with many hidden layers, had particularly disappointing results; back in the old days, it was often argued that a shallow neural net is better than a deep one. Things took a sharp turn in 2012, when the rebranded deep learning won a major computer vision competition. After that, within 5 years, it invaded many research fields simultaneously, and became dominating in all of them.
So, born in World War II, dormant for half a century, power-awakened in 2012, and dominating since then.
Yet, deep learning is not particularly hard to do. The fundamental concepts of deep learning are easy to understand, without complicated math or theory. Basically, you are given a toolbox with neurons, layers, connections, and your job is to assemble them into a network, train the network with your data, and watch it improves gradually — just as watching a baby learning to speak. If everything is right, the neural network starts to solve your problem after enough training, and the results can be so good that even human experts can’t reach the same level of accuracy.
What’s more, your deep learning toolbox is being built by international tech giants, such as Google, Facebook, Amazon and Microsoft. These companies spent a huge amount of money building these tools, and they give them out for free. They give you not just basic tools, but entire systems, such as a complete, end-to-end machine translation system with state-of-the-art accuracy. And these are the same tools and systems they use themselves.
On top of that, Google let you use to their hardware for free. That hardware is the Tensorflow Processing Unit (TPU), which is even stronger than the state-of-the-art GPU, Tesla V100, as of April 2019.
Is there still an excuse not to build up your deep learning skills? You may say that Tensorflow is rather difficult to use, and TPUs especially complicated. This tutorial series aims to eliminate this last excuse, with a library called Fenwicks that provides one-liner functions for most common purposes.
So, time to get started.
