Signal 1: ML’s growth trend has surpassed Moore’s law
In Tsinghua-Google AI Symposium in Beijing July 2018, Jeff Dean, Google AI Chief, and David Patterson, a Turing Award winner and computer architect, unravel Machine Learning(ML) System Architecture Blueprint as outlined in their co-published paper A New Golden Age in Computer Architecture: Empowering the Machine-Learning Revolution. Mr. Dean’s team claims that the growth trend in ML, measured by the number of the related arXiv papers, has already surpassed that of Moore’s law. This reiterates the advent of the world’s new decision-making system in which ML thrives with a robust platform on which expensive algorithms can run freely. ML is no longer a cool gadget for the R&D centers but has consolidated its role in the mission-critical value chain.
Decision-making using ML is as computationally expensive as it is powerful. Expertise and computing are expensive, yet there are eager entrepreneurs and academia motivated to tackle this problem. The symposium highlights that ML’s growth is still mired by few barriers, namely the lack of dedicated ML hardware as well as that of experts in the industry. Mr. Dean delineates the solution to the former with the following concepts: Training, Batch Size, Sparsity and Embeddings, Quantization and Distillation, Networks with Soft Memory, Learning to Learn (L2L). L2L is one of the examples of such efforts where automated training is done using accuracy as a reward signal, a model can learn to self-improve over time. For the latter, Data Science as a career sets new records with each academic census, and there are efforts to automate much of the ML project lifecycles.
