TinyML Packs a Punch
When you hear the terms artificial intelligence (AI) and machine learning (ML), you may think these technologies require humongous compute power to perform their magic, but tiny ML (“tiny machine learning”) refers to deploying these technologies on small, low-power processing engines like microcontrollers.
It’s not so long ago that technologies like artificial intelligence and machine learning were of academic interest only. As far back as the 1840s, Charles Babbage’s assistant Augusta Ada Lovelace recognized that the data processed by computers could represent things other than numerical values, and she even postulated that it might be possible for computers to perform tasks like composing music. Of course, Ada was thinking in terms of Babbage’s mechanical engines — she had no idea of computers in today’s sense of the word.
The founding event of what we would now consider to be artificial intelligence was the Dartmouth Workshop, which took place in 1956. Unfortunately, progress was slow and hard-fought for, and — as I noted in What the FAQ are AI, ANNs, ML, DL, and DNNs? — artificial intelligence remained largely in the realm of academia until circa the early 2010s.
Some time ago, the American research, advisory, and information technology firm Gartner came up with a graphical presentation called the Hype Cycle, which is used to graphically represent the maturity, adoption, and social application of specific technologies.
The Hype Cycle is divided into five main sections: Technology Trigger, Peak of Inflated Expectations, Trough of Disillusionment, Slope of Enlightenment, and Plateau of productivity. Some technologies can take years or even decades to progress all the way through the cycle; others may progress faster; and some take everyone by surprise.
In the case of the 2014 Hype Cycle, for example, machine learning wasn’t even a “blip on the radar,” as it were. Just one year later, in the 2015 Hype Cycle, machine learning had already crested the Peak of Inflated expectations.
Now, in 2020, applications based on artificial intelligence and machine learning are appearing all over the place, such as embedded vision, embedded speech, object detection and recognition, face recognition (including determination of age, gender, and emotion), handwriting recognition, robot control, medical diagnosis, and… the list goes on.
To be honest, one is hard-pressed these days to find application areas and market segments that are not using artificial intelligence and machine learning in one form or another.
Before we proceed, let’s briefly remind ourselves of the high-level process involved in creating an AI/ML application. We start by defining the network using an architecture like Caffe or Google’s TensorFlow. This architectural definition includes the number of artificial neurons per layer, the number of neural layers, the way in which the layers are connected, and so forth. At this stage, we’re typically talking about a network that uses 32-bit floating-point values to represent its coefficients (“weights”), although higher or lower floating-point precisions can be used.
The network is trained using a humongous database. In the case of an image processing application, for example, the training database may involve tens of millions of tagged images. Once the network has been trained, it is passed through a converter, which translates it into a fixed-point representation suitable for deployment on the target platform: MPU, MCU, GPU, FPGA, or a specialized AI device (see also What the FAQ are CPUs, MPUs, MCUs, and GPUs?).
Many companies have made tremendous strides in this area, such as the guys and gals at CEVA with their CEVA Deep Neural Network (CDNN), which targets CEVA DSP cores. There is also the chaps and chapesses at Lattice Semiconductor with their Lattice sensAI Stack, which includes everything you need to evaluate, develop, and deploy FPGA-based AI/ML applications.
In addition to converting the original floating-point network used for training into a fixed-point equivalent suitable for deployment as an inferencing engine, the interesting thing about these tools is that they allow the user to perform intelligent quantization, in which each layer has only the precision required to perform its function to the specified level of accuracy. As a result, it’s possible to deploy some AI/ML applications on 8-bit processors.
So, what is TinyML? Well, according to the tinyML Summit: “Tiny machine learning is broadly defined as a fast growing field of machine learning technologies and applications including hardware (dedicated integrated circuits), algorithms, and software capable of performing on-device sensor (vision, audio, IMU, biomedical, etc.) data analytics at extremely low power, typically in the mW range and below, and hence enabling a variety of always-on use-cases and targeting battery operated devices.”
There are always going to be AI/ML applications that demand the humongous compute capability afforded by the cloud, but there’s an increasing move to perform AL/ML on the edge, and TinyML takes this to the extreme edge of the internet in the form of the microcontrollers that interface with the real world.
According to the Why TinyML is a Giant Opportunity article on VentureBeat.com, “There are 250 billion microcontrollers in the world today.” Furthermore, IC Insights forecasts that microcontroller sales will grow to 38.2 billion units a year by 2020.
Even though the concept of TinyML is so new that it doesn’t even have a Wikipedia entry at the time of this writing, companies like Ericsson are already offering TinyML as-a-Service.
I’m sure we will be talking about TinyML more in the weeks and months to come. For the moment, let’s close by noting that the next TinyML Summit will take place 12–13 February 2020 in San Jose, California. Also, that the book TinyML: Machine Learning with TensorFlow Lite on Arduino and Ultra-Low-Power Microcontrollers by Pete Warden and Daniel Situnayake came out in December 31, 2019, which is less than two weeks ago at the time of this writing.
I’m thinking of ordering this book and developing my own TinyML application. How about you? Are you ready to dip your toes in the TinyML waters?
