Getting Started with ML on Embedded Systems
Embedded Systems are really small and task specific computers. They are power-efficient and compact with a long operating time on small battery capacity.
Machine Learning on the other hand can be quite resource intensive, requires a lot of data and consequently involves a ton of computation. Increased computations directly leads to more power consumption. Further more, ML algorithms and models are not necessarily designed for being stored on small memory modules. One may come across several models, which are a mere 10–100 kB in size, but in Embedded Systems which have very small memory, usually between 16–64 kB, this is an overkill.
Just on the basic working features listed above, one might surmise that perhaps ML and Embedded Systems were never meant to operate on the same platform. The ML models can run on a cloud server and the Embedded device can be a node which sends data to the cloud. It is a simple solution and not surprisingly, it is how things are done in a majority of IoT applications.
The Problem with Cloud Processing
While cloud processing can appear as the best method to combine the features of ML and Embedded Devices, it often is not.
For one, the increasing number of devices causes massive congestion of bandwidth. This increases latency and makes it impossible to implement real-time systems. 5G protocols will handle that, but it is extremely costly to setup and will not cover a vast majority of population.
That is not all. IoT devices rely on collecting your data and sending it to the cloud server for processing. Several aggregators collect and analyse your data. This is legal in the sense, they have your permission to do so, however you never really had a choice in the first place.
Most of the data that is collected might appear inconsequential to you. So what if they know how you walk, or which room you are in or which route you take on your way to work? But imagine this, they know your entire lifestyle. Where you go, where you live and who you meet. Most internet services rely on this data to give you better options when online and it is really good. But it still puts at stake your privacy and rights to control that data.
The Solution
Privacy and access to cloud servers are only some of the downsides to this picture. Embedded Devices on the other hand totally mitigate the issue of privacy since they are localized and their access to data can by physically controlled by you.
However, in the introductory section, I had mentioned how and why Embedded devices don’t have ML built-in into it. That limitation is changing. The embedded devices are not increasing their memory, rather developers are coming up with more innovative methods to modify algorithms to adhere to the limit set by embedded devices.
There are a few ways which can allow ML algorithms to run on embedded devices. The easiest method is to store the vectors and matrix values that ML models generate into header files and then write the code to use these values for either regression or classification. At the end of the day, all ML algorithms are mathematical operations and C\C++ runs math operations 10–50 times faster. The downside is that the coding will be complex as there is a restriction on the data structures that can be used. However, with efficient and smart use of pointer arrays, it can be pretty simple to handle such operations.
There are numerous libraries that allow users to convert their sklearn models into C header files. Some of them include:
- emlearn — Random Forest and Tree Classifiers (github project)
- micromlgen — SVM, SVC, PCA (github project)
- Microsoft ELL — Neural Network (github project)
- Tensorflow Lite
This method will however mean that to update the model, we will need to rebuild the hex file for the microcontroller and flash it with the new config each time that a change needs to be made.
Embedded Learning
If a user case requirement needs the model to be updated regularly without having to overwrite the FLASH memory program, there is one easy way to do this. The headers files which earlier held all data for ML classification need to be saved on the FLASH memory.
Now, developers will need to read from the FLASH memory to make ML predictions and to update the model, they will need to modify the values stored in the FLASH. It is however worth noting that the FLASH memory tends to be very small and can sustain a limited number of Read/Write cycles.
Note: To know more about FLASH programming and how it works on microcontrollers, you can refer to my article on the same topic: https://medium.com/theteammavericks/programming-flash-rom-in-stm32-f5b7d6dcba4f
Things You Need to Know before doing ML on Embedded Devices
Since we are focusing on Embedded Systems, programming language is a major determinant of how efficient our code may run on any module. There are several options available the most popular of which are:
- C\C++: This is the most popular and widely used suite of languages used for embedded development. If you look into their history, they were developed in the 1970’s, at a time where the computers were not much better than embedded devices today. It thus should come as no surprise that C\C++ remain the most relevant language in embedded development. Additionally, there is a huge volume of legacy code which can be referred to even on modern applications making C\C++ an ideal choice.
- Python: Python is currently the most popular scripting language and also the language of choice for many ML developers. It is easy to learn and easy to build and maintain across several OS platforms. However, it is not compact and neither is it fast. Two of the most important requirements of embedded systems. Thus, to counter this issue, an open source project called ‘MicroPython’ was developed. It is a recompiled Python with bare bones architecture so as to fit onto a microcontroller.
- Rust: Rust is another very interesting scripting language and is becoming increasingly popular in the developer community owing to it’s ease of learning and programming, while still allowing a much finer control over resources like memory. I personally do not have much experience working on rust, but if you are interested, you can refer to the links provided at the end of this article.
Along with the programming languages, there are some concepts that are a must know for embedded programming in general.
- The most important is interrupt handling. Mostly overlooked by early developers, working with interrupts is an essential skill needed while programming on embedded devices. Several MCU’s can be put into sleep mode and activated only when there is a sensor reading in the form of an interrupt. This will help reduce power consumption by as much as 30% which is a huge deal in embedded firmware development.
- Memory management of data structures and use of reusable variables is another core skill needed to make better firmware. A good understanding of pointer to handle multi-dimensional array will prove to be very useful in accessing memory locations. It will also reduce the complexity of the code and thus improve performance.
Useful Links
Embedded ML Libraries:
- emLearn: emlearn/emlearn: Machine Learning inference engine for Microcontrollers and Embedded devices
- microMLgen: micromlgen/svm.jinja at master · eloquentarduino/micromlgen
- MS Embedded Learning Library (ELL): The Embedded Learning Library — Embedded Learning Library (ELL)
- Tensorflow Lite: https://www.tensorflow.org/lite
C\C++:
- https://www.programiz.com/cpp-programming/pointers-arrays
- https://codeforwin.org/2017/12/access-two-dimensional-array-using-pointers-c-programming.html
- https://www.electronicshub.org/basics-of-embedded-c-program/
- https://www.edureka.co/blog/embedded-c-programming/
Python:
Rust:
