Edge AI
Brief Introduction to Edge AI
In this topic, we will take a tour of the Edge AI world. We will focus on what Edge AI is, what distinguishes “Edge AI” from other AI, as well as some of the key use cases. The goal is to get the answers to two critical questions:
a) What exactly is Edge AI?
b) Why would we need it?
Definition of Key Terms
Each field of technology has its own set of buzzwords, and edge AI is no exception. In fact, the term edge AI is a fusion or union of two buzzwords that have been fused together to form one powerful term. It’s often heard alongside its siblings, embedded machine learning and TinyML.
We wouldn’t go into detail about the siblings because they are a separate topic in themselves, but here is a quick overview of each.
Embedded: In layman’s terms, embedded systems are computers which control the electronics of various physical devices, ranging from simple microcontrollers to engine control units. Embedded software that runs on them. Figure 1–1 shows a few places where embedded system can be found
Artificial Intelligence (AI): Phew! This is a big one and has multiple definitions. Artificial intelligence (AI) is a broad concept and as such is terribly hard to define. To keep it simple, Artificial intelligence (AI) refers to a computer or machine’s ability to fulfil tasks that normally would require human intelligence, such as pattern recognition, learning from experience, and decision making.
Machine learning (ML) : Machine learning is simpler to define compared to AI. It’s a type of AI, a way of discovering patterns in how the world works — but by running data through algorithms, automatically.
Edge Devices: As the name states, Edge devices are devices that are located at the edge of a network. These devices are piece of equipments that serve to transmit data between the local networks and cloud. Performing computation on these edge devices is referred to as edge computing(Figure 1–2).
Understanding Edge AI
Now coming to the part “What is Edge AI by the way?” followed by “Why would we need it”. Edge AI, as one might anticipate, is the combination of artificial intelligence and edge devices.
IoT devices have traditionally been thought of as nodes that capture data from multiple sensor devices and transmit the information to a centralized location for processing. The issue with this method is in the fact that sending such huge quantities of low-value data is extremely expensive. Not only the expenses for connectivity are high, data-transmission also consumes a significant amount of energy, which is a major issue for battery-powered IoT devices. As a result, the majority of information accumulated by IoT sensors is typically discarded. We’re collecting a lot of sensor data but can’t utilize it in any way.
The answer to this problem is edge AI. Instead of sending data to a remote location for processing, why not do it directly on the device where data is generated? Instead of needing to rely on a centralized server, we can now make decisions locally — no network connectivity required. If we still want to report the information back to upstream servers or cloud we can transmit critical information instead of a every data. This should save cost and energy.
There are many ways to deploy intelligence to the edge. Figure 1–3 shows the continuum from cloud AI to fully on device intelligence.
As we have seen, AI technology can be applied to a wide range of concepts, from being a touch of human insight conveyed in some basic conditional logic to some of the extremely sophisticated recent advances in deep learning.
Edge AI works on similar lines. Edge AI, at its most basic, is about making decisions on the network’s edge, close to where data is generated. However, it can also make use of some incredibly cool features.
Now coming to a million-dollar question, benefits of edge AI, as the saying goes just BLERP, BLERP? Jeff Bier, founder of the Edge AI and Vision alliance, https://www.eetasia.com/whats-driving-ai-and-vision-to-the-edge/ expresses the benefits of Edge AI in plain five words, namely:
- Bandwidth
- Latency
- Economics
- Reliability
- Privacy
With BLERP, it’s indeed simple to remember and articulate the advantages of edge AI. It can also be used as a filter to determine whether edge AI is appropriate for a given application.
Bandwidth: IoT devices frequently capture more data than there is available bandwidth to transmit. This implies that major part of sensor data collected is never used and is simply discarded. In many cases, however, there is insufficient bandwidth or energy budget to send a continuous stream of data to the cloud. That means we’ll have to discard the majority of our sensor data, even if it consists of useful signals.
This is where edge AI swoops in. What if we could directly run data analysis on the IoT device, eliminating the need to upload the data? In such a scenario, if the analysis revealed that the device was poised to fail, we could use our limited bandwidth to send out a notification. This is far more practical than attempting to stream all the data.
Latency: Data transmission is time consuming even if you have a good amount of bandwidth is available. As such, a round trip from the machine to the server can take tens to hundreds of milliseconds. Latency can thus be measured in minutes, hours, or days in some cases.
Edge AI solves this issue by eliminating the round-trip time completely. A self-driving car is a fine example of this. On-board computers power the car’s AI systems. This enables it to respond almost instantly to changing environments, such as the driver in front slamming on their brakes.
Economics: Connectivity too is expensive. Connected products cost more to use, and the infrastructure on which they rely is pricey to the manufacturers. Higher the cost, the more bandwidth needed. Things get especially painful for devices deployed in remote locations which require long-distance satellite connectivity.
Edge AI systems tend to reduce or minimize the costs data transmission over the network and processing it in the cloud by instead processing it on-device. This can open up a wide range of previously unattainable use cases.
Reliability: On-device AI-controlled systems have the potential to be more reliable than cloud-connected systems. When we add wireless connectivity to a device, it introduces a vast, overwhelming web of dependencies, ranging from link-layer network technology to Internet servers which may run our application.
Reliability is more often a tradeoff, and the degree of reliability required differs based on the use case. Edge AI could be an effective tool for increasing product reliability.
Privacy: In the recent years, many people have reluctantly accepted a tradeoff among both convenience and privacy. According to the theory, if we want our technological resources to be smarter and more useful, we must give up our data. Because smart devices typically make decisions on remote servers, frequently sending streams of sensor data to the cloud. This may be sufficient for some applications; for example, we may not be concerned if an IoT thermostat transmits temperature data to a central, remote server. However, privacy is a major concern for other applications.
The potential of edge AI to facilitate privacy opens up a plethora of exciting use case scenarios. It is especially important in industry, security, education, healthcare, childcare applications.
Now that we have a better understanding of the benefits of Edge AI, let us conclude by simply putting the Edge AI factors for good.
Edge AI for Good
The unique benefits of edge AI provide a new set of tools that can be applied to some of our world’s biggest problems. Edge AI is already being used to make a significant impact by technologists in fields such as conservation, healthcare, and education. Here are a few examples that have piqued our interest:
· Smart Parks are using collars running machine learning models to better understand elephant behavior in wildlife parks all over the world.
· Izoelektro’s RAM-1 helps prevent forest fires caused by power transmission hardware by using embedded machine learning to detect upcoming faults.
· Researchers like Dr. Mohammed Zubair, from King Khalid University in Saudi Arabia, are training models that can screen patients for life-threatening medical conditions such as oral cancer using low-cost devices.
· Students across the world are developing solutions for their local industries. João Vitor Yukio Bordin Yamashita, from UNIFEI in Brazil, created a system for identifying diseases that affect coffee plants using embedded hardware.
The properties of Edge AI make especially well suite for application to global problems
“Fit for purpose” is often the goal
When it comes to traditional AI, the goal is typically to achieve the best possible performance at any cost. Deep learning algorithms used in server-side applications can indeed be gigabytes in size, and they rely on powerful GPU compute to run in a timely manner. When computation is not an issue, the most accurate model is recurrently the best choice.
The advantages of edge AI are coupled with significant constraints. Edge devices tend to have less capable compute, and there are frequently tricky trade-offs to consider between on-device performance and accuracy.
This is a challenge, but certainly not a barrier. There are significant benefits to running AI at the edge in a variety of use cases, which easily outweighs the penalty of slightly lower accuracy. Even a miniscule fraction of on-device intelligence is preferable to none at all.
The goal is to create applications that take advantage of this “Fit for Purpose” approach, which Alasdair Allan describes elegantly as Capable Computing. The key to its success is to use tools that assist us in comprehending the performance and effectiveness of our applications in the real world, after any performance penalties have been taken into account.