What is Edge Intelligence?
Post co-author: Zanita Rahimi
What is Edge Intelligence? This sounds like an easy question but it’s not easy to answer when some of the conceptual requisites are not clear.
The TL; DR is the following:
Edge intelligence = Edge computing + Artificial Intelligence.
In this article, we will try to give a detailed answer on how the state of the art has reached such practice, together with some use cases directly from the field.
Introduction
The data world is continuously changing. Back in the days when we needed a whole room to process 10–20 MB of data, nobody had an idea of what the tech industry was about to become.
Over the years computers have become more and more powerful and engineers have done a great job in both decreasing the amount of energy needed and increasing memory and disk size.
It is still debated whether the famous Moore’s Law will last in the current century or not, but this gives us a great opportunity to introduce today’s topic.
Since the number of transistors in a chip has doubled roughly every two years in the last 60 years, whereas the cost has been halved at the same time, we understand how good humans have been in designing good hardware and therefore how cheap and easy is to run workloads on a mini server.
Having these developments in mind, it’s not so surprising that smartphones nowadays have more RAM than some of the laptops in the market 10 years ago. Installing small computing hardware equipped with internet connections has become more and more affordable and that’s why different companies have invested in this new technology called Edge Computing.
In order to understand Edge Computing, we should know how computing works in a centralized environment, and what problems are related to such architecture.
Centralized topologies gather data from different sources, you can think of them as different sensors. Nowadays you can find such sensors everywhere: in a smart home, in a smart car, or in an industry storing and computing data in a single location, often being the cloud.
This architecture, however, faces some challenges:
- Increased latency — since the network traffic is going in a single direction
- Lack of resources
- Difficulties in architecting a data platform that is efficient enough to organize data coming from different sources.
Let us walk you through an example that emphasizes the importance of edge computing. Consider an application that provides face recognition services. Before the edge computing era, the image of the person that the application is trying to recognize must travel to a hidden main server, where the algorithm is executed, and the response travels back as soon as the processing is done.
All this process can also happen closer to where the data is generated — your phone, thanks to edge computing.
With the concept of Cloud Computing clearer, it should not be too difficult to understand what Edge intelligence means. An example has been already provided: that facial recognition algorithm running on your smartphone is an example of an Edge intelligence application. The phone is an edge device, namely a piece of computing hardware close to the action (the face of the person whose identity is needed), which is running an AI algorithm.
Edge computing + Artificial Intelligence = Edge Intelligence
As an end to this introduction section, we would like to clarify that there are different names in the literature addressing the same technology: ML Edge Computing, Edge AI, AI at the Edge, etc.
Luckily in this case changing the order of the operands does not change the result, the concept stays the same. In this article, the term Edge Intelligence will be used.
Fog computing, Cloudlet Computing, Edge computing
In computing history, a lot of shifts in the paradigms have been seen. It all started with a single big main frame in the 60/70s, then slowly those monoliths were connected thanks to the internet and then we realized that in order to perform more complex workloads, we needed more computing power.
The cloud computing era was starting. We started building, designing and optimizing data centers. We came back to big rooms full of hardware but this time the amount of data processed increased enormously. With the rising popularity of artificial intelligence in technologies such as machine learning, deep learning, and AI-related applications, the need for computation power kept increasing so much that the network resources proved to be a bottleneck. The low latency communication requirement of smart vehicles or smart industries pushed the research over its own limits. As mentioned in the introduction, the aim of edge computing is to overcome these challenges.
The way that the different layers of IoT devices are organized, the different communication protocols they use, and the network used, leads to different types of architectures. In the literature we can find three main categories:
- Fog Computing
- Cloudlet computing
- Edge Computing
Fog Computing
Fog computing introduces a middle layer between the edge devices in the cloud.
The objective is to reduce the amount of information reaching the cloud at the same time. The middle layer is composed of different nodes, devices, and gateways closer to the edge but not necessarily on the sensor itself.
The role of such a layer is to store and precompute part of the data easing the Cloud job.
Less data would reach the main server, less network resources would be used and therefore the latency should be reduced.
Fog nodes compose a distributed architecture and usually sit closer to the sensors: this reduces both the response time of the application and offers better privacy since part of the data can be analyzed separately before sending them into the cloud.
Cloudlet computing
The cloudlet concept is like fog computing but not quite the same.
We always have a three-tier architecture with the cloud at the top and the edge devices at the bottom, just the middle layer changes.
We don’t have nodes but a scaled-down version of a data center.
Similarly to fog computing, better performances are achieved by introducing intermediate computing layers.
Edge Computing
Edge computing can be considered as a subset of fog computing where the computation happens on the edge devices themselves. With this technology, the data is first processed in place and sent instantly to the cloud, with a tremendous performance gain in real-time scenarios.
Edge Intelligence
Now that edge computing as a concept is clear, understanding edge intelligence is straightforward. The edge computing architecture uses edge devices to perform computations, if the algorithm belongs to the AI family, we can talk about edge intelligence. Federated learning, embedded AI, and coordination mechanism are just a few keywords around this topic but till now we have made enough abstract examples, let’s deep dive into some real use cases.
Edge Computing Use Cases
Video Analysis
Video analysis is about understanding the scenes that are captured by multiple cameras, and often this process must be efficient, low-latency, and real-time. As mentioned in the previous section, video feeds cannot be efficiently analyzed in a cloud computing architecture. These challenges can be easily met by using an edge architecture, and we will see use cases from different domains in the following paragraphs.
Data Reply’s Demo — Object detection in real-time data on edge devices
There are specific workloads that can be run on Cloud and others on Edge, and we will discuss more of this in the next article of this mini-series. But we can also benefit from something called a hybrid solution.
Hybrid cloud and edge solutions offer unmatched flexibility, value, and security for enterprise computing applications; therefore, we treated Edge Computing as an extension of the Cloud while exploiting the benefits of both worlds, for building a demo that performs video analysis.
We built an image recognition system that predicts certain categories of objects in a video stream with low latency. The Edge Cluster consists of 4 Raspberry Pi nodes where the video stream is read and sampled into images for our image recognition system, which returns the prediction results, while the analytics and long-term storage are offloaded to the Cloud.
This is an end-to-end product that includes:
- Object detection in video streams,
- The discarding of similar frames (stateful stream processing),
- Long-term storage for backup and compliance purposes,
- Ad-hoc Real-Time, and Hybrid Queries
Such a solution can be used for different use cases which are elaborated on in the following sections. One of the use cases can be seen in manufacturing, where edge-enabled smart cameras can scan the barcodes of each pallet to keep track of what stock has been processed in each micro-location, thus speeding up warehouse processes.
If you are interested in the technical details of the system’s architecture that has been developed, you can read more in the dedicated blog post written by Tayfun Wiechert.
Transportation Logistics
With a similar setup, a wide variety of use cases in transportation logistics can be covered. Examples include:
- Predictive maintenance — if an issue is detected, a notification can be sent to the responsible parties, who can then ensure that the issues will be resolved.
- Driver performance & traceability — to identify the total kilometers driven, and the number of stops.
- Tracking of objects — by using CNN (Convolutional Neural Networks) logistics objects can be identified and localized.
Autonomous Driving
How do you tell an autonomous car where to go? Autonomous vehicles use sensors to determine exactly where they are on the planet and where they want to go. When an obstacle is encountered, the vehicle starts sending its lidar and cameras data waiting for clearance. The amount of data generated is enormous.
A Stanford seminar published data from an autonomous vehicle at Nissan consisting of 15 cameras, 2 lidars, 1 radar, 1 GPS, and 1 CAN. The total data output was 140,235 MB /s. With simple calculations we get:
- 1 Minute -> 10 GB
- 10 Minutes -> 100 GB
- 1 Hour -> 500 GB
- 24 Hour -> 12TB
A huge amount of the generated data is transient, so we do not have to save them in cloud data centers. Also, we do not have the commodity of doing that both in the speed and storage space aspects. This use case is not possible without edge capabilities because of the low latency needed to analyze data and return instructions immediately to function safely.
Some researchers proposed a methodology where they cache the content needed in self-driving cars.
And why do we need to cache the content in such setups? Well, edge devices might need to upload video/image content to edge servers for further processing. By making use of caching, they can upload only different pixels or frames, and for repeated scenes, they can reuse the results thus avoiding unnecessary computations.
With 97.82% accuracy in predicting what needs to be cached, this approach can minimize latency brought on by a multi-edge architecture for autonomous vehicles.
Manufacturing
The low latency enables immediate response to problems on assembly lines, improving quality & efficiency, detecting production errors, improving communication and interaction between machines, reducing energy consumption, planning manufacturing processes, allocating resources, etc.
One other application of edge computing that our clients seem to be interested in is the ability to track the inventory, by monitoring the stock, where the inventory is located, and what quantities are available. Real-time inventory tracking, which is a crucial component of the supply chain, can be tackled with an edge-computing solution and the low latency offered by it.
Smart Grid
Smart Grid uses a broad range of resources to deliver electricity to millions of households, thus reducing energy wastage and cost. Processing and analyzing the edge servers and their power consumption let us have a real-time system that can prevent transmission line faults.
The use cases would include smart monitoring of the grid status & predictive maintenance, load forecasting (demand management and prediction) & production optimization.
Cloud Offloading
Network-connected technologies such as portable devices, PCs, smartphones, etc., must have low-latency and real-time processing capabilities.
To improve the request-response time, we can utilize edge devices to offload the workload by caching either the data or the operations that are to be performed, which is exactly what we at Data Reply did in our demo. Object detection logic is running on our edge cluster, while the real-time dashboard presenting the recognized items runs in the cloud.
