Demystifying the Deterministic Latency Engine: The Secret Weapon that Makes HUAWEI Mate 30 Series so Fast and Stable

On September 26th 2019, Huawei commercially launched its new flagship HUAWEI Mate 30 Series in Shanghai, China. There were a number of new features presented which were the highlight of the launch event, such as the impressive 5G experience. However one particular high point of the presentation was the “Deterministic Latency Engine”, a groundbreaking technology debuted at the Huawei Developer Conference in August. Huawei announced that the HUAWEI Mate 30 Pro will feature the Deterministic Latency Engine, . Tthis will deliver a smooth and stable performance, with a 30% lower operation latency and 27% improvement in performance fluctuation, rivaling other devices on the market such as the iPhone 11 Pro.

So, what is the Deterministic Latency Engine and how can it improve smartphone performance? To answer this we would need to start from how resource scheduling is currently handled by the Android operating system.

01. Inadequate Resource Scheduling Ability of Android

Android is obviously a Linux-based operating system and therefore its resource scheduling is rooted in the Linux kernel. Compared to PCs, smartphones have relatively limited resources yet support significantly more complex and dynamic user scenarios. As a result, the Linux-based resource scheduling system is less capable of efficient resource management. That’s why slow performance or lagging is very common in Android phones. To change this situation, it is necessary to make some changes in the Linux kernel, which means improving the resource scheduling of three major factors: CPU, memory, and Input/Output.

Inappropriate CPU Scheduling

As one kind of system resource, the CPU needs to be managed, scheduled and allocated for applications that take priority, to help them achieve the highest efficiency. This is where schedulers work to increase efficiency, and we have seen schedulers as early as Linux 2.2. For many years, developers have been trying out different models including single queue, multi-queue, thread group and process group schedulers to design a more efficient scheduling method. Linux’s CPU scheduling tends to be fair in resource scheduling, while Android is constantly running in a resource critical state. This situation deteriorates the accuracy and efficiency of CPU resource management in smartphones.

Chaotic Competition for Memory Resources

Memory is the bridge for the system to communicate with the CPU, all applications in the phone need to use RAM (Random Access Memory) to run, therefore memory performance has a great influence on smartphone performance. Since memory is allocated and recycled in the same way for both foreground and background applications in Linux, the system treats every application request fairly, no matter if it is from the foreground or background. In this case, a multi-tasking system will become slow and laggy because applications from the foreground and background are continually competing for memory resources.

Overloaded Input/Output Requests

Almost every smartphone operation replies on input and output (read and write) to read data from the storage device into the memory as well as to write persistent data back to the storage device. When there are a large number of Input & Output (I/O) requests in the system which greatly exceeds the concurrent processing capacity of the smartphone’s storage device, I/O requests that cannot be processed promptly will be cached in the system’s software queue and the storage device’s hardware queue. For example, you can drive a supercar at 200km/h if the road is clear, without traffic. But the supercar can only crawl along at the same pace as other ordinary cars if it is in traffic jam.

02 Three Solutions to Address Three Major Factors

First, the Deterministic Latency Engine redefines the meaning of foreground applications by giving dynamic attributes to the “foreground” tasks and relating them with specific user scenarios. The redefined concept of “foreground”, together with reasonable resource management of CPU, memory and I/O, ensures rapid and stable response for user requests.

For example, when you are playing a heavy-load mobile game, the system needs to execute many program segments from the beginning to the end. All program segments including foreground and background will be temporarily tagged as “foreground” as long as they are relevant to user interaction. If the background programs are taking up too many resources of the CPU, memory or I/O, the Deterministic Latency Engine will provide three solutions to appropriately allocate resources and deliver smooth and stable user experience. They are VIP Priority Scheduling, Priority Adjustment and Resource Reservation.

Solution one: VIP Priority Scheduling and Timely Response

As mentioned before, the downside of Linux’s CPU scheduling is being too fair and not allocating the right resources to priority applications. The Deterministic Latency Engine’s first solution is VIP Priority Scheduling. It is like when you want to deposit money at the bank counter, if you are a VIP member, you can skip the queue and go to the VIP counter directly, to ensure a higher efficiency.

In addition, Huawei has modified the Networking Protocol Stack to restrict the bandwidth consumption of low-priority operations at protocol level. Given a certain network bandwidth, the Deterministic Latency Engine will try its best to prioritize the I/O requests of “foreground” applications.

Solution two: Dependency based Priority Adjustment

Since the Linux Kernel is not aware of the upper-level applications, it has no way of knowing the dependencies between processes. Therefore, if one high-priority task relies on a low-priority task, it is ideal to bump up the priority of this particular low-priority task. To that end, the Deterministic Latency Engine starts with redesigning relevant data structures to gives the CPU and I/O schedulers the ability to accurately identify which processes are truly critical to user operations at a given time.

For example, customer A goes to an ATM to withdraw money, and customer B is waiting in the queue right behind customer A to deposit money. Unfortunately, the ATM is out of money at the moment, so customer A either waits for cash replenishment of the ATM or raises customer B’s priority and lets him deposit money first so that customer A can immediately withdraw money deposited by customer B. The advantage and disadvantage are obvious between these two solutions, because you don’t know when the bank will load the ATM with new bills, therefore customer A needs to depend on customer B in this case. It’s these instantaneous and smart tweaking of CPU and I/O resources for “foreground” requests that ensures the best experience for users.

Solution three: Resource Reservation to Foresee Requests

Taking memory scheduling as an example, the intelligent foreground application scheduling can not only ensure the priority memory resource allocation for “foreground” task requests, but also can reduce the number of times they enter the slow “path”, as well as alleviate Memory Churn when it is heavily loaded.

It is like bus lanes in the city. As we all know, a bus lane is a lane specially assigned for buses to pass along at a specific time. It is mainly for dealing with various road congestion situations during rush hour or for emergency use. Bus lanes keep the traffic flow smooth during peak hours. Similarly, the Deterministic Latency Engine has improved Linux’s resource scheduling strategy. It can accurately identify which tasks are the “buses” and whether these tasks can use the “bus lane”. If so, the Deterministic Latency Engine can reserve relevant resources for them, to improve their processing efficiency and ensure adequate supply of key resources for user operation.

03 Building core competence with Innovative Technology

Earlier this April when Huawei released another groundbreaking technology, the Ark Compiler, Richard Yu said, “Only leading companies like Huawei can make breakthroughs in core technologies such as operating systems and compliers. Other companies now can only bring small innovations, while Huawei is committed to core technology innovation, which requires core abilities”. This is also true for the innovation of the Deterministic Latency Engine. When many manufacturers claim that their own system is a deeply customized version of Android, Richard Yu has pointed out the essential difference between Huawei and others in the industry.

To develop the Deterministic Latency Engine, Huawei has invested a team of more than 200 technology experts for almost two years. The team has “performed major surgery” on the whole Android system, from framework to core. With Huawei’s R&D strength in CPU, baseband, Wi-Fi, storage and chips, the Engine is also bringing out the best of Huawei’s software and hardware collaboration. The Deterministic Latency Engine combines Huawei’s technology innovations and advantages in the industry. It would be hard for others in the industry, to match the technological innovations that Huawei so often provides.