Understanding Dart/Flutter Messaging System

It is known that Dart VM runs in an event loop. There are two queues inside the event loop, event queue and microtask queue. After main() function executed, the Dart app will flush the microtask queue then handle next event in the event queue, And once an event is handled, Dart app moves on to flush the microtask queue and handle a new event, and so on.

Dart code is running within isolate, isolates can not communicate with each other directly. Instead, they have to send or receive messages via ports. But how it works inside Dart VM? There is a messaging system under the hood. This messaging system is the reason why your Dart/Flutter app is running as you expected, or known as event driven. Not only the event loop inside an isolate rely on this messaging system, the communication between isolates, timers, IO and other outside events (like touch events, platform channels in Flutter) are also rely on this messaging system. Understanding this messaging system, you will also get the answers of questions like why http request won't block your Dart/Flutter app.

Dart Messaging System

When an isolate is running, it is running in an os thread(mutator thread). Dart VM uses a thread pool to run isolates. From this point of view, a running isolate is more like a task running in this thread pool. When we say “running”, it actually means running the message handler of the isolate. Once there is an incoming message, the message will be queued and isolate’s message handler will be scheduled to run within the thread pool as a task.

Message

A valid message must has a valid destination port. A port is actually an integer, just like an ip address or id number. There are 2 kind of messages, one is normal message, the other is oob (out-of -brand)message. oob message is for control of spawned isolates, used to pause, resume or kill other isolates. oob message has higher priority than normal message.

Message Handler

Each isolate has a dedicated message handler, which will handle all incoming messages. All incoming messages will be put into a message queue, then if the message handler is not running, Dart VM will schedule this message handler as a task to run in thread pool. When message handler is running, it will dequeue and process each message in fifo manner.

Message Queue

There are 2 message queues in message handler, one for normal message and another for oob message. When posting a message to message handler, message handler will enqueue the message to one of these message queues by message type. Normal message goes to normal message queue, oob message goes to oob message queue.

Port Map

In Dart messaging system, the key is the port, if you have the port, you will find the message handler bound to this port, when you have the message handler, you will have the isolate. So in order to track all port and message handler pairs. Dart VM has a global port map to store all these port-message handler pairs.

When an isolate creates a new receive port, port map will allocate a port number and bind to this isolate’s message handler. This new port-message handler pair is then stored in port map.

When sending a message to an isolate, port map will look up all stored port-message handler pairs and find the pair match the destination port in message. When found, this message is enqueued into corresponding message handler’s message queue.

When handling the message. message handler will dequeue a message and if it is a normal message, message handler will enter into Dart domain and further deliver the message to each closure of corresponding receive port. After a message is handled in Dart code, microtask queue will be flushed immediately.

When a receive port is no longer needed, it should be closed in time, otherwise there could be “resource leak”.

This port-message handler mechanism is only for “one-way”, that is, if 2 isolates need to send message to each other, they both have to have opened receive ports.

Timer

Besides send/receive messages between isolates, Timer is another important event source, There are 2 kind of timers, timers with delay and timers without delay. Only timers with delay relay on OS capability.

Dart VM uses event handler to manage low level timer resource, When Dart VM initialized, it will start a new thread called “dart:io EventHandler”. The actual realization depends on the embedded os platform. For example, with Android, event handler uses epoll to provide timer counting down and waking up.

If an isolate needs to communicate with event handler, because they are in different threads. Their communication have to go through the messaging system. It looks like both isolate and event handler need to open ports. But because event handler is a global singleton and can be accessed anywhere in Dart VM, so only isolates need to open port to receive messages form event handler. When an isolate needs to send message to event handler, a function call to “EventHandler_SendData” will do the job.

In Dart domain, all timers are managed by _Timer. _Timer will open a receive port to receive time out events. For these 2 kinds of timers, they are handled differently.

If a new timer has no delay is created, this timer is queued in a list called “ZeroTimer” and then _Timer send a message with id “_ZERO_EVENT” to itself.

If a new time has delay is created, this timer is placed in a binary heap called “_TimerHeap”, and the nearest time out time is updated to event handler. When time out happens, EventHandler will send a “_TIMEOUT_EVENT” to _Timer.

When either message is received, _Timer will find out all pending timers and execute their callbacks.

IO

The IO of Dart is also based on the messaging system. In Dart domain, all IO operations are managed by _IOService. When _IOService is initialized, it will open a receive port to receive all IO messages. _IOService defined all IO operations as specific op code. For example, “open file” is defined as 5. There are total 43 op codes. including all kinds of file, directory, socket IO operations.

When Dart code started an IO operation, A new port is created by port map, This new port will be bound with a native message handler, which will handle IO message from _IOService and do the real IO work. Just like isolate’s message handler, native message handler will also be scheduled as a task to be run in thread pool.

When native message handler finished IO job, it will send result back to _IOService. _IOService is keeping all pending IO requests of current isolate. the incoming result will be matched by port of native message handler and corresponding future is completed with the IO result.

Flutter Customization

Flutter has made some modifications on Dart messaging system. It is known that Flutter engine runs on 4 threads , These are Platform, UI, GPU and IO threads. Each thread runs a task runner:

• Platform Task Runner
• UI Task Runner
• Raster Task Runner
• IO Task Runner

A task runner is basically a message looper. The UI task runner is where the engine executes all Dart code for the root isolate. The root isolate is a special isolate that has the necessary bindings for Flutter to function. This isolate runs the application’s main Dart code. But for a normal isolate, as we described above, is running in thread pool as a task. So in order to make root isolate running on UI task runner, Flutter made some customizations on original Dart messaging system.

• Firstly, forbidden message handler of the root isolate to be run in thread pool. For a normal isolate, before run any Dart code, Dart VM will call “MessageHandler::Run()” to assign a thread pool to this message handler. when start root isolate on UI thread, Flutter engine skipped this function call. Thus the message handler of the root isolate dose not associated with any thread pool and can’t be run in thread pool.
• Secondly, make message handler of the root isolate to be scheduled by UI task runner. When initializing root isolate, Flutter engine set UI task runner as message notify callback of the message handler. thus each time when there is an incoming message, UI task runner will be notified and schedule message handler to run on UI thread.

Another customization is microtask. Flutter engine move handler of microtask queue from Dart domain to engine by setting its own microtask scheduler. Since message handler of root isolate is running on UI thread. Each time a UI task is executed, microtask will be flushed. Another place where flush microtask happened is between “_beginFrame” and “_drawFrame” function calls.