isolate_agents: Easy Isolates for Flutter

This article introduces a new package for Flutter and Dart that makes isolates easier to work with. It helps anyone that wants to offload work from one isolate to another. If you want to skip ahead, check out the new isolate_agents package.

Background

As Flutter projects mature and become more complex, it’s not uncommon to start seeing bottlenecks in Flutter engine’s UI thread. Dart’s async/await feature is great for working with API’s that are asynchronous, but that doesn’t help you write new code that is executed off of the UI thread. Early in Dart’s history, Isolates were added to address this problem. Wikipedia classifies Dart as an Actor Model language and, while Dart has isolated regions of memory associated with threads of execution, it lacks a predefined protocol or API for communicating between them.

In order to use an Isolate in any way that is more complicated than just “fire a task and get a notification when complete”, one must:

1. Devise a protocol for how the Isolate will communicate input and output, for example, how is data encoded across the SendPort API?
2. Perform a SendPort handshake to establish a communication channel.
3. Manage a queue of handlers to be executed serially when the results are calculated.
4. Devise an error handling protocol.
5. Devise a shutdown protocol.

The Flutter team recognized that using Isolates is cumbersome and added the compute function. This makes the “fire a task and get a notification when it’s complete” use-case easier to write, but it completely abstracts away the Isolate, which doesn’t give the developer full control over the performance of their code. You can’t control the overhead of spawning an Isolate or store state in the Isolate. Both of these facts contribute to suboptimal performance, when the whole point of the function was to improve performance.

If only there was a higher-level API on top of an Isolate that retains control of the Isolate, and the ease-of-use of compute… It could simplify multiprocessing tasks and make complex multiprocessing tasks achievable.

Clojure Agents

When approaching a solution to the problem of the extra work Dart Isolates require to be useful, I looked back to my experimentation with the Clojure programming language. Clojure is a functional Actor Model language whose data is immutable by default. It was designed with heavy multiprocessing in mind and it doesn’t have the same problem with difficult verbose multiprocessing that Dart has. The designer took a slightly different approach to Actors than languages like Erlang did, instead being inspired by functional languages. In Clojure, the concept of an agent is a piece of data associated with a thread. In that sense it’s similar to Dart Isolates, but unlike Dart Isolates, agents have an established protocol, channel, and response mechanism. The protocol sends closures to the thread and executes them there — the results replace the held state.

A simple usage of an Agent:

; Create the thread with state = 0
(let [foo (agent 0)]
  ; Execute x + 1 on the background thread and assign it
  ; to the value associated with the thread.
  (send foo (fn [x] (+ x 1)))
  (send foo (fn [x] (+ x 1)))
  ; Request and wait for the value from `foo` and print it.
  (print (deref foo)))

isolate_agents Package

The new package, isolate_agents, implements a similar pattern as the aforementioned Clojure agent. It creates a standardized protocol for communicating between isolates and eliminates the handshakes and state management you must perform to use them effectively.

Here is the same example as above, instead written in Dart with the isolate_agents package:

void main() async {
  // Create the thread with state = 0
  Agent<int> foo = await Agent.create(() => 0);
  // Execute 1 + 2 on the background thread and assign it
  // to the value associated with the thread.
  foo.update((x) => x + 1);
  foo.update((x) => x + 1);
  // Request and wait for the value from `foo` and print it.
  print(await foo.exit());
}

Agent patterns

The previous example using isolate_agents was very simple. Here are some more interesting patterns that can be accomplished using Agents as building blocks.

Long-lived background handler

The compute method spawns and kills a new Dart Isolate: this incurs overhead when spinning up the Isolate. You can eliminate that overhead by using a long-lived Agent:

// An agent that lives for the duration of the main isolate.
final Future<Agent<String>> _agent = Agent.create(‘’);

Future<String> _receiveEncryptedMessage(String encrypted) async {
  Agent<String> agent = await _agent;
  agent.update((_) => _decrypt(encrypted));
  // Notice the use of `read` (not `kill`) to keep the agent alive.
  return agent.read();
}

Cross-Isolate state management

Managing mutable state across multiple isolates is tricky, but assigning one Agent to manage the state makes it easy:

// Executed on the root Isolate.
void _addUser(Agent<DataModel> model, String name) {
  model.update((database) {
  database.addUser(name);
  });
}

// Autosave writing to disk by periodically executing on
// the background isolate.
void _autosave(Agent<DataModel> model) {
  DataModel db = await agent.read();
  db.saveToDisk();
}

Pipeline

Imagine you want to set up a pipeline where one work job is being prepared while another is being serviced. In this example, one Isolate generates a tree that is concurrently rendered as an image by another Isolate:

class Pipeline {
  Pipeline._(this.builder, this.renderer);

  final Agent<Tree?> builder;
  final Agent<Object?> renderer;
  
  static Future<Pipeline> create() async {...}

  Future<Image> process(int index) {
    ReceivePort receiver = ReceivePort();
    SendPort sender = receiver.sender;
    builder.update((_) => _buildTree(index));
    builder.update((tree) {
      renderer.update((_) {
        Image image = _buildImage(tree);
        sender.send(image);
      });
      return null; 
    });
    Object? object = await receiver.first;
    return object! as Image;
  }
}

Future<List<Image>> _startJobs(Pipeline pipeline) {
  List<Future<Image>> images = [];
  for (int i = 0; i < 10; ++i) {
    images.add(pipeline.process(i));
  }

  return Future.wait(images);
}

Background operation cache

This use case has a heavy calculation that needs to be executed on a background isolate, but the result should be cached. In this way, the same calculation is never performed twice, and new calculations can be based previous calculations:

final Future<Agent<WorldMap>> _agent = Agent.create(() => WorldMap());

Future<LocalMap> getLocalMap(int latitude, int longitude) async {
  Agent<WorldMap> agent = await _agent;
  agent.update((worldMap) => {
    if (!worldMap.hasLocalMap(latitude, longitude)) {
      // Perform heavy loading of the map on background isolate and memoize
      // the result.
      worldMap.loadLocalMap(latitude, longitude);
    }
    return worldMap;
  });
  return agent.read(query: (worldMap) =>
      worldMap.getLocalMap(latitude, longitude));
}

Flutter example

You can check out an example that uses isolate_agents in the romeo_juliet project on GitHub.

The app performs the following steps:

1. Load the text asset with all the decrypted messages, split them up, encrypt, and write them to the documents directory.
2. Query the platform’s documents directory and store it on the agent.
3. Start a timer that (every second) triggers a job to read the encrypted message from the documents directory and decrypt it.
4. When the job finishes, if we have a new decrypted message add it to the cache on the root isolate, and reload the scroll view.

How does this code differ from using Isolate directly?

1. This approach requires less code and provides a consistent interface to working with Isolates.

How does this code differ from using Flutter’s compute function?

1. The Isolate is reused whenever the app receives the request to decrypt a new message.
2. The Isolate can store state, in this case in the documents directory path, so it doesn’t have to recalculate it on the Isolate or send it over every time.

Both of these things contribute to better performance.

A note on performance

While isolate_agents unlocks more complicated multiprocessing use-cases, it’s somewhat hindered by the performance of Dart’s SendPort API. One must consider the overhead of the payloads that are sent between Agents. For this reason the following methods were added:

• The Agent.read method allows reading a smaller portion of the state held by an Agent with the query parameter.
• The Agent.exit method allows reading the value of a dying Agent in constant time.

Dart tries, in most cases, to optimize data communication between Isolates but unfortunately it isn’t explicit about when those optimizations happen. I hope that, at some point, Dart can have move semantics that allow us to assert constant time transmission of data between Isolates, but that seems unlikely to happen anytime soon.

In closing

Thanks for giving isolate_agents a look. I hope soon you’ll be slamming all the cores of your device with ease. You’ll notice that I haven’t designated the package as a 1.0.0 release. I want to keep the design open for feedback, so feel free to reach out to gaaclarke on GitHub or the Flutter Discord channel.