Flutter in Parallel
Parallelism (often confused with Concurrency), is not something we really associate with Flutter/Dart.
My first bug that I fixed for mobile phones back in 2006 was a Threading issue. As part of my job at Tikal, I come across many cases where applications need to do things in parallel.
Let me talk to you about something that I experienced when developing an app for Flutter…
About 20 years ago, I made a desktop app called Electric Fields.
Initially developed using Delphi for Windows. It was supposed to be a simple app that just looks nice by simulating electric charges in a field. It is interactive in that the user can click the screen to manipulate electric charges, by adding them, or inverting their charge, or changing their magnitude. Each image progressively renders smaller rectangles until they are single pixels.
A few years ago, I ported the app to Android, using AsyncTask, and shortly after migrated to use Rx.
Problem? What problem?
When I later ported the app to Flutter, I experienced some challenges:
- Flutter does not support mutable bitmaps natively.
- Flutter does not support multi-threading natively.
Aside from the no-bitmap problem, let me describe parallelism to better understand the non-threading issue.
Before jumping to my solution, let’s overview some parallelism concepts.
What is parallelism?
Basically, parallelism is when we want to run some software in parallel.
Parallelism utilizes multiple resources to execute tasks simultaneously, making processes faster.
Parallelism is the backbone of modern computer architecture. As software engineers, we learned about parallelism way back in high school and college/university.
What is the problem with parallelism?
Poorly-designed software can lead to synchronization issues:
- Race conditions
- Deadlock
Race conditions are where an app might be running 2+ threads that mutate some variable at the same time.
Deadlock is where multiple threads need some resource that the other thread is holding.
Dart tries to avoid some of these problems by being “single-threaded”. Unfortunately synchronization issues are still possible, and jank is still likely.
Flutter uses Dart, but Dart is can run independently of Flutter.
Main culprits of blocking threads:
- I/O
– Reading and writing files
– Reading data from a server, and writing data to a server - Computation
– Animations
– Rendering frames, e.g. a game that uses ray-tracing
– Audio/Video
My app falls into the category of having to do complex computations.
Example in Android
@MainThread
override fun onCreate(savedInstanceState: Bundle?) {
super.onCreate(savedInstanceState)
val response = fetchFromNetwork()
val result = process(response)
output(result)
}If we were to run this code in an activity or fragment …
D/AndroidRuntime: Shutting down VM
E/AndroidRuntime: FATAL EXCEPTION: main
Process: com...demo, PID: 27953
java.lang.RuntimeException: Unable to start activity ComponentInfo{...}:
android.os.NetworkOnMainThreadException
at android.app.ActivityThread.performLaunchActivity(ActivityThread.java:3800)
at android.app.ActivityThread.handleLaunchActivity(ActivityThread.java:3976)
at android.app.servertransaction.LaunchActivityItem.execute(LaunchActivityItem.java:85)
at android.app.servertransaction.TransactionExecutor.executeCallbacks(TransactionExecutor.java:135)
at android.app.servertransaction.TransactionExecutor.execute(TransactionExecutor.java:95)
at android.app.ActivityThread$H.handleMessage(ActivityThread.java:2315)
at android.os.Handler.dispatchMessage(Handler.java:106)
at android.os.Looper.loop(Looper.java:246)
at android.app.ActivityThread.main(ActivityThread.java:8550)
at java.lang.reflect.Method.invoke(Native Method)
at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:602)
at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:1130)Android is “hinting” to try another way that will not cause jank.
Example in Dart
void main() {
final response = fetchFromNetwork();
final result = process(response);
output(result);
}If we were to run this Dart code in a command line, the user would not really feel the lag. However, if this Dart code ran in a Flutter device, the user would experience an unresponsive app, especially if the network response and/or latency is significant.
Let’s look at some of the more popular languages used for developing mobile apps …
Java
Java was designed to be parallel. It has many built-in features:
- Thread
- java.util.concurrent
- Executor
- synchronized
- RxJava
JavaScript
JavaScript is single-threaded by design. There are some newer helpers:
- Promise
- Web Workers
Java + Android
- AsyncTask (deprecated)
- Loader (deprecated)
- Handler
- Activity.runOnUiThread
- View.post
- RxAndroid
- Service component (Runs in the same process by default, on the main thread, and no UI.)
Kotlin
- suspend (means that this function is likely to be blocking.)
- Coroutines (co-operative, same thread)
Swift + iOS
- Thread
- Grand Central Dispatch
- Operations and queues
Dart + Flutter
Dart is single-threaded by design. The async and await keywords produce asynchronous code that looks like synchronous code.
- async → Future (async means that the function is potentially blocking, so the event loop can process other events.)
- async* → Stream (Stream is like a list of Futures)
- await (wait for a future to complete)
- yield (notifies the caller of the Stream that a result has been produced)
- Isolate
- RxDart
How?
How does Dart process code on its “thread”? Dart calls its threads — “isolates”. Each isolate has an event looper that processes “events”.
The event looper processes each event in its queue.
The even looper will remove the event that is at the head of the queue, and add it to its micro-task queue.
After handling a paint event, the next event in this example, is a user touch.
The even looper continues with the next event until the queue is empty.
Example — Synchronous I/O
Here is a sample Dart app that reads some JSON from a file:
void main() {
// Read some data.
final fileData = _readFileSync();
final jsonData = jsonDecode(fileData);
// Use that data.
print('Number of JSON keys: ${jsonData.length}');
}
String _readFileSync() {
final file = File(filename);
final contents = file.readAsStringSync();
return contents.trim();
}Can we make this synchronous code asynchronous?
Example — Asynchronous I/O
Here is the same sample app as above, but slightly modified to be “asynchronous”. How do the await keywords affect the parallelism?
void main() async {
// Read some data.
final fileData = await _readFileAsync();
final jsonData = jsonDecode(fileData);
// Use that data.
print('Number of JSON keys: ${jsonData.length}');
}
Future<String> _readFileAsync() async {
final file = File(filename);
final contents = await file.readAsString();
return contents.trim();
}When the event looper reaches an await instruction, it can handle other events while waiting for the result.
The event looper will put the awaiting instruction at the back of queue, and process other events.
The last event will then read from the file and return its contents.
Very nice, but this does not completely solve our jank. Let’s first explain Dart’s solution to doing too much work in the “main thread” — Isolates.
What are Isolates in Dart/Flutter?
An isolate is something by itself, like when someone has to isolate themselves in quarantine.
The lifecycle of an isolate is simple:
- Have some function as an entry point
- Process events in a loop
- Exit — possibly returning some result
The main isolate has “void main()” function as its entry point.
We can create other isolates by spawning them. The isolate will run its own code not related to the main isolate.
Isolates are like threads or processes, but each isolate has its own resources, its own garbage collector, and a single thread running its event loop. Isolates in the same group share the same code.
Each isolate has its own memory heap, ensuring that none of the state in an isolate is accessible from any other isolate. Because there’s no shared memory, you don’t have to worry too much about mutexes or locks. Although… a mutex might be necessary within an isolate itself for critical code.
As each isolate runs in a separate thread, garbage collection events for each isolate should not impact the performance of others.
Now you might be wondering, “but how do isolates talk to each other?”
Ports
Isolates talk to each other via ports. There are some limitations though:
- A port can only use “serializable” value objects.
- A port cannot send “executable code” such as functions and callbacks.
- SendPort — Can only send to another isolate.
- ReceivePort — Can only receive from another isolate.
SendPorts are retrieved from ReceivePorts.
Any message sent through a SendPort is delivered to its corresponding ReceivePort.
Any message that is sent through its SendPort is delivered to the ReceivePort it has been created from. There in the receiving isolate, the message is dispatched to the ReceivePort’s listener.
A ReceivePort is a non-broadcast stream, and only one listener can receive its messages.
Let’s try make our JSON example even more parallel.
All the code in this example is native Dart, without use of external libraries.
To parse the JSON, we spawn an isolate and wait for the first message which is sent when the spawned isolate exits.
void main() async {
// Read some data.
final jsonData = await _parseInBackground();
// Use that data.
print('Number of JSON keys: ${jsonData.length}');
}
// Spawns an isolate and waits for the first message
Future<Map<String, dynamic>> _parseInBackground() async {
final port = ReceivePort();
await Isolate.spawn(_readAndParseJson, port.sendPort);
return await port.first;
}
void _readAndParseJson(SendPort port) async {
final fileData = await File(filename).readAsString();
final jsonData = jsonDecode(fileData);
Isolate.exit(port, jsonData);
}The arguments to spawn must match the function signature to be invoked.
exit sends a result message which the spawner waits for.
Networking
Isolates can be long-running. Another good use-case for isolates is multiple network requests. Instead of spawning an isolate for each network request, we re-use the same worker isolate.
How can I use my worker isolate to paint?
Painting
My app spawns a painter isolate, and it produces images which it sends to the main isolate.
This is how I expect the isolates to behave. In my app’s main widget, it will display an image that was painted by the isolate.
At some point the user will either restart a new painter, or exit the app.
class EFWidget extends StatefulWidget {
void start() async {
final port = ReceivePort();
final painter = await EFPainter.paint(port);
await for (var image in port) {
setState(() {
_image = image;
});
}
}
}The main isolate is waiting for images on its receiving port, and then updating the widget’s UI.
class EFPainter {
EFPainter(this.sendPort);
static void _paintIsolated(EFPainter painter) async {
painter.start();
}
static void execute(EFPainter painter) async {
await compute(_paintIsolated, painter);
}
static Future<EFPainter> paint(ReceivePort port) async {
final painter = EFPainter(port.sendPort);
execute(painter);
return painter;
}
}paint is called by the widget.
compute is a wrapper function that spawns an isolate, handles errors, and then exits the isolate when finished.
_paintIsolated is the isolate’s entry point.
Now the painter can start painting and send images to the main isolate:
class EFPainter {
void start() async {
while (_running) {
final image = _paint();
sendPort.send(image);
}
}
void _paint() async {
final recorder = PictureRecorder();
final canvas = Canvas(recorder);
canvas.drawColor(Colors.white, BlendMode.src);
…
while (resolution > 1) {
…
}
}
}_paint function does all the complex calculations and rendering of the rectangles.
At first, I tried to paint the rectangles using Canvas which requires PictureRecorder. I would then display the recorded picture, and then paint the read-only picture on top of the new canvas.
This code seems pretty straightforward, right? But if I try to use this code in an isolate…
[ERROR:flutter/lib/ui/ui_dart_state.cc(209)] Unhandled Exception:
Exception: UI actions are only available on root isolate.
#0 PictureRecorder._constructor (dart:ui/painting.dart:5091:59)
#1 new PictureRecorder (dart:ui/painting.dart:5090:23)
#2 EFPainter._run (package:electric_flutter/EFPainter.dart:81:22)
#3 EFPainter.start (package:electric_flutter/EFPainter.dart:56:5)
#4 EFPainter._paintIsolated (package:electric_flutter/EFPainter.dart:201:13)
#5 _IsolateConfiguration.apply (package:flutter/src/foundation/_isolates_io.dart:83:34)
#6 _spawn.<anonymous closure> (package:flutter/src/foundation/_isolates_io.dart:90:65)
#7 _spawn.<anonymous closure> (package:flutter/src/foundation/_isolates_io.dart:89:5)
#8 Timeline.timeSync (dart:developer/timeline.dart:163:22)
#9 _spawn (package:flutter/src/foundation/_isolates_io.dart:87:35)
#10 _delayEntrypointInvocation.<anonymous closure> (dart:isolate-patch/isolate_patch.dart:300:17)
#11 _RawReceivePortImpl._handleMessage (dart:isolate-patch/isolate_patch.dart:192:12)“root isolate” is the main UI isolate.
So I had to change the way I painted the images — not using the built-in Flutter components, but rather manually paint rectangles into byte buffers. Then the painter isolate sends the raw bytes to the main isolate, where it converts the buffer image to a Flutter image on its main thread. Except for that, all the heavy computations happen in the painter isolate.
class EFWidget extends StatefulWidget {
void start() async {
final port = ReceivePort();
final painter = await EFPainter.paint(port);
await for (var img in port) {
final image = await _toImage(img);
setState(() {
_image = image;
});
}
}
}Web
Error: Unsupported operation: dart:isolate is not supported on dart4web
at Object.throw_ [as throw] (http://localhost:64022/dart_sdk.js:5063:11)
at Object._unsupported (http://localhost:64022/dart_sdk.js:61503:15)
at isolate$._ReceivePort.new.get sendPort [as sendPort] (http://localhost:64022/dart_sdk.js:61173:23)
at paint (http://localhost:64022/packages/electric_flutter/ElectricFieldsPainter.dart.lib.js:241:145)
at paint.next (<anonymous>)
at runBody (http://localhost:64022/dart_sdk.js:40211:34)
at Object._async [as async] (http://localhost:64022/dart_sdk.js:40242:7)
at Function.paint (http://localhost:64022/packages/electric_flutter/ElectricFieldsPainter.dart.lib.js:240:20)
at ElectricFieldsWidget._ElectricFieldsWidgetState.new.start (http://localhost:64022/packages/electric_flutter/ElectricFieldsWidget.dart.lib.js:272:74)
at start.next (<anonymous>)
at runBody (http://localhost:64022/dart_sdk.js:40211:34)
at Object._async [as async] (http://localhost:64022/dart_sdk.js:40242:7)
at ElectricFieldsWidget._ElectricFieldsWidgetState.new.start (http://localhost:64022/packages/electric_flutter/ElectricFieldsWidget.dart.lib.js:270:20)Isn’t Dart supposed to be cross-platform?
Since Dart 2 the Dart web platform no longer supports isolates and recommends developers use Web Workers instead. But that’s for another time.
Summary
I have outlined some of the problems and solutions of Flutter/Dart parallelism. I hope you enjoyed sharing the journey I took to discover more performant Flutter apps.
