The Layer Cake

How Flutter uses Widgets, Elements and RenderObjects to create delicious eye-candy at 120fps.

Flutter is a great UI framework and it’s a bliss to build beautiful user interfaces at light speed. Just jot down a few lines of code, hit save [INSERT MAGIC SUB-SECOND HOT RELOAD HERE] and voilà —your app is running smooth as silk at up to 120fps. However, have you ever asked yourself how it comes that Flutter is so fast? What’s the secret sauce it’s made of? Or — how Flutter actually works? Well, then search no more! Go grab a fresh coffee (or tea) and read on.

Maybe you have already heard that Flutter is all about widgets. Your app is a widget, a text is a widget, the padding around a widget is a widget and you even recognise gestures through a widget. Well, that’s not the entire truth. What if I told you that widgets make Flutter development fast and easy but you could also build a whole Flutter app without even using a single widget. Let’s find out how by digging a little bit deeper into the framework.

The Four Layers

Probably you’ve already seen an overview of Flutter’s architecture in one of those ‘Intro to Flutter’ talks but somehow you weren’t ready to grasp the powerful concept behind all those different layers back then. Maybe you are like me and you just didn’t get it by simply staring at that abstract diagram for the exact 20 seconds the slide was projected onto the wall. Don’t worry, I’m here to help. Have a look at the following graphic:

The different layers of the Flutter framework. For more detail on this, head over to https://flutter.io/docs/resources/technical-overview

The Flutter framework itself is composed of many layers of abstraction: At top there are the commonly used Material and Cupertino widgets, followed by the more generic Widget layer. Most of the time you’ll find yourself using widgets from those two layers and that’s perfectly fine. The probability that you have already seen (and used) one of them out there in the vast wild should be quite high (think of the Scaffold or the FloatingActionButton from the Material library or the Column and the GestureDetector from the widgets library).

Below the widgets layer you’ll find the rendering layer which simplifies the layout and painting process and is another abstraction for the dart:ui library at the bottom. dart:ui is the last Dart layer which basically handles the communication with the Flutter engine.

To put it simply one can say that the higher levels are easier to handle whereas the lower ones give you more fine-grained control with added complexity.

1. The dart:ui library

The dart:ui library exposes the lowest-level services that Flutter frameworks use to bootstrap applications, such as classes for driving the input, graphics text, layout, and rendering subsystems.

So basically you could write a ‘Flutter’ app by just instantiating classes from the dart:ui package (e.g. Canvas, Paint and TextBox). However, if you are familiar with directly painting onto the canvas, you know that everything that goes beyond painting a still image of a stick figure will be a pain to manage. And then think not only about painting but also about orchestrating the layout and hit-testing the elements of your app.

So what does this exactly mean? It means that you would have to manually calculate all coordinates used in your layout. Then mix in some painting and hit testing to catch user input. Do that for every single frame and keep track of that. This approach may be manageable as long as you plan on building a simple app which just displays some text centered within a blue box, but not so great if you try to build more complex layouts like a shopping app or even a small game. Don’t even dare to think of animations, scrolling or other fancy UI stuff we all love. I am telling you based on my own experience, this is endless fuel for developer nightmares.

2. The rendering library

The Flutter rendering tree. The RenderObject hierarchy is used by the Flutter Widgets library to implement its layout and painting back-end. Generally, while you may use custom RenderBox classes for specific effects in your applications, most of the time your only interaction with the RenderObject hierarchy will be in debugging layout issues.

The rendering library is the first abstraction layer on top of the dart:ui library and does all the heavy math work for you (e.g. keeping track of the calculated coordinates, etc.). In order to do that it uses so called RenderObjects. You can compare RenderObjects to the engine of a car — they are the components that do the actual work to bring your app onto the screen. This tree composed out of RenderObjects will later get layed out and painted by Flutter. To optimise this complex process Flutter uses a smart algorithm to aggressively cache those expensive computations in an intelligent way to keep the amount of work on every iteration minimal. Amazing.

Most of the time you’ll find Flutter uses a RenderBox instead of another RenderObject. That’s because the people behind the project realised that a simple box layout protocol works very well to build performant UIs. Think of every widget placed in it’s own box which is calculated and then arranged with other pre-laid-out boxes. So if only one widget of your layout changes (e.g. a button or a switch), only this relatively small box needs to be recomputed by the system.

3. The widgets library

The Flutter widgets framework. What else?

The widgets library — probably the most interesting layer — is another layer of abstraction which provides ready-to-use UI components we can simply drop into our app. All widgets you find in this library also fall in one of the following three categories which are handled by the appropriate RenderObject for you:

1. Layout. E.g. Column and Row widgets which make it easy for us to align other widgets vertically or horizontally to each other.
2. Painting. E.g. Text and Image widgets allow us to display (‘paint’) some content onto the screen.
3. Hit-Testing. E.g. the GestureDetector allows us to recognise different gestures such as tapping (for detecting the press of a button) and dragging (for swiping through a list).

Typically you will use many of those ‘basic’ widgets and compose your own widgets out of that. As an example you could build a button out of a Container which you wrap into a GestureDetector to detect a button press. This is called composition over inheritance.

However, instead of building every UI component by yourself, the Flutter team has created two libraries which contain frequently used widgets in the Material and Cupertino (iOS-like) style.

4. The Material & Cupertino library

Flutter widgets implementing Material Design & the current iOS design language.

Flutter is all about abstraction and making your life as a developer easier. This is the fourth level which contains pre-built elements from the Material design specs and some recreated iOS-style widgets. Think of AlertDialog, Switch and FloatingActionButton. If you are an iOS user, CupertinoAlertDialog, CupertinoButton and CupertinoSwitch should look familiar to you.

Putting it all Together

How are RenderObjects connected to Widgets? How does Flutter create the layout? Or what is an Element?

Enough with the talking, let’s start walking and see how things add up in real life! Consider the following (simplified) widget tree:

In the real world widgets like Text are composed out of many other widgets. To keep things simple, we introduce the phantasmal SimpleContainer and SimpleText.

The app we are building is pretty simple for now. It just consists out of three stateless widgets: SimpleApp, SimpleContainer and SimpleText. So what happens when we hand it over to Flutter’s runApp(SimpleApp()) method?

The first time runApp() is called, a bunch of things happen in the background:

1. Flutter will build the widget tree containing our three stateless widgets.
2. Flutter walks down the widget tree and creates a second tree which contains the corresponding Element objects by calling createElement()on the widget (…what are Element objects again? Hold on, we get there in a second!).
3. A third tree is created and filled with the appropriate RenderObjects which are created by the Element invoking the createRenderObject() method on the corresponding widget.

Here is a picture of what the current situation looks like after Flutter went through the three steps described above:

A forest of trees. Back to the roots of abstract infographics!

The Flutter framework has created three different trees, one for the widgets, one for the elements and one for the render objects. Every Element holds a reference to a Widget and RenderObject. “Hey, I know Widgets but what are Elements and RenderObjects?” I hear you ask.

The RenderObject contains all the logic for rendering the (corresponding) actual widget and is quite expensive to instantiate. It takes care of the layout, painting and hit-testing. It’s a good idea to keep those objects in memory as long as possible and maybe even recycle them (since they are quite costly to instantiate). That’s where the Elements come in. Basically, they are the glue between the immutable Widget tree and the mutable RenderObject tree. Elements are principally objects that are really good at comparing two objects with each other, in our case the widget and the render object. They represent the use of a widget to configure a specific location in the tree and keep a reference to the related Widget and RenderObject.

Why is it such a good idea to have three trees instead of one? The short answer is it’s really performant. Every time the widget tree changes Flutter uses the tree of Elements to compare the new widget tree with the already existing RenderObjects. When the type of a widget is the same as before, Flutter does not need to recreate the expensive RenderObject and just updates its mutable configuration. Since Widgets are very lightweight and cheap to instantiate they are a perfect for describing the current state (also referred to as ‘configuration’) of the app. The ‘fat’ RenderObjects (which are expensive to create) are not recreated every time and reused whenever possible. As fellow Simon pointed out, “The whole app acts like a huge RecyclerView”.

However, in the framework those Elements are very well ‘abstracted away’ so you won’t have to deal with them very often. The BuildContext passed in every build(BuildContext context) function is actually the corresponding Element wrapped into the BuildContext interface and that’s why it’s different for every single widget.

Computing the Next Frame

Since Widgets are immutable, with every configuration change the widget tree needs to be rebuilt. When we change the color of our container to red, a rebuild will be triggered by the framework which will recreate the whole widget tree since it is immutable. Next, with the help of the Elements in the element tree, Flutter will compare the first item in the new widget tree with the first item in the render tree, then the second item in the new widget tree with the second item in the render tree and so on.

Change the container color to red. Nothing special here.

Flutter will follow a basic rule here: check if the old and the new widgets are from the same type. If not, remove the Widget, the Element and the RenderObject from the tree (including subtrees) and create new objects. If they are from the same type, just update the configuration of the RenderObject to represent the new configuration of the widget and continue travelling down the tree.

In our example, the SimpleApp widget is the same type as before and has the same configuration as the appropriate SimpleAppRender object, so nothing will change. The next item in the widget tree is the SimpleContainer widget but with a different color configuration. As the SimpleContainer still needs a SimpleContainerRender object in order to be drawn, Flutter just updates the color attribute on the SimpleContainerRender object and asks it for a redraw. The other objects will stay untouched.

The state of the three trees after rebuilding the widget tree and updating the config of the RenderObjects. Notice that the Element and RenderObjects are still the same instances.

This process is fast because Flutter is really good at creating those simple widgets which just represent the current configuration of the app. The ‘heavy’ objects will stay untouched until the corresponding widget type is removed from the widget tree. What happens if the type of a widget changes?

The SimpleText got replaced by a SimpleButton.

Again, Flutter will iterate over the newly built widget tree and compare the type of the widgets with the type of the RenderObjects in the render tree.

The new widget tree. SimpleText, the corresponding Element and SimpleTextRender have been removed from the trees.

Since the SimpleButton does not match the type of the Element at the position in the element tree, Flutter will remove the Element and corresponding SimpleTextRender from the two other trees. It then continues to traverse down the newly created widget tree and instantiates the appropriate Elements and RenderObjects.

The final trees.

And bingo! The new render tree has been built and will now be layed out and painted to the screen. For that Flutter will make use of a lot of optimisations and aggressive caching strategies so you don’t have to take care of them manually. Pretty cool, ugh?

Conclusion

Now you should have a clearer picture in your head about how Flutter actually works and why it is relatively fast in rendering even complex layouts. We could go on and discuss how bringing in the State would make the whole process even more performant but unfortunately I have to tell you that this is it for today (will cover that topic in a an upcoming post — promised!). I hope this article has helped you understand the architecture of Flutter a bit better and I would really like to hear your feedback! Follow me on Twitter for some more Flutter related things ;-)

Frederik Schweiger (@flschweiger) | Twitter

Flutter Community (@FlutterComm) | Twitter