Under the Hood with Flutter for Web

An inside look at how we built the NY Times KENKEN App at Google I/O ‘19

https://www.nytimes.com/games/prototype/kenken

On May 7, 2019 the tech world watched with amazement as Google finally released the first public preview of Flutter for web (formerly known as Hummingbird). When it was first announced at Flutter Live in December 2018, it came as a shock to many who saw Flutter as purely a framework for building cross-platform mobile apps. Together with the announcement of Flutter Desktop Embedding, it was clear that the Flutter project was much more ambitious than we had previously thought.

With Flutter apps now capable of running on iOS, Android, macOS, Windows, Web, IoT, and soon Fuchsia, we are witnessing Google’s complete vision for creating the world’s first universal UI platform. This is a veritable game changer for application developers everywhere, who now can deliver applications for all these platforms from a single codebase. Not to mention that these apps are higher quality, built by fewer developers, and in drastically less time.

My name is Martin Rybak, and I am the organizer of the New York Flutter Meetup, one of the largest Flutter meetups in the world. I am also a co-organizer of the first International Flutter Hackathon on June 1 (be sure to join!). I have the privilege of working for Very Good Ventures, the company that built the Hamilton app and the Slides app from Flutter Live. When Google Executive Tim Sneath called us to see if we could build a demo app for the New York Times that would run on mobile, desktop, and web, we were thrilled. We would be one of the first teams outside of Google to ever use Flutter for web! Our challenge was to update the current Adobe Flash-built KENKEN app on The New York Times website.

We only had 48 hours to build a working prototype that would validate its feasibility, and we happily accepted the challenge! I’m happy to say that it worked, and over the next several weeks we worked closely with the New York Times team to explore and implement multiple designs, animations, and features. While we can’t share our final source code, we are happy to share our experiences and some of the challenges we faced.

First Impressions

We collectively held our breaths as we first ran Flutter for web for the first time. Watching the well-known Flutter sample app appear in a browser brought a huge smile to our faces — we were web developers again, but without the JavaScript! 😁 Looking at the example Flutter code, the first thing that stood out was that the web imports were slightly different than mobile:

import 'package:flutter_web/material.dart';
import 'package:flutter_web_ui/ui.dart';

Since Dart has no preprocessor, there was no compile-time way to share these with the standard mobile imports (Google’s goal is to eventually merge Flutter web into the main Flutter framework). Also, images and fonts had to go into a separate location on the filesystem and be declared in separate manifest files. So we decided to separate our Git repo into 3 branches: master (web), mobile, and desktop. To minimize merge conflicts, we decided to do all of our development off of master (web) and then periodically merge changes into the mobile and desktop branches. Any mobile- or desktop- specific code changes, such as import fixes, would go directly into their respective branches.

The final game board.

We had a basic working prototype in 48 hours, which is truly a testament to Flutter’s development velocity. One developer worked on the game board and the other on the overall layout and game play. At the core of the game is a Game model which validates and stores game data, maintains the timer, and detects wins. Game data is fetched from the backend API via a separateApiManagerthat uses the http package, which worked beautifully. The format of the API response was a little tricky, so we created a GameData parser to convert it into a more usable format. The Board widget then knows how to render itself using this data, and interacts with the Game model using touches, mouse, and keyboard events. The rest of the code consists of widgets and custom painters.

Debugging

To run a Flutter web app in debug mode with hot reload enabled, run the following command (provided .pub-cache/bin is in your PATH):

webdev serve --auto=restart

This launches a web server at http://localhost:8080 which you can open in any desktop browser. We used a tool called ngrok to access the site on our mobile test devices. Hot reload works exactly as expected, and maybe even better than on mobile! Flutter for web comes with a file listener that refreshes the browser whenever it detects any code changes. Unfortunately though, you cannot yet connect the IDE debugger to a running Flutter web app. However, amazingly, you can debug from within Chrome using DevTools. Flutter for web uses source maps which let you see your original Dart code directly in the browser! (Note: this is only useful in debug mode. Release builds use minification to obfuscate your code.) There you can set breakpoints, examine the call stack, and view and edit variables. You can also use the Network, Performance, and Memory tools to see how your app performs in those areas.

Dart source code, right in Chrome DevTools! 😲

Speaking of release builds, what exactly gets shipped in a Flutter web application? Turns out it’s just a single-page web app with all of your Dart code and package dependencies compiled into minified Javascript! This is thanks to Dart’s powerful and battle-tested Dart to Javascript transpiler. Pretty amazing!

It’s all just Javascript.

Performance

Flutter for web was still a bit raw so we did our best to report any issues. There were some problems involving text rendering and input, but drawing and animations worked amazingly well! (Under the hood, Flutter for web draws most elements to the canvas directly except text, which is rendered by the browser).

Originally, we built the Board widget using a GridViewcontaining customTile widgets. However, we ran into a few issues. First, there were visible artifacts between the tiles which is a known bug. Second, we ran into some slowness in the build() method that was a result of the large amount of text widgets we were using. To fix both problems, we opted to use a CustomPainter to draw the board lines and render the text directly onto the canvas using a TextPainter. However it was still slow for the animation-intensive end of game display. In release mode for the 8x8 grid we were seeing an average of 46ms for the paint() method. In order to prevent dropped frames, we had to get it down to less than 16ms. So we tried an unorthodox technique and converted the numeric characters directly into Path objects. Doing this dropped the paint() method down to less than3ms. The Flutter for web team has been working hard on improving performance, so it is quite possible that these optimizations are no longer necessary. Still, it’s good to know that if you run into any performance bottlenecks, you can always drop down to lower layers.

Keyboard and Mouse Input

Listening to data from the keyboard was a bit tricky. On Android and macOS we could use RawKeyboard.instance.addListener() but the codes for the various input keys were different. On Web we had to use window.onKeyDown.listen()from the html package, which means a code divergence between web and mobile. To solve these this, we created a KeyboardService class that acts as an abstraction layer for all keyboard input. At runtime, a factory constructor returns the correct instance for the current platform. This approach also supports the on-screen keyboard used by the mobile layout, so no additional code is needed. Mouseover support for Web, Android, and Desktop was added in a similar way, with differing implementations hidden behind a common abstraction we called HoverDetector.

Responsive Design

A big challenge for web compared to building for mobile is supporting variable browser window sizes. This is known as responsive design. Whereas on mobile there are a relatively standard set of screen ratios, a browser window can be resized to any height and width. How can you ensure that a Flutter web app looks good on a nearly infinite set of screen sizes? There are a few possible techniques:

• Use breakpoints. Using a root LayoutBuilder, you can change the layout of the whole app based on screen height/width. You can show a “tablet” version on wider screens and a “mobile” version on narrower screens. For example, on narrow screens we make the headers shorter as well as show the on-screen input keyboard.
• Scale content. To make a widget “scale” to fit any size, you can use a LayoutBuilder to make things like font sizes, border widths, shadows, corners, and margins all scale proportionally relative to a base size. For example, the game board scales perfectly no matter what size it is.
• Use scrollable containers. If scaling is not desirable, you can embed your widget in a scrollable container such as aSingleChildScrollView. The initial game selector screen behaves like this.
• Enforce a minimum size. If the widget’s size constraints are simply too small to show your content correctly, you can simply show some text asking the user to increase the size of their browser window. We will probably implement this in the future, as the game board is simply unusable past a certain size.

End Game Animation

One of the most eye-catching moments in the app is after the user wins a game, and we show a complex animation that converts the board into a rocket ship! This animation was originally designed in After Effects, and is implemented 100% in Flutter. This would have been a great candidate for Flare, but unfortunately it isn’t yet supported on Flutter for web. We collaborated with Simon Lightfoot to create this complex yet delightful animation. How did we do it?

The main transition is a standard Hero widget animation. When the modal appears, Flutter seamlessly transitions the source Board into a read-only version of the Boardat the destination. When the modal is dismissed, the animation is automatically reversed. However, Hero animations don’t work with PopupRoutenavigation routes (modal transitions), so we had to create a custom PageRoute subclass.

After the Hero animation completes, a number of animations kick off simultaneously. The background begins shooting stars at random locations, built with a CustomPainter. The flame effect then begins and animates to the left. This was also made with a CustomPainter that is simply rotated by 45° and anchored to the center of the board. Both the board and flames experience a brief overshoot animation before settling down with a persistent shake animation. Finally, the text flies in at an angle with a fade transition. These animations were all accomplished using the powerful AnimatedBuilder widget, which performs transforms on widgets efficiently without triggering a rebuild on every frame.

Final Thoughts

It was an absolute pleasure to have worked on Flutter for web so early on. We had the chance to work closely with the awesome Flutter team at Google and resolve various issues. Development continues at a breakneck pace, especially in overall stability and performance. Over the next few weeks we look forward to working with the New York Times team to bring this game, and perhaps more, to production soon.

That said, Flutter for web is still a work in progress. Here are some areas of improvement that we’ve identified:

• Web-specific code that requires the html package. Keeping separate web/mobile/desktop branches is tricky.
• No native debugger. While Chrome DevTools is great, working with the IDE debugger is a superior experience.
• Offline persistent storage that works across web & mobile.
• Dart isolates are not supported in Flutter for web. Javascript is single-threaded.
• Text input does not yet match platform conventions.
• Selecting/copying of text is not yet supported.
• The browser back button works just like the Android back button, but the forward button is not supported yet.
• Making Flutter for web more SEO-friendly.
• Printing with a @media stylesheet isn’t yet supported.
• Anything else you find? Please file an issue!

Special thanks to Albert Larzidabal, Kevin Gray, Simon Lightfoot, Kevin Moore, and Yegor Jbanov.

Very Good Ventures is the world’s premier Flutter technology studio. We built the first-ever Flutter app in 2017 and have been on the bleeding edge ever since. We offer a full range of services including consultations, full-stack development, team augmentation, and technical oversight. We are always looking for developers and interns, so drop us a line! Tell us more about your experience and ambitions with Flutter.