An invasive surgery: upgrading a full-stack Dart application to Dart2 / Angular5

At Skawa Innovation we like to take the road less traveled. Living on the bleeding edge (technology wise) is not uncommon for us. We have been early adopters of many new technologies, Dart being one of them. Easyling, our website translation platform contains traces of Dart since the beginning of 2014.

In 2017 we were contracted to create a modern web application helping mothers to get in shape after giving birth. The contract came from the “Gyerünk, anyukám!” — loosely translated “‘C’mon, Mum!” — movement in Hungary. The owners already had a website, but it was not meeting their expectations neither feature-, nor performance-wise. We were given the green light to create a new platform from scratch — just the dream of every development team.

Nóri, founder of the movement with her boys

Our team was long waiting for an opportunity to utilize Dart both on the front- and the back end. We were aiming for a highly scalable, high-availability solution, and chose the following stack:

1. Google Datastore for persistence
2. Google App Engine Flexible Environment with Dart runtime for back end
3. Angular Dart front end

Using Dart on both the front- and the back end opened up the possibility for extensive code sharing. We used shared message objects passed between client and server, serialized and deserialized by Dartson, which saved us the tedious work of creating serialization boilerplate for each class.

Google claimed that for some of their internal Dart projects (like AdWords and AdSense) development teams reported up to 100% increase in productivity, solely attributed to Dart language features. I don’t know if we could claim such a productivity boost, but in our project Dart certainly felt easy and natural — it seemed like just the right tool for the job.

6 moths and ~125k lines of source code later, on January 4, 2018, we went live with the new application.

We used Dart 1.24 and Angular 3.1 for our development, and due to tight deadlines couldn’t allocate time to upgrade to Angular 4. After going live, the next couple of months were still dedicated to intense bug fixing and rolling out new features that before launch were pushed back to the backlog. It was early summer when we finally caught our breath, and had enough time to plan for the inevitable foundation layer upgrade.

Angular 4 was appealing, but a much bigger promise was hovering on the horizon: Dart 2 with its sound type system, and Angular 5 with many sexy features like generic components, better tooling and significantly smaller production code, when compiled JavaScript. Weighting all the pros and cons, we opted to wait for the new versions — the possible benefits were so overwhelming that it seemed simply stupid not to.

The summer went by, as we checked the status of open issues for both Dart 2 and Angular 5 daily. There were times when there was only a single issue remaining for the 2.0 milestone, but the next day our hopes for a stable version were shattered by newly found critical issues and postponed due dates.

Finally, late August, both Dart 2.0 and Angular 5.0.0 debuted in the stable channel. We felt like racehorses kept for too long in the stable. We wanted it all, and we wanted it immediately. Our estimation for the total upgrade effort was two weeks. We got the approval. That’s where it all started.

The first bumps

There are two quick guides on how to execute the upgrade (here and here) which gave us a good starting point. Our strategy was roughly along the lines of

1. Update dependencies
2. Eliminate all errors discovered by static analysis
3. Rewrite conceptually deprecated parts (like transformers)
4. Create a build
5. Pass tests and deploy to a development server through CI
6. Do a full regression test
7. Roll out

As a first step, we updated all dependencies, and there came the first surprise. Angular 5 was stable, but a very integral part of Angular, the router, was only at alpha state. This was a clear warning sign that maybe — despite being in the stable channel — the products weren’t so stable after all. But our initial momentum and optimism laughed in the face of experience, and we thought, how bad could it be? We just assumed that the router will soon reach stable, so we went on with the upgrade. It has been four months since, and the router is still alpha.

After all the dependency updates and running pub get, the analyzer painted the whole landscape red with errors, which was expected. What was unexpected though, the 5000+ errors and similar amount of warnings, hints and lints crashed the analyzer server quite frequently, which was working against our imminent goal — i.e. to eliminate all static analysis errors.

There were many easy fixes, and we were grateful for those. Renaming JSON.encode to jsonEncode or Duration.MILLISECONDS_SINCE_EPOCH to Duration.millisecondsSinceEpoch required little thinking but eliminated a bunch of errors, just by doing a search and replace. There is even an upgrade tool to do most of these replacements automatically for you (we didn’t use it though). Apart from the search and replace operations, there were other low hanging fruits. Take, for example, the syntax change of int.parse — for input-safe parsing, in Dart 2.0 we would write int.tryParse(value) ?? 0 instead of int.parse(value, onError: () => 0). These light changes were done quite effortlessly.

The heavy lifting

A lot more effort went into adjusting our codebase to fundamental, conceptual changes.

Angular routing was turned upside down between angular_router package version 1 and 2. We had several dozens of modules with their own sub-routes, router links cross-linking the whole site, and so on. For each module we had to throw away the old, annotation based route configuration, create new classes (for RoutePaths and RouteDefinitions, replace old array based RouterLink configurations. The new router relies on generated factory constructors instead of class references. Router lifecycle callbacks have changed. Classes our code depended on were removed (Instruction and RouteParams, for example).

This was all anticipated, but we still managed to significantly underestimate this part of the effort. What we did not anticipate was that the matrix url notation that we used for optional parameters in 1.0, just disappeared without any trace from 2.0. Well, not without any trace, because the documentation mentions it, but unfortunately it isn’t implemented. Not cool at all.

The new sound type system also gave us a lot of work on both the front and the back end. We thought that we were type-disciplined enough already with our Dart 1.24 code, but Dart 2 proved us wrong. Later on we ran into many runtime type exceptions, especially when passing around composite objects with nested generic types — we needed to tighten many function signatures and introduce additional template parameters to comply with the new, strict regulations. For some generic typed classes, we started to use the covariant keyword which came very handy allowing certain type overrides.

Transformers were gone, the whole concept replaced by the new build system. Again, we knew that when preparing for the upgrade, it just turned out to be a bit more actual work than planned. We had some custom transformers, like inserting version information and Google Tag Manager code snippet into our index.html, and all those had to be re-implemented as build steps.

Dartson, that we used for serialization, was also gone. Although there is the official json_serializable package, we weren’t too keen to adopt it. Mostly because it would have made necessary to introduce build at the back end. And our back end was doing fine without any build steps, simply running on Dart VM, and doing all the serialization magic through the dart:mirrors library. So, in the end, we rolled our own serialization library, that used mirrors on the back end, and code generation on the front.

Extensive use of generated code references in Angular 5 took its toll on us as well. What used to be a simple class reference, became a reference to a generated factory constructor. Imports had to be supplemented with further imports of compiled code. Bootstrapping logic has significantly changed. All this, of course, in favor of better tree-shaking and producing smaller code, but it still can get confusing coming from earlier versions of Angular.

There were many other parts of our code we had to update — I’ll give you some git statistics at the end of this article. Some changes were straightforward, some less so. For example: do you remember the @Injectable() annotation on classes? Did you know that now it shouldn’t be used any more as it can increase the code-size of your app somewhat dramatically? But do you also know that if you use legacy bootstrapping, then you still have to use the annotation? So many questions. I think we are not the only ones who found the answers by many iterations of trial and error.

Despite all the above difficulties, the upgrade process was more or less in line with our expectations, except for the, how to put it nicely, perceived immaturity of the whole platform.

From the rock bottom of my heart

As we moved deeper and deeper into the migration, we started to hit SDK bugs. And builder bugs. And analyzer bugs. And dart2js bugs.

All in all, the environment was adding another layer of challenge to an already stretched situation. I think the low point came when we already had a successful local build, but our CI pipeline failed the front end build task about 80% of the cases. I’m not kidding. Most of the times it failed with the very informative dart2js exited due to unknown reason message. But occasionally it managed to finish without errors and produce the build artifacts. I was sitting over the failed pipeline, pressing the job restart button several times, hoping for the version to finally crawl out to our development server, and fantasizing about a different career in either coal mining or human resources.

As we later tracked down, the underlying issue for the failed builds was that that particular version of dart2js was a memory hog. It simply ate up all the available memory on the 8Gb Kubernetes nodes trying to compile our Angular application. The flaky behavior was due to Kubernetes’ non-deterministic node utilization, leaving sometimes more, sometimes less available memory for the container, which proved to be the breaking point for dart2js in this particular case. We redirected this job to a high memory dedicated VM and suddenly we had a consistently working pipeline.

We were long past our originally estimated two weeks, but nowhere near going live, when time pressure and feature squeeze closed upon us. We needed to start rolling out new features on top of the old version. That meant an increasingly painful master merge into the migration branch after each release. We were developing new features with the sinking feeling of producing code that was already obsolete, and there wasn’t much we could do about it.

Looking back, I estimate that features we rolled out during this period had an average of 20% overhead due to the compulsory refactoring for Dart 2 and Angular 5.

The light fantastic

Somehow during all the new feature developments and merges back to the migration branch, we increasingly got closer to a working version of our upgraded application. We already had a working pipeline, a running development environment, and we kept hunting down and fixing runtime issues at the cost of extra hours we put into the project.

Meanwhile Dart 2.1 and Angular 5.1 appeared in the stable channel with a lot of bug fixes. So in the middle of our ongoing upgrade, we decided to up the ante and updated our dependencies to Dart 2.1 and Angular 5.1. This, finally, was a good decision. Nothing broke. Some things improved. Confidence grew.

Soon, we felt we were ready to go live.

Going into production with our upgraded application was a battle very easy to lose, and almost impossible to win — at least from our customer’s perspective. We didn’t add any feature to this release, so the best thing to happen was a big, fat nothing. We were particularly worried about browser support, as the docs stated:

The production compiler (dart2js) supports Internet Explorer 11 and the last two versions of the following browsers

and we knew that about 3–4 percentage of our user base didn’t satisfy this requirement. Despite all tension, switching over was a breeze. We caught a couple of early bugs by intensely looking at log files. Some of our users with really old mobile devices complained about broken features. Apart from that, after three months of punishment, we were finally in the clear.

Have some fries — here are some takeaways

I promised to give you some git statistics. Here is a single line that sums it up quite well, the result of a git diff --shortstat on before and after hashes:

1078 files changed, 14893 insertions(+), 28304 deletions(-)

During the upgrade, we’ve touched about 30% of our total codebase, and slimmed down the LOC total with about 10%. Front end was a lot more affected than back end. The reduction in the source code came primarily from utilizing component inheritance. Our admin UI benefited mostly from this — whereas before we had separate admin list components for all entity types, now we had a single generic component with type parameters and a shared template file. Non-minified Javascript code enjoyed about 10% reduction in size.

We could have mitigated migration risks by upgrading first to Angular 4, and only afterwards to Dart 2 and Angular 5. I don’t think it would have made all the difference, but it would have been certainly be the more conservative and sensible approach. A lesson was learned here.

Another conclusion we arrived at was that migrating to a major version we only read about in announcements and change logs is risky. We simply didn’t have on hand experience with the new Angular router and all the other cool features. It was hard to refactor code while, for weeks, we were unable to try it, lacking a build. I ended up creating an Angular 5 kitchen sink application where I started to test various new features of the framework. It was a lot easier to refactor our main application once I saw how a particular feature actually works.

I would say I will be a lot more cautious with our next upgrade. But wait a minute… What?! Angular 5.2.0 just got published? See you later, I’m off to upgrade again.