Migrating to TypeScript without Migraines
Our fearless dev team embarked on a voyage from Flow to TypeScript. How we avoided danger — and stress migraines — along the way.
If you were to send a text message in the early 2000s, you likely would have flipped open your phone at lightning speed and started pounding its keys in close succession just to say a simple, “How r u?” A new language was born. For the last two decades, global web commerce — including the sort conducted by Wealthsimple — has relied heavily on its own universal language: JavaScript.
But twenty years is an eternity in programming, and just as cell phones went from 10+ keys to none, JavaScript has begun to show limitations. In particular, its lack of static type checking can make it hard to manage, resulting in slower development and gnarlier bugs. Thankfully, there are two mature type-checking systems that rectify this problem for anyone doing JavaScript at scale: Flow and TypeScript. Each has its pros and cons.
Since 2016, Wealthsimple has been a Flow shop. This was largely decided by our early adoption of React Native for our mobile apps. Since React Native and Flow are both parts of the Meta (nee Facebook) ecosystem, it made sense to use them together. However, a lot has changed since 2016.
What began as a tight two-horse race ended with a runaway victory for TypeScript. Internal developers prefer TypeScript to Flow. New hires prefer TypeScript to Flow. Third-party tools are starting to require TypeScript instead of Flow. TypeScript’s relentless improvements in developer productivity, library support, and speed have made it clear that we backed the wrong horse.
But we were all-in on Flow, with over 40 repositories using it heavily.
As the painpoints grew with Flow, it became clear that we would need to migrate. We also knew it had to be a one-shot deal — we couldn’t ask developers to learn two different type systems. However, the prospect of migrating more than 700,000 lines of code across web, mobile, and node repositories full of interdependent libraries which seemed as daunting as trying to take a cute selfie in 2002. It would take a ton of time for our engineering teams to do this migration themselves, and one thing hasn’t changed since the heyday of the Motorola Razr — time is still money.
Everything changed this year when we built a Front-End Platform team. Finally, we had a chance to really dedicate some time, effort and brainpower to this problem. We knew that a lot of work and problem-solving lay ahead, but we also knew it would all be worth it if we could remove Flow from our JavaScript ecosystem for good.
Phase One: Playing whack-a-mole
Our first challenge was planning a migration that wouldn’t impact developer productivity. At Wealthsimple, the bulk of our JavaScript is on the front-end, where we follow a micro-frontend architectural pattern. We started with the advantage of not having to deal with migrating a monolithic codebase while our engineers were busy working on it. Still, we had an awful lot of repos to get through. It was obvious we would have to automate most of the process if we had any hope of maintaining our collective sanity. We also wanted to preserve the types that already existed in Flow.
Looking into community tooling turned up some promising options for converting from Flow to TypeScript, but playing around with these tools always left us with TypeScript code that wouldn’t compile because of the remaining type errors. And boy, were there ever a lot of type errors.
The problem came back to Flow. Because Flow’s type-checking is not as strict or as widely supported by third-party libraries, a lot of the types we had previously written wound up being incorrect or incomplete once we had converted them to TypeScript. It seemed like fixing these errors manually — all 3,000 of them in just one of our services — might be our only option. But what started as a large (though not impossible!) task turned into a game of whack-a-mole; we’d get the number of errors down to a workable threshold, only for 1,000 more to pop up.
For a while, we explored the option of removing all of the types we had written on Flow just to get on the new toolchain. The hunt for new tooling eventually led us to AirBnB’s ts-migrate. ts-migrate came with an ignore command that would automatically add @ts-expect-error comments, which suppressed type-checking errors. Presto! Finally, we’d found our magical middle-ground: We could keep all of our working types and declare bankruptcy on whatever was left by telling the compiler to skip over them.
Finally, we had an approach that would work. Now all we had to do was execute it.
Phase Two: The art of strategy
Once we’d charted a path forward, we needed to dig into the nitty-gritty details and figure out the best way to match our previous setup in a way that wouldn’t make the dev experience worse. Because TypeScript is compiled, whereas Flow is not, there were a lot of implications for migrating that went above and beyond changing the code. We had to rework the rest of our toolchain to make sure that every single repo was building, testing, linting and Storybooking, and that all of our libraries and micro-uis were still playing nicely with our container applications.
This work involved a lot of elbow grease. We took advantage of the opportunity to standardize all of the configuration across our JavaScript ecosystem, bringing it all into a js-toolbox repo owned by the Front-End Platform Team. The more we learned from working through all our core libraries, the more we were able to build into our standardized configuration.
This wasn’t a passive process — we had to go through each, one by one, and we had to be especially careful with the packages that everyone relies on, like our broadly-used component library. Once we knew how to do the conversion for a given repo, we wrote the steps into a script that could be run within our CI pipeline and continuously report whether or not the complete conversion was successful. This allowed us to run the script in unconverted repos and get ongoing feedback on the status of the conversion while we made changes and small fixes. It also allowed us to work on the migration while other team members were building features, avoiding the deadly “migration branch” that never works because of merge conflicts.
We hammered away on our core repos until they were all converting cleanly. Once that was achieved, we didn’t immediately cut them over to TypeScript. Instead, we used the CI script to publish both Flow and TypeScript packages of each library, which allowed developers to continue working in Flow ecosystems without losing type fidelity. It also enabled us to cut over any repos with dependencies on these libraries and immediately take advantage of TypeScript types.
With most things scripted and some key repos under our belt, we were ready to call in the cavalry.
Phase Three: The final push
Once we got through our core libraries, we were ready to ask for help. As we worked on developing our conversion approach, we kept everyone in engineering in the loop on our progress. With high-visibility core libraries converting, we had built up some momentum. People knew that we were close to the finish line, so they were happy to step up and help us cross it.
Our volunteer engineers took the script we had written and, with our support, modified it until the repos they owned were converting cleanly. When the repo was finally ready, the cutover ended up being pretty painless. We alerted teams when it was going to happen, committed the output of the script and any last-minute bespoke fixes, and when the team came in the next morning, they were writing TypeScript — we didn’t have to interrupt anyone’s workflow. While this is what we’d been planning for all along, it felt pretty great to have it go off without a hitch.
Today, I’m really proud of the work we did on this migration. What felt like a (relatively) painless cutover for our dev team was the result of months of planning, trial and error, automation, and compromising to accept imperfection. We’re still ironing out some of the details, but we’ve built a stable tooling foundation for the future. It probably won’t carry us through the next 20 iterations of the iPhone, but honestly, I’ll happily settle for five. And with fewer type-checking problems to worry about, I’ve got the headspace to daydream about other priorities, and finally stop playing whack-a-mole.
Written by Renee Blackburn, Staff Software Engineer at Wealthsimple, in collaboration with Sarah Bellstedt. Edited by Mark Adams.
Wealthsimple is a new kind of financial company. Invest, trade, save, spend, and even do your taxes in a better, simpler way. “Maker Stories” is an inside look at how we get things done. Interested in joining our team? Visit our “Work With Us” page to learn more and view open roles.
The content on this site is produced by Wealthsimple Technologies Inc. and is for informational purposes only. The content is not intended to be investment advice or any other kind of professional advice. Before taking any action based on this content you should consult a professional. We do not endorse any third parties referenced on this site. When you invest, your money is at risk and it is possible that you may lose some or all of your investment. Past performance is not a guarantee of future results. Historical returns, hypothetical returns, expected returns and images included in this content are for illustrative purposes only. Copyright © 2021 Wealthsimple Technologies Inc.
