Migrating 500k lines of Flow code to TypeScript

A look into migrating a large codebase from one type checking tool to another.

Foreword

This article is not meant to justify why you should migrate from Flow to TypeScript. I’m assuming you’ve done the upfront work of making this decision on your own. If you want to talk shop on Flow vs. TypeScript, hit me up on twitter @_MichaelSholty. My goal of this article is to empower other teams to be able to make the switch if they choose to do so.

Introduction

So, you and your team made the decision to migrate from Flow to TypeScript, but now what? This is not an easy task.

If you were looking to migrate from pure JavaScript to TypeScript, it’s not so bad. There are plenty of articles out there that document this process. Many teams have gone down this path and have lived to tell about it. There’s little risk involved here. Using allowJs allows you to “get started” without changing a single line of code, then iteratively migrate from there. TypeScript is an opt-in feature, so if you start the migration process and decide TypeScript is not for you, you can undo it relatively easily.

What if you’re already statically typing your code with Flow, and quite far down that rabbit hole? Even though the two tools offer similar features, TypeScript cannot read Flow syntax, and vice-versa.

If you work in a large codebase, you’ve already come to terms with the fact that you’ll need to migrate your codebase iteratively. This means you need to keep both TypeScript and Flow happy, and ideally working together.

Paving New Roads

There isn’t a lot of documentation on how to migrate from Flow to TypeScript, but I did find one article. However, there is one glaring issue:

Easier said than done!!

I quickly realized that this endeavor I was about to embark on was a path few have traveled down. In the very least, few people have documented their journey. That changes now!

Migration 101

I’ve migrated a small codebase from Flow to TypeScript before. Depending on things like lines of code, complexity of dependency graph, and risk/pain of your code causing outages, you may be able to migrate your repository in one go. But let’s be honest. If you could have done that you wouldn’t be reading this article right now would you?

If you have a medium-to-large codebase, I pray to the TypeScript gods that you do not hand a teammate a PR where you migrate hundreds of thousands of lines of Flow code to TypeScript and expect them to review it in good faith. It ain’t gonna happen.

At Zapier we have about 500k lines of Flow/JavaScript code (and that’s after you strip out empty lines and node_modules code). We need an iterative way to do this work. More importantly, we need an automated way.

One early thought you may have is that you can identify the leaf nodes in your project’s dependency tree and migrate from the bottom to the top. This is a great plan, until you realize everything depends on everything else.

Maybe this is how `yarn` got its name.

Migrating one part of your codebase from Flow to TypeScript will just cause the entire ball of yarn to unravel. Flow and TypeScript do not work together at all out-of-box! How can you migrate just one part of your codebase without bringing down the house?

I’m going to use specific tools and organization patterns in my examples. My goal is to convey the details of this migration as I experienced it. You may not be using the same tools, so take what you can from this article and adapt to your own codebase. The concepts will still hold true.

A Quick Note On Flow <> TypeScript Compatibility

I wanted to make a point here that was hard to fit in any other section. If you are using Flow, you’re using a babel preset/plugin to strip Flow out of your code when you build your application. If you want to start writing TypeScript, you only need to add a few more lines of configuration to make it happen. Add an override to your babel configuration to handle TypeScript code appropriately (see example below).

This will make your application work when executed, barring any additional webpack loader configuration you need to add. The issue we are solving here is not making your application execute correctly, it’s making Flow type checking compatible with TypeScript while we are in the midst of a migration.

The end goal of course is to uninstall Flow from our codebase and be 100% TypeScript 😃

Build the Robot: How to Automate Migrating A Single File

If only there was a machine that could automate moving these leaves around…maybe by blowing air at them…

Migrating code from Flow to TypeScript manually is about as exciting as raking leaves. Good luck to you if you choose to go down this path.

At Zapier, one of our core company values is “Don’t be a Robot, Build the Robot”. Let’s build the robot to do this work so we can spend our time doing more valuable things.

Luckily for us, there is a babel plugin that was written specifically for this job: babel-plugin-flow-to-typescript. Here’s a very quick-and-dirty node script that utilizes this plugin to transform code:

It’s important to give a file the `.tsx` extension if it has JSX syntax. Otherwise, TypeScript will not be happy.

This script will take transform any Flow syntax into TypeScript, and will create the new .ts or .tsx file. There will be minor things you need to touch up in the files processed this way, like whitespacing, but ultimately it will get you most of the way there.

Generate an index.js.flow file

During the migration process you’ll likely run up against a situation where a newly generated TypeScript file is imported by another Flow file, and you’ll want both Flow and TypeScript to both work as intended. They need to be in agreement.

This is essentially what we’re going for with our iterative migration.

Even if you begin your migration from the bottom of your dependency tree, you’ll still run into a situation where your shiny new TypeScript file imports a Flow file.

There is an easy way to get TypeScript to be agreeable with the Flow. For example, if you have a packages/utils/src/capitalize.js file that some TypeScript code depends on, you can create a packages/utils/src/capitalize.d.ts file that looks like this:

I am quite amused with myself that I created a one line gist.

On the other hand, if you import a TypeScript component in a Flow file, it will cause Flow’s type checking to break so the component is of type any. You may even get new Flow errors that weren’t there before. We need to generate an index.js.flow file to pair with this migrated code so Flow knows what to expect. This file will contain Flow type definitions for the associated TypeScript files.

Again, you could do this manually, but what if someone makes a change to the TypeScript code post-migration? That contributor could forget to update the index.js.flow file and accidentally break something. Even worse, Flow may let it slide silently!

So how do we generate the index.js.flow? There are two steps involved:

1. First, generate a .d.ts file with all of the contents of the package’s publicly exported types. I am using dts-bundle-generator, however in hindsight I think this can be done with tsc --declaration --emitDeclarationOnly combined with concatenating all .d.ts files generated through a cat file1.d.ts file2.d.ts command. Your mileage may vary.
2. Generate a Flow definition file for the .d.ts file generated on step 1. I found flowgen able to do the job, albeit with some quirks that made me need to rework a few type definitions in my TypeScript code.
3. Tell Flow about this file so it’s aware. In your .flowconfig you will need to add this file under your [libs] section. I tried adding module.name_mapper.extension='ts' -> '<PROJECT_ROOT>/index.js.flow to my .flowconfig but it didn’t seem to work as expected, but I am migrating off of a relatively old version of Flow.

Automating the index.js.flow Generation

While the code migration doesn’t need to be done more than once, you’re going to want to regenerate these index.js.flow files every time the underlying TypeScript code is changed. Remember — the contents of the index.js.flow file represent the type constraints on that code itself. If you add a new prop to a component and don’t update the index.js.flow file, you now have two realities for how that component will be consumed. You’re going to run into an error in Flow or worse, production! ☠️

A way to solve this is to make sure the index.js.flow files are generated whenever something changes. I found the best way to do this is to have a pre-commit hook run on the developers machine so code never even makes it up to origin with incorrect Flow typings. You can even loop through the staged files and whatever command you come up with for each file. If files are organized together in a package, you can run it once for that entire package. Just make sure your index.js.flow files are always in sync. If you believe someone may commit some code with --no-verify to bypass this, add a step in CI that runs this command and checks to see if there is a diff between the output and what is in master.

Now that we’ve figured out how to migrate Flow code to TypeScript, and generate an index.js.flow for that file so no type constraints are broken in existing code, let’s talk about migration strategies.

Strategy 1: Migrating one Lerna/Yarn Workspace Package At a Time

Using lerna or yarn workspaces to organize your code is a common pattern that lots of projects are using these days.

Here’s an example project layout:

I feel like every project has a `utils` folder. It’s like the junk drawer I had in my house growing up.

Even in this simple example, you can imagine a scenario where all packages rely on utils, most rely on ui-components, and state-or-entities-or-whatever could have circular dependencies with utils. This stuff gets gnarly fast.

One strategy we discovered is to migrate one package at a time.

If a package has an index.js that exports all publicly facing exports (functions, variables, etc), you will need to generate one sibling index.js.flow for that file. This file will represent the Flow-parsable type representations for your TypeScript code. This may also help your mental model of knowing what is TypeScript and what is Flow in your codebase.

Big indicators that Strategy 1 is a good plan to migrate a package:

1. It has relatively few public exports in the top-level index.js
2. It’s small enough that you could imagine reviewing the entire package in one sitting without a gun to your head. If you’re not sure about that, run the migration script on the entire package and take a look at the diff. Would you expect someone else to review that in good faith?

If that’s the case, go for Strategy 1 to migrate that package.

If you have a package that is massive in size with hundreds of files, perhaps something like ui-components, then this strategy isn’t going to work. Remember, sending a PR to people for review with tens of thousands of lines of code changed is not ok and will result in a lot of stuff being glossed over. You’re creating unnecessary risk here. There is a better way.

Strategy 2: Iterative Migration of One Package

If you find that you have one single package in your codebase that is too large to migrate in one go, you’ll need to iteratively migrate this package.

You’ll need to set up a bit more infrastructure to simultaneously support a package in Flow and TypeScript. I want to stress that this is not a long-term solution. I would never suggest a codebase persist as both Flow and TypeScript, this is simply an ends to a means to achieve the goal of being 100% TypeScript with 0% Flow code. The biggest win with making a package compatible with both Flow and TypeScript is that it encourages atomic changes to the package that are easy to review and digest on their own. There is no technical advantage to running both Flow and TypeScript in the same codebase.

To get an individual TypeScript package to be compatible with your Flow code, harken back to the part of this article where we generated a index.js.flow file for that package. Now, instead of adding this to your .flowconfig under [libs], you can use this as your entrypoint to your package 🤯 This means going into your package.json and changing main to reference this dynamically generated file. The only thing you need to do to this dynamically generated file after it’s created with flowgen is to append export * from ‘./index.js; export * from './entry.ts';

What does that do? It makes your dynamic entry point import/export all your TypeScript components in entry.ts, and also import/export all your Flow components in index.js. Above those two lines will be all of the dynamically generated output of flowgen to statically type all of the TypeScript code in entry.ts to keep Flow happy.

With every component migrated from Flow to TypeScript, you can go into index.js and delete the import/export, and subsequently move it into the entry.ts file. All of your components are still accounted for, and the TypeScript ones have Flow typings defined in the dynamic entrypoint.

When you run dts-bundle-generator for this package, run it on entry.ts to make sure you only generate typings for TypeScript files (or whatever is imported/exported from that file).

A Means to an End

This is helpful if you have a package that offers many exports in the top-level index.js, and would be inconceivable to review in one Pull Request. The awesome part here is you can ideally open one PR per component/file. This makes each PR atomic and easy to digest. If you happen to break something in the migration process, you can revert just that component and not impact the rest of the package.

The downside to this method is that it requires additional infrastructure to work correctly, and will draw out the migration process longer. Talk to your team and decide for yourself how you will migrate each part of the codebase — there is no Medium article out there that will give you the perfect advice for your use case.

Conclusion

I hope this article helped you wrap your head around the monumental task of migrating a large codebase from Flow to TypeScript. If you have any detailed questions about it, please reach out to me on Twitter at @_MichaelSholty. Your own experiences may help me update this article so it’s more detailed for others to reference.

Thanks for reading!