Translating 800K lines of Javascript to Typescript

For some reason, GitHub refused to display my diff.

Over the last two months, I’ve been translating Zoosk’s 800,000-line ES5 Javascript codebase into Typescript. The process was long and harrowing, and I crashed every single external tool I used during the process. But I persevered, and here’s how.

Planning

The most important part of any big project is planning. Here’s the breakdown of how our code was when I started:

• ~800,000 lines of Javascript written from 2008 onward
• No ES6 anywhere — all vars and functions
• No imports or exports — everything is output to the same place
• Type annotations added in JSDoc, thanks to our use of the Closure Compiler
• Code runs both in the browser and on the server via Node
• Files are built using the version of plovr from 2015

Not the most cutting-edge codebase. Here’s how I wanted it to look:

• ~800,000 lines of Typescript!
• All ES7: lets, consts, rest, spread, you name it
• All files should use imports and exports
• Code should still be deployable to both client and server

Swapping the syntax with gents

I like to say “a lazy programmer is a good programmer,” mostly because it’s almost always a good idea to take an already-existing solution. Typescript is the official language of Angular, and luckily the Angular Team maintains a tool called gents that translates @type-annotated JS into Typescript. I highly recommend starting here if you’d like to follow in my footsteps.

Don’t get too comfortable, though. Gents’s use case is so specific that the tool hasn’t had too much time to develop. It will help you greatly in that it will produce a legal-TS-syntax version of your codebase. It doesn’t guarantee that the output will work. Check the issues page before starting and submit an issue if you run into any trouble.

So after running my code through gents, I had 800,000 lines of Typescript. What I didn’t have was compiling code.

Automated fixes with Python

Robin$ tsc | wc -l  # Count the compiler errors
66213

Alright, so there are a few errors.

What do you do when you have more errors than will fit in 16 bits? You start automating updates to the code. Most of the time I’d recommend an AST-based solution, but for simple fixes it’s faster (both in writing and runtime) to use 15–20-line Python scripts.

Here’s some base code that you can use to get started.

One of the most common things that I did was use the replacement function option of re.sub in Python. Here’s a dumb example that just capitalizes its match:

def repl(match):
    return 'I am ' + match.groups()[0].upper()
re.sub('I am (\w+)', repl, 'I am hungry!')
# Returns 'I am HUNGRY!'

Of course, this is possible in JS as well. String.prototype.replace has the same functionality.

I used a ton of tiny scripts like the one above to make far-reaching edits to my codebase. Everything from removing duplicate imports to alphabetizing fields to correctly linking up our templating system was scripted in Python.

Building the code

The next big challenge was how to deploy. Our one big ask here was that we continue using the Closure Compiler for the browser code. This is due to the huge gains that advanced mode provides like dead code removal, inlining, and gzip optimizations.

The Closure Compiler is very specific about its input in advanced mode. Luckily the Angular Team came to our rescue again with tsickle, a wrapper around tsc that outputs advanced mode-compatible JS.

For me, getting started with tsickle was a headache. I ended up diving into the source of tsickle and fixing a couple bugs. Thanks to the magic of open source®, you won’t have to deal with those issues!

Our new build flow

On the server, the extra compile step was one that we already had: linking up the output of Soy template files to code running in Node. That step had to be slightly modified to match the new style of requires that tsc outputs.

Code quality improvements

One of the big reasons we wanted to move to Typescript was that it would help us adhere to a higher standard of code quality. We expected this to come in time (and more will!), but the initial migration actually forced us to shape up a bit. Here are some things that Typescript was not happy with initially:

• Usage of fields marked with @private outside of their owning class
• The same thing being imported twice in one file
• Enums declared as static class members instead of by themselves
• Using properties of this before calling super
• Child components importing and manipulating their parents

Additionally, the compiler found seven bugs that we had missed before, completely automatically!

Join us in the sun

If you’re thinking of migrating your large codebase over to TS, I think the benefits drastically outweigh the costs. It took me two months to do ours, but a good three weeks of that time was working out bugs in the open source ecosystem, between gents, tsickle, tsc, and the Closure Compiler. I’ve submitted PRs for everything, so you should be fine.

Additionally, I spent the first month of this project working all by myself, and gained one developer during the second. A team of three devs would have been able to do 800K lines in about three weeks.

Finally, if you’re thinking that you want to move over incrementally, you can actually use Typescript on JS files with almost no effort! This will give you the ability to introduce new TS files one at a time.

That said, if you want to work on Typescript without the work, you can always apply at Zoosk.