Adding TypeScript to the Juniper Square Codebase

Whether you use TypeScript or not, you’re probably familiar with some of the benefits and reasons why teams choose to adopt it.

TypeScript allows you to introduce static types into your usually-dynamically-typed-JavaScript, which can help reduce errors in your code by catching bugs before they actually become bugs. It can increase productivity across large teams and codebases by giving type-hints, providing more structure, and helping with debugging and confidence when refactoring.

With our quickly growing engineering team, and our React frontend still getting off the ground, adding TypeScript to our arsenal was a no brainer.

I had the pleasure of tackling this project for my “new hire project” when first starting at the company. Here, at Juniper Square, a “new hire project” is a project that you’ve identified and that you get your first few weeks to work on. This project should have low (or no) business context required, and may be something that you found during your initial onboarding as a spot that could use improvement. Or perhaps some tool, process, library, etc. that you used at your previous job that could really benefit the team here.

In this post, I’ll be sharing more details around the 8-month, major engineering team effort to add the strictest type checking that TypeScript offers into an existing project that wasn’t type checked at all. You’ve just heard a bit about why we decided to take on this project, but I’ll dive in further to the incremental approach taken to reach a fully type-safe codebase.

An incremental plan

So, how exactly do you go about adding type checking into a codebase that isn’t type checked already?

If you just change all your files to be included, you’ll probably see a cool few thousand errors pop up. That’s not going to work. And if you try to fix them all before merging in any changes it would take forever, literally forever, because you’d be chasing a moving target as the rest of the team continues to commit new code into the repo.

Enter, the incremental approach. It’s not realistic to stop all feature work in order to add types and fix type errors, so we had to figure out how to do this incrementally. The first step was to make sure we weren’t introducing any new type errors.

At Juniper Square, our incremental approach consisted of the following steps:

1. Add and configure TypeScript as a dependency to the project
2. Use tsc to find files with existing type errors and add them to the excludes list within the tsconfig
3. Add a pre-commit hook and CI check to ensure type safety of new files
4. Incrementally fix existing errors! When a developer makes changes to a file, they remove it from the excludes list and fix the type errors within that file.
5. Add CI checks to ensure step 4 was being followed.
6. Once all errors are addressed, remove the excludes list (which should be empty now) and the related CI checks

Executing the plan

Configuring TypeScript

We started with getting TypeScript setup — adding it as a dependency, adding a tsconfig.json file to specify our settings, and using @babel/preset-typescript plugin for our compile step. This babel plugin basically strips the type annotations and just does standard bundling when building the code.

This step doesn’t actually “use” TypeScript though, it just sets up the ability to have types and transpile to JavaScript but is not actually doing the valuable part of type checking.

Ensuring type safety of new files

To ensure all new files were type safe, we set up a TypeScript check step and added a pre-commit hook. In the package.json we added a script yarn check-types that runs the TypeScript check command without emitting any code tsc --noEmit (alternatively, add noEmit: true to your tsconfig). This will run the type checks on all files covered in the tsconfig and print out any type errors if they exist.

We also added a step to our CI checks on pull requests to enforce type checking, to ensure we were always making strides in the right direction as a team.

With all of this in place, we could get to work incrementally fixing the existing errors.

What files to exclude

The main part of this incremental approach was coming up with the list of files to be covered in the tsconfig; the files to actually be type checked.

Instead of adding files incrementally, we decided to exclude files that had existing type errors. This approach allowed us to see how much we had left to fix and address them one file at a time.

TypeScript will type check all files that are imported by a specific file when checking it, even if these imported files are on the excludes list. In order to minimize the errors we needed to fix each time, it was important to keep files that imported many other non-typesafe files (e.g. index files) on the excludes list until we had ensured the type safety of the imported files.

The tsconfig file has a field for excludes which are file paths that it will not cover during the type check. To generate our excluded file list, we used the following command:

yarn tsc — listFiles | grep -Fv -e ‘node_modules’ | grep ‘error TS’ | sed ‘s|(.*||’ | sort -u

The above command spits out any file paths that contained a type error during a type check, and sorted them alphabetically. Any file that was already typesafe would not be excluded from future type checks.

Fix existing errors

Now, how do you go about actually fixing the files on the excludes list?

When engineers make changes to an existing file, they would remove it from the excludes list and fix the types before checking it in. While there were some exceptions (files with lots of imports of non-typesafe files), generally this was the rule.

In order to better understand which files we should try to tackle first and which ones should wait until the end, a member of the team created a script to analyze the imports in all the files. The thought was that there were particular core files used generously throughout the codebase that, if identified, could help us focus our efforts in certain spots.

The script printed out the analysis to:

• List excluded files by how many excluded files depend on them
• List excluded files which do NOT have dependencies in excludes (and are thus easy to remove)

Enforcing with CI checks

We added in checks for our CI to ensure that the types were being addressed.

First, we added a mandatory step to run the actual type checking; if a PR failed this check, it could not be merged.

Then we added a step to see if any files modified in the PR were still included in the excludes list:

- run:
name: Validate that changed files are not in tsconfig excludes list
command: |
cd “$HOME/main”
MERGE_BASE=$(git merge-base $GIT_PR_TARGET $GIT_LOCAL_HASH)
UNFORMATTED_FILES=$(git diff $MERGE_BASE..$GIT_LOCAL_HASH — name-only — diff-filter=ACDMR | sed -e ‘s/^jsq\/client\///’ | sort | comm -1 -2 — <(jq -r ‘.exclude | .[]’ $HOME/main/jsq/client/tsconfig.json | sort))
test -z “$UNFORMATTED_FILES” && exit 0
echo “Please remove these files from tsconfig excludes list if all type errors are fixed:”
echo “$UNFORMATTED_FILES”
exit 1

This check was a warning rather than a hard requirement. We operated on good faith, allowing exceptions for partial type fixes on files with tons of imports, hot fixes, etc. We didn’t want it to be fully blocking.

Conclusion

Overall, adding TypeScript allowed us to reduce our overall code size and reliance on defensive coding, and still remain confident that our code wasn’t broken. It has helped us to reduce our bug count, especially those around accessing properties on non-existing objects. It may have slowed us down for a while, but is speeding us up now. This project was a massive team effort that everyone contributed to. Kudos to the entire Juniper Square Engineering team!