Converting Codecademy to TypeScript 3: Learnings and Next Steps

Previously: Part Two: Technical Changes

Learnings

Even within the first month of conversion, several better strategies were made immediately apparent. 20/20 hindsight is inevitable. We learned from our mistakes and worked to improve in our surface-level behaviors and underlying reasoning.

Don’t Change JavaScript

Every single bug we introduced as a part of the TypeScript conversion was from a JavaScript change we thought was safe. Every single one! We were in what Dan Vanderkam refers to as the Uncanny Valley of Types.

When changing a file, such as when converting it from one language to another, you can either:

• Manually test every changed line
• Rely on your build steps to catch bugs

At the scale of converting thousands of files, it’s not feasible to test every single changed line at a granular level while still expecting the conversion to finish in a reasonable time. The best you can do is start with decent test coverage and perform high-level feature passes in manual testing.

We learned the hard way that our test coverage was not particularly high in many places and making arbitrary JavaScript changes in PRs touching dozens of files was not safe. If you were a Codecademy user in 2019 and the site crashed for you, maybe that was from a TypeScript conversion? 😉

Someone could probably write a tool to flag changes to the output JavaScript in a TypeScript conversion. That would have been useful. Ah well.

The One True Way

Pick one way to do something and stick with it. Even for choices such as interface vs. type that practically never matter, it's good to have a single recommended approach. Differences confuse newcomers and introduce new, unnecessary topics for learning.

Declaring React Components

As an example, there are many different ways you can declare a React component in TypeScript. Do you use an interface and function? How about a type and a () =>?

We eventually decided on React.FC for React components. React.FC adds in a children?: ReactNode field to whatever props type you give it, which is useful for not having to manually redeclare your general children type.

Interface or Type?

As the above snippets suggested, we decided on type over interface altogether. We were already leaning towards types for advanced types, so it made sense to consolidate on only ever using one of those two nearly identical constructs.

The only places we still intentionally use interfaces are for modifying global declarations such as Window.

Minimal Type Declarations

Way, way back in the day, many TypeScript programmers decided to prefer the TSLint [typedef](<https://palantir.github.io/tslint/rules/typedef>) rule, which enforced adding TypeScript : type declarations in locations regardless of whether TypeScript would infer the type anyway. A few common reasons for that are:

• Many early TypeScript adopters came from typed languages such as C# that encouraged or required manual type declarations.
• TSLint’s typdef documentation doesn't recommend against it, which can be interpreted as a recommendation to enable it.
• IDE features show up closer to where functions and variables are declared with types, instead of when their inferred types are used incorrectly in other locations.

We very much discourage typedef and in general the practice of redundant type annotations. We would much rather use a rule against unnecessary type annotations.

• Any extra typing to make things work is extra difficulty for writing code. We don’t want to add extra time to reading or writing our web code.
• TypeScript is very good at inferring types, and in the last few years, has gotten to the point where its inferences are often better than what you would typically type out.
• Complex object literals are particularly annoying to write type declarations for when multiple are created inline in a function.

The TypeScript team now recommends against typdef:

At the risk of issuing a rare style proclamation from the TypeScript team, we strongly advise against the typedef rule. It's actively harmful in a lot of cases, and we're not going to add language features to work around a lint rule.

Note that per the last bullet in favor of explicit type annotations, we’ve occasionally added them for more complex variables such as test mocks of large Redux states. But that’s very rare.

DefinitelyConfusing

I have yet to observe a good way to introduce the conjoined concepts of .d.ts files and @types to a TypeScript newcomer. They are simultaneously necessary for any TypeScript project with many npm dependencies and require some thorough understanding of how a type system works. What is a module lookup algorithm in the type system? How do wildcard modules work? What does it mean for the type declaration to be wrong?

[source]

A single incorrect line in a .d.ts file can be disastrous for TypeScript enthusiasm if it interferes with someone's ability to get their work done. We found it was necessary to loudly declare a team preference for removing any troublesome @types/ definition when they caused trouble - thus any-casting the module's declaration.

To be clear: the issue is not that today’s TypeScript type declaration systems are bad or undocumented; it’s that their edge cases are fundamentally difficult to understand when you haven’t gotten a full handle on gradual type systems.

As a side note, we found our .d.ts format for Webpack-style asset imports to be helpful in explaining how they work:

@types/react Duplication

Has this ever happened to you?

node_modules/@types/react-dom/node_modules/@types/react/index.d.ts:2777:14 - error TS2300: Duplicate identifier 'LibraryManagedAttributes'.
2777         type LibraryManagedAttributes<C, P> = C extends React.MemoExoticComponent<infer T> | React.LazyExoticComponent<infer T>
                  ~~~~~~~~~~~~~~~~~~~~~~~~
  node_modules/@types/react/index.d.ts:2776:14
    2776         type LibraryManagedAttributes<C, P> = C extends React.MemoExoticComponent<infer T> | React.LazyExoticComponent<infer T>
                      ~~~~~~~~~~~~~~~~~~~~~~~~
    'LibraryManagedAttributes' was also declared here.

It felt like every time we updated a React-related package, there’d end up being two versions of the React global types.

Every. Freaking. Time.

What it’s like to investigate duplicate React typings issues the first few times. [source]

This issue is caused by multiple incompatible semver versions of the @types/react existing and then getting picked up by the TypeScript compiler. You can verify what's going wrong by checking your package-lock.json/yarn.lock for multiple @types/react version resolutions.

The solution is generally one or both of:

• Updating React-related packages to the latest all at the same time
• Hand-editing that lockfile (ugh) to only specify one version

A quick Google search for this issue shows that we’re definitely not the only ones.

Redux Ecosystem, UGH

Redux state on its own is a wonderful system to model in a typed language. State is really a series of type declarations; actions are a fantastic use case for discriminated unions; reducers usage of ... is fully supported by TypeScript.

But wow do your typings fall apart when you add redux-actions, redux-saga, redux-saga-test-plan, and the other seemingly neverending Redux helpers we cycled through. Each library on its own is generally fine, but combining them can be problematic. Does an Iterable from one library work as an IterableIterator in another? How do undefined verses void return values interact? How about the three to four subtly different Action type declarations? 😖.

What it’s like to investigate any complex issue in Redux. [source]

We found ourselves any-casting all over the place in our saga test files. Some battles are better fought after you've built up many more months of collective TypeScript experience than we have now.

Next Steps

Very few technological shifts are truly complete after their first implementation wave. We shipped an MVP and were pleased with the results. If 2019 was our year of adopting TypeScript, 2020 is our year of perfecting its usage. Time to iterate!

Project References

TypeScript added the ability to split a single TypeScript project into separate tsconfig.jsons that reference each other. This gives you drastically improved incremental type checking times, better enforced internal project dependency graphs, and the option to use different compiler settings for different areas of the codebase.

Our web platform team has been slowly chipping away at an effort to better map out project dependencies between our systems. For example, utilities that exist within the directories for our complex learning environment should not be imported by the user dashboard code; shared utilities have their own place. We use ESLint’s no-restricted-syntax rule in a few projects' .eslintrc.jsons for now:

Import restrictions will be much cleaner and much less manual-regular-expression-y once they’re enforced by the TypeScript compiler.

We’d also love to use different compiler settings for source files, unit tests, and end-to-end tests. Test globals such as describe() and it() shouldn't be available in code shipped to production. The Jest and TestCafe test libraries both declare different test globals and thus can't be included at the same time.

Any Casts

As of the week after finishing the initial conversion, we have a little over 600 as any and : any types in our code. A dozen are in .d.ts declarations, ~250 are in tests, and the remaining ~350 are in source files.

Additionally, running the TypeScript type checker with --noImplicitAny enabled reports roughly 2500 implicit anys. ~400 of those are in test files; the remaining ~2100 are all in source files.

We clearly have work to do in completing our type coverage. Our plan for cleaning our types up is to:

1. Enable the --noImplicitAny compiler flag — likely on a project-by-project basis once the aforementioned tsconfig.json project references are added
2. Enable the typescript-eslint no-explicit-any rule, with difficult-to-fix exceptions manually disabled using // eslint-disable comments
3. Clean up all those // eslint-disable comments

Automated --noImplicitAny Fixes

The TypeScript language service can provide a suggested fix to add a missing type to a parameter or variable that’s implicitly of type any. It's superbly useful for quick fixes during development. TypeStat is able to programmatically pull all those fixes in and repeatedly apply them.

Unfortunately, there are still many instances where TypeScript gives unhelpful, even bizarre type suggestions: #28991, #29321, #29322, #29324. We’d love to take the time to contribute fixes for those issues up to TypeScript. We will almost certainly not have the time for all of them, so please: if you’re reading this, we’d love it if you helped out! 🙌

More Linting

Our ESLint configuration is mostly leftover rules from before our TypeScript conversion. We still haven’t gone through the impressively extensive list of typescript-eslint rules. Each of those rules has a JIRA ticket in our backlog with a request to:

1. Investigate whether the rule would be useful for us, and if so, how many violations it would report if enabled
2. Depending on the number of violations: if there are only a few, fix them quickly; otherwise, file followup tickets per code area to fix a few violations at a time
3. Once the violations are all fixed, enable the rule overall

Split All The Things!

Our work splitting to get many team members involved in the late stage TypeScript conversions served us well. We already plan on using it again in all three of the any cast phases and lint rule enablements.

In Conclusion

No two technological shifts are the same, but these tenants helped us with a successful TypeScript conversion:

• Think deeply on technology changes — first on whether they’re worth it given your team’s pressing needs, and then how to best achieve them.
• Don’t forget to focus on internal knowledge sharing and evangelism, especially through entry-level tasks.
• Don’t hesitate to adapt your game plan in real time to minimize developer friction as you learn the technology.
• Figure out your user’s common questions and concerns, and establish easy patterns for resolving them.
• @babel/preset-typescript, typescript-eslint, and TypeStat were key to our successful TypeScript conversion.

If you’ve got a JavaScript codebase larger than a few dozen files, we’d highly recommend giving TypeScript serious thought. It might feel more daunting than it really is. Let us know if you have any questions or comments — we’d be happy to help!

💖, the Codecademy FrontendTeam