Start with “why?”
Over time our app has grown quite a bit, and is now sitting somewhere at 150 thousand lines of code. At Hootsuite we invest a lot in the technical aspects of our products. We do that by always assessing whether what we are doing is sound from a technical point of view. Any major project must start from a problem definition, so let’s talk about what were our technical challenges.
This problem is somewhat compounded by using React. Since it is built on functional programming principles such as immutability and composition over inheritance, it encourages developers to treat components as small, reusable functions. In a medium or large application it’s typical to have hundreds, thousands (or even tens of thousands) of components. Data can flow through tens of components at a time.
Imagine having to read through a tree of a dozen React components with no information on what the data looks like at each step. This was what the world looked like in 2015 to us. Now, as astute readers will point out, React offers a feature that helps with this: PropTypes. But they are optional and quite difficult to maintain. We didn’t use them from the beginning, we adopted them a year later and started adding them to new components. In our experience they did not deliver enough information to balance the maintenance effort required. There are also more practical concerns: most IDEs do not attempt to parse PropTypes to offer hints, you need tools to enforce their correctness, and it is a limited type system, not robust enough for our needs.
Another problem we experienced was that, due to a lack of information about the data flow, it was difficult to perform basic refactoring. Moving files around, splitting large components into smaller ones, or renaming files — these operations were both time consuming and risky. Our team embraced unit-testing since the very beginning, so we could not crash our app by moving files around. But it was still time consuming.
This led to an interesting realisation: developers rarely refactor code unless it is cheap and safe to do so. The reality was that our code base was in a dire technical debt state. Developers were wary of making major changes for fear of breaking functionality or spending too much time on what they initially thought to be basic tasks.
Wow, writing code without any IDE suggestions is hard!
So to sum it up, we were facing three major challenges:
- Difficulty in following the data flow
- Expensive refactoring
- Lack of contextual information in IDEs
We did have a very critical restriction. Hootsuite is a business, millions of customers across the world depend on us, so we couldn’t stop development while we migrated our code to another language. The migration had to be gradual. We would write new code in our new language, while slowly rewriting older code, without interfering with the product roadmap we had to ship. So we made a shortlist of options, and debated them one by one.
Other options that we analysed:
- Dart: this language is becoming popular again due to Google investing in Fuchsia. Dart is its de-facto UI tech stack. The lack of a transparent roadmap and uncertain future meant a very clear no go.
- PureScript: this is Haskell for front-end development. Same arguments as using Elm. To be honest, it has an even steeper learning curve than Elm.
It was not easy to choose between these two given their similarities. We invested some time in research spikes to see first hand what development looked like in both cases. What we found was that Flow was more difficult to set up than TypeScript, and that the developer experience felt sluggish. This has improved though in the last year, so if we were to analyse them again today, this wouldn’t have made such a big difference.
We also looked at the communities around these two projects. Flow is a language that, at least a year ago when we ran this analysis, was being driven by Facebook in a very closed manner. Development was never transparent, there was no public roadmap, and very few people outside of Facebook were contributing to the project. TypeScript, in contrast, embraced open-source development since moving to GitHub a few years ago. They keep an up-to-date roadmap, accept outside contributions, and generally keep a very close relationship to the community. Anders Hejlsberg, a Microsoft fellow and chief architect, speaks at conferences (such as TSConf) throughout the year, keeping everyone in the loop on what features are coming next and how the project is performing.
So as you might have guessed, we chose to go with TypeScript. This was a pretty long intro on why we did it, but it serves as a good template for other teams when deciding to change tech stacks. I’ve seen too many teams do tech switches by starting with the “how”, rather than the “why”. Rather than stating that your team wants to switch over to another language (or stack), start by understanding your problems.
- What are our current difficulties?
- How do we prioritise them?
- Are they solvable within the current stack?
You should always go through these steps so that you may reach the best possible solution.
Continue with “how?”
At this point we had reached a consensus that we wanted to begin rewriting our code base in TypeScript. The first step we took was to ensure that we could actually write TypeScript code.
We built Hootsuite Analytics on top of modern software practices. We have a continuous integration pipeline that helps us deliver code to production only after changesets pass several checks:
- Formatting (using Prettier)
- Code coverage
- Acceptance tests
We have three different environments: development, staging, and, of course, production. When thinking about adding TypeScript, we had to make sure it didn’t interfere with the pipeline. And we also had to make it seamless for developers.
I won’t go too much into the specific technical details of how we integrated TypeScript into our code base. There are far too many excellent articles and tutorials out there that describe the process better than I can, and I suggest you check them out:
I will, however, describe our philosophy, our pain points, and what we’ve learned along the way.
One of the most important decisions we made at the beginning was to enforce strict mode. Strict mode is a combination of several compiler flags that, when turned on, perform extra type safety checks on your code. I would recommend any team that starts a TypeScript project to use strict mode from the get-go. Moving code written in non-strict mode over to strict mode is actually really difficult, and you’ll save yourself time in the long term by enabling this mode early.
To solve this issue the community has done an amazing job with the DefinitelyTyped repository, filling in for authors who have neither the time, interest, or expertise to write and maintain their own declaration files. We didn’t have any problems finding declarations for all of our dependencies after consulting it. Unfortunately we did run into a few problems upgrading them down the line. For reasons better explained in this GitHub issue, it can be challenging to find the right combination of versions. With projects such as Redux (which we use and love), there’s an entire ecosystem of dependent projects, each with their own community driven declaration files. You usually need to upgrade all of them at once so as you can imagine, it can get frustrating.
To not get stuck on this particular problem we ignored the import errors. We made a commitment to go back and write the declaration files (which we did, in time). I would recommend this approach to other teams as well, since if you’re just getting started with TypeScript, it’s a strain to write declaration files. Depending on how dynamic your APIs are, it could require intricate knowledge of the type system to finish. In the early adoption days when you’re getting the hang of things, you don’t want to be spread too thin.
With tooling solved, we had to decide how to approach the code migration. Like I mentioned earlier, there would be no huge one time rewrite, we had to continue supporting our product roadmap, which meant delivering features and maintaining the application. We ended up with two rules that would guide our work.
- New code must always be written in TypeScript. No exceptions.
We still wanted to uphold the vision, so we decided to take a look at the major offenders that were causing the large changesets. These were mostly older core React components or utility functions. These modules had been reused across the entire project, so changes made to them cascaded across the entire codebase. Rather than spend time upfront in rewriting them, which would have potentially caused delays in our product roadmap, we wrote declaration files for them. This helped a lot in reducing the size of subsequent pull requests. We delayed rewriting these modules until we were further along with the migration.
Of course we also shipped some bugs in production during rewrites. The amazing value in type systems is that you can catch so many bugs at compile time instead of runtime. That was never the case for us before the migration, though, so we had a lot of lingering bugs waiting for the right conditions to manifest themselves.
Was it worth it?
The best measure of success that we have so far is the number of production deployments. We built our pipeline in such a way that any merged pull requests trigger a production release. This means we push new releases out to customers on a daily basis. Our hunch was that, due to the uncertainty that governed our code base before the migration, we never deployed as often as we could have. And we were right.
The number of times we release to production has doubled compared to 2017.
It’s never easy to attribute statistical success, so I am going to eliminate the usual suspects that may have influenced this chart.
The number of front-end developers on our team has largely remained constant. So while there is some correlation between headcount and production pushes, it is not enough to explain the trend.
We did not put Analytics as a product on pause, our roadmap has been a constant flux of features, so that’s also out of the question. There are some seasonality influences. We enjoy a peaceful code freeze during Christmas holidays each year. During that time we cannot push any code to production. This explains the low activity at the end of each year.
We are still early in analysing the impact this migration has had on our code base. Quantitative data suggests we are shipping more often than ever before. Qualitative information is also very valuable. Our developers love the contextual information that the types offer (especially now that everyone in the team uses VSCode or WebStorm). They are writing less tests because bugs are now uncovered during compilation. They are generally happier about the quality of their code. On-boarding new developers into the project, which historically has been very difficult, is now faster. We plan to do another check-up in six months and see if these initial results still hold up.
This migration would not have been possible without the passion, grit, attention to detail, and hard work by our developers in Hootsuite’s Bucharest office. I’d like to thank Bogdan Zaharia, Flavius Tîrnăcop, Gabriel Ilie, Sergiu Buciuc, and Sînziana Nicolae for keeping the TypeScript dream alive, and in general for an amazing past year. I’m happy and proud to be working on a team that can get motivated when faced with a project like this. Looking forward to our next achievements.
Have you ever migrated a code base to another language? Share your experience in the comments, and let us know if you found this article useful.