Driving adoption of Sorbet static type checking in Rails

At CZI, we’ve adopted Sorbet, a fast and powerful static type-checking tool for Ruby. In our previous post, we talked about the technical challenges of adopting Sorbet in a Rails codebase, which led us to develop the sorbet-rails gem. After a year of using Sorbet, we have made significant progress covering our system with types. Currently, we have reached 92% file-level typed coverage and 75% callsite-level typed coverage. Most importantly, Sorbet is now integrated into every engineer’s development flow. In this post, we’ll go through our phases of adopting Sorbet and the challenges and the lessons we learned along the way. We have been able to adopt Sorbet gradually while still benefiting from it on day one.

Setting up

Setting up Sorbet was fairly easy: the adoption guides of Sorbet and sorbet-rails left the codebase with 0 type errors. The type coverage metrics were also promising: we got our codebase to about 80% files type-checked and 40% callsite-level typed coverage. It was a good start! However, it still left out important Rails models and libraries. Many files might not be type-checked (typed: false), or worse, ignored.

We also disabled Sorbet’s runtime checks initially. They are a key component of Sorbet to make sure method signatures are correct and match runtime behavior. However, they can cause errors in the production app if someone were to write an incorrect signature. At this stage, we only enabled them in RSpec tests to minimize its effects on the production system.

Building a foundation

Our next step was to develop an understanding of Sorbet and how it interacted with our system so we could develop a strategy to type-check our code. This also included figuring out what code is worth type-checking, and what could be left un-typed.

This was where we identified common code patterns and figured out how to make them type-checkable. This is important because when you figure out how to write these patterns with types, you enable people to write new code with types. This included core Rails abstractions like Model and Controller as well as abstractions and patterns that are created. Here’s a simplified example of a code pattern in our codebase:

We made this pattern more friendly to type checking by changing the structure to let Sorbet validate the argument type automatically:

We also realized it would be important to have guardrails to ease people into adopting Sorbet. Here are a few integrations that we set up. They were invaluable in driving the adoption later:

• Git integration: run type checks locally before creating or updating remote branches
• CI integration: run type checks automatically for each build
• CI integration: track metric changes after every commit merged to the master branch

We found the adoption metrics important to understand how our adoption was going. On top of the metric Sorbet provided, We also found it useful to track individual contributions to type coverage and method signatures, because it tells you who is engaging with the tools and who may need more support. We recommend setting them up as early as possible!

Ramping up type coverage

Once the foundation was set, it was time to dive in and type our code! We started looking at our models and controllers and fixing type errors in them. It was exciting when we found a lot of dead-code or small bugs that wouldn’t be found without a tool like Sorbet!

We found many bugs and dead-code using Sorbet

We also faced many technical challenges in integrating Sorbet. We built sorbet-rails and documented a lot of tips & tricks about handling those challenges. Sometimes, we used T.unsafe to bypass type-checking on one or two lines of code so we could type-check the rest of a file. These were healthy compromises because our goal was not to blindly type-check everything, but instead to take advantage of type-checking when it is most useful.

We deemed these files important and set typed: true on all of them:

• All models
• All controllers
• All data mutation code
• And all files enforcing our privacy controls!

It doesn’t require a lot of effort to make files typed: true, because the requirement for typed: true is low — it only checks that the Ruby syntax is correct and methods exist. This drove up the file-level coverage to 90% and increased Sorbet’s surface area significantly. Once we made the files typed: true, it was also easier to ask other engineers to follow suit and use the same sigil in new files.

We also recruited engineers in the team as early adopters, who were eager and patient with initial hiccups. They were invaluable in providing insightful feedback about bugs and flagging issues that made Sorbet difficult to use. The latter was important: fixing code patterns that were hard to type-check enabled people to write type-checked code more easily, which was crucial in the next phase.

Transitioning the team

Our next goal was to make type-checking part of everyone’s development process. We ran a few training sessions to familiarize people with Sorbet and how to write type-checkable code. Following the training, we enabled type-checking in everyone’s development environments. At this phase, it was valuable to add non-blocking lint rules or checks to guide engineers in the team, for example:

Lint that new files must be typed: true (check out rubocop-sorbet)

• Lint that every pull request passes Sorbet type-checking.
• Configure run-time checks to log in production. Errors are logged and reviewed without affecting production runtime.

Our configuration was set to log errors in production, rather than raising them

It was important to make the rules non-blocking because we did not want people to see Sorbet as an obstacle for them to do their work. If they needed to make some changes quickly and didn’t have time to study the new tool yet, they should be able to skip over it. After 4 months, there were only 5 or 6 times when srb tc reported any error on our master branch.

We also celebrated people who contributed. We would give shout-outs to those who adopted type-checking early and contributed significantly to increasing the type-coverage on Slack and in our team meetings. Many adopters become advocates for Sorbet too: they found immediate benefits in using Sorbet and encouraged others to participate.

At this stage, the lift on engineers in the team was low, but the team increased the callsite-level type-coverage up 10% to 50%. We got there simply by writing code that doesn’t violate Sorbet type-checks and leveraging the signatures provided by Sorbet and `sorbet-rails`.

Making it the norm

By this point, we had all of our models, controllers, and new files typed: true. We gave people time to get used to running Sorbet type-checks and fixing type errors. It was time to speed up the adoption.

We upped the enforcement level of our lint checks:

• Required Sorbet type-checking to pass for the build to pass.
• Piped run-time type errors to our error-reporting system, which directed them to responsible teams immediately.
• Required that every new function is accompanied by a signature, again using the rubocop-sorbet gem.

We started recruiting people to write signatures for existing code. One effective strategy was having new engineers type-check an old part of the codebase. It feeds two birds with one scone: they got to learn how the existing code worked, and the code got better type coverage. Sometimes, adding a few method signatures to a commonly used class or method (like ApplicationController) yielded a significant return in type-checking coverage.

These strategies accelerated the callsite-level typed coverage to 60% after about 2.5 months. Since then, callsite-level typed coverage in our codebase has steadily increased. People felt the benefits. At the same time, the amount of untyped code remained the same; there is still a lot of room to add types to existing code.

One thing to note, we are still using non-blocking run-time checks in production. This makes people feel safe and comfortable writing method signatures. We don’t need them worried that they’ll break production if they make a small mistake like using String instead of Symbol in a method signature.

Adoption challenges

While our adoption has gone pretty well and has greatly improved our development process, it came with some challenges:

• Type-checking existing code is hard and error-prone: When you change code, there is always a risk you’ll break it. This is why tools like “soft” runtime errors are crucial.
• Some code feels more verbose or clumsy than a Rubyist would like. We’re not talking about Sorbet sig — they are actually very easy to write and useful as documentation. We’re talking about programming constructs such as shapes (a Hash with named keys) and enums (eg string enum in Typescript). Sorbet offers alternatives using T::Struct and T::Enum; however, they also make the code more verbose and harder to convert.
• Some patterns are very difficult to type, such as the DSL pattern used in RSpec or the method overloading pattern. This is due to various reasons, sometimes due to the dynamic nature of Ruby code makes it hard to capture in static typing, and sometimes due to the limitations of Sorbet as a tool. For the latter, Sorbet is being actively developed and I hope it will become more powerful with every version.

Lesson learned

To adopt Sorbet quickly and smoothly, here’s what we learned:

• Focus on covering many files first, then drive the type-coverage within the file later.
• Make sure common code patterns in the codebase are type-checkable.
• Use non-blocking checks to guide people into adoption gradually.
• It’s easy to write new code that is type-checked. Make it the default.
• Type-checking existing code is an opportunity for new engineers to learn the codebase.
• Celebrate along the way: bugs you find and fix, adoption progress, positive anecdotes. Sorbet is meant to be adopted gradually anyway!

Summary of the checks we have at each adoption stage

Sorbet is a great tool, and we hope that by sharing these lessons and the tools we developed, you will also adopt it successfully. We probably will never reach 100% type coverage, but that’s not the goal either. Some libraries do hacky, dynamic, untyped things to achieve awesomeness. But type-checking makes it easier and safer for our engineers to develop features, and our team is seeing the benefit of using Sorbet every step of the way.

Although there is still a lot to do, I’m optimistic our team will continue to move faster and ship more reliable code with Sorbet in our toolchain. We wouldn’t have been able to do it without incredible support and eagerness from the team. I’d like to express my gratitude to everyone on the CZI Education team!

Additional resources

You may read more about adopting sorbet in following resources:

• Sorbet set-up guide
• Sorbet adoption tracking guide
• Use Sorbet to find most “impactful methods” to type
• Static Type Checking in Rails with Sorbet by Hung Harry Doan
• https://blog.heroku.com/static-typing-ruby-with-sorbet by Heroku team
• Static typing for Ruby by Alexandre Terrasa (Shopify team)