Reimagining the Rails Upgrades: A Scalable Path Forward

Woop! Rails 6.1 is finally on the Hnry platform!! 🎊🥳. This work is the result of months of collaboration between our engineering teams! — championed by the Platform Squad (the team I am a part of). However, given that the Rails 6 to 6.1 upgrade impacted nearly every aspect of the codebase, we encountered significant challenges.

As we set our sights on Rails 7, a project even more technically ambitious for our monolith, this article is a reflection of our previous work and an exploration of a scalable path forward!

Upgrading to 6.1, a quick recap

At a high level, our Rails monolith consists of internal gems and a main application codebase. To make the upgrades possible, we maintained parallel builds of the main app & internal gems housed in separate branches — where one is compatible with Rails 6 and 6.1. This way, the “upgrade build” is isolated from the build our developers are iterating on.

Clear isolation between the 6.0 and 6.1 builds during our upgrades

In the upgrade build, we first sorted out the incompatible dependencies, and then we let it hit the CI pipeline to surface and resolve the breakages. The build was frequently updated by merging the production branch with the upgrade branch to capture any new breakages that entered the codebase.

Once we resolved all of the issues, the Rails 6.1 version of our application was released!

Wow, that was easy to write! 😅 In reality, we encountered significant issues with this approach. Let’s take a look at them…

Challenges 🌀

1. Stale Branches & Merge Conflicts 🌳⚔️

As we worked to solve the breakages associated with the upgrades. The long-lived branch maintained by the Platform Team was often left stale. When we merged the ever-evolving production branch into ours, we had to deal with both merge conflicts and newly introduced issues.

The conflicts were especially hard to deal with as these can arise in unfamiliar places within the codebase.

2. Responsibility Grey Area 🗺️

With any issues we were tackling, we couldn’t tell who was best equipped to solve a specific problem without polluting comms. This often left us wondering if the issue was best addressed internally (within the Platform Squad 👷) — thus taking more time.

3. Feedback Lag 🐎🏃‍♂️

As we maintained a long-lived branch, there was a “lag” in getting feedback on any introduced breakages. For instance, an engineer might introduce a change in Rails 6.0 without knowledge that it doesn't work in Rails 6.1.

The need for a tight feedback loop🪞

Most of the challenges encountered stem from the extreme lag in feedback the isolated build approach necessitated. If I were to draw it, it would resemble this:

The rough structure of the feedback loop necessitated by our isolated build approach

💡 Can we somehow tighten this feedback loop? There MUST be a way can we bring this closer to the engineer.

An alternative approach 💡

Let us imagine that when an engineer adds a feature. What if they are immediately given feedback on its incompatibility with the next version of Rails?

This rears its head against all of the major challenges. If a deprecation was introduced, it would ideally be solvable by the engineer who has the most context given the problem and it could live in the same branch!

⚡️ Clearly, this is a state that we want to get to. But how do we do this?

CI Matrix Builds ♻️

We probably need to lean into the strengths of our CI/CD pipelines. During our Ruby 3.1 upgrades, we leveraged matrix builds to test multiple versions of Ruby in an automated manner. As an engineer, it was to see if my changes worked for multiple versions of Ruby!

What’s stopping us from doing this for Rails? Several things:

1. Framework v Programming language: Unlike Ruby, Rails is a framework — a collection of libraries and tools.
2. Gem Compatibility: We need to think of a way to allow different versions of the Rails gem to be tested against the same codebase. This is particularly challenging because how can we house incompatible code in the same place between different Rails versions?

Fortunately for us, this problem is not uncommon and one of the more interesting approaches comes with the idea of Dual Booting.

Dual Booting 🥾

One of the gems enabling dual booting is the next_rails gem (see GitHub). Their approach allows us to place conditional logic within the Gemfile allowing the same app to run under different versions of the same gems.

Let’s have a look at some code:

# in Gemfile
if next?
  gem "rails", "~> 7.0.0"
else
  gem "rails", "~> 6.1.0"
end

This code injects logic into the generation of the .lock file. The predicate next? is controlled by whether we choose to run bundle <cmd>or next bundle <cmd>. This gives us the power to be selective with gems and/or gem versions!

As for incompatible changes, they can live within the same code via a similar conditional!

if NextRails.next?
  # breaking change
else
  # current code
end

Thankfully, this NextRails.next? snippet can both be run in the app and any pulled gems. For example, if the main application is run on next bundle <cmd>, the runtime behaviour of the pulled gem is also modified. So, while being on the same code, next acts like a switch. 🤯

Prepending `bundle` commands with `next` allows for modified behaviours!

Scaling the Rails 7 Upgrade 🌊

The technical feasibility of matrix builds with different versions of rails is made possible via dual booting. So what does this mean? — Scalable Problem Solving!

Let us imagine again: You are an engineer who adds a feature, and it passes the current CI test suite, BUT this fails the Rails 7 CI.

Dual booting and CI Matrix builds enable tighter feedback loops!

Two things are enabled here: The engineer with the most context to the problem got immediate feedback. Another is that the functional area to which the failure arose was made very clear by the CI suite. The engineer can immediately apply (or prioritise) the fix and an additional layer of communication is not required! 🤩

Conclusion ✨

After reflecting on the Rails 6.1 upgrade and exploring a new approach for Rails 7, I hope I’ve instilled the seeds of hope and optimism in your thoughts, dear reader!

We outlined how we applied the parallel build strategy and the challenges this brought us. Thankfully, we can look forward to a scalable way of working made possible by matrix builds & dual-booting.

Undoubtedly, this new approach will have its own set of challenges. But the promise of something better is what compelled me to pen this article! And while there will be tradeoffs, I believe the rewards far outweigh them.