Ensuring Stable Schema Migrations
At One Medical our primary application is written in Rails. There are a bunch of advantages of Rails, like how quick it is to develop in or how strong the open source community is, as well as its share of disadvantages, like how the abstractions it introduces can make it easier to write bad code or the rigid structure it provides can cause friction. Rails migrations are a great example of both sides of this: it’s really easy to write migrations and to propagate those modifications to other engineers, but it’s also really easy to write dangerous migrations, and for changes specific to your machine to bleed out and affect every other engineer in the company.
There are some great libraries out there to help you write better migrations (I’m a big fan of strong_migrations), but there isn’t a clear solution for making sure your migrations work smoothly on every other engineer’s computer. For most migrations, this isn’t a concern: your average change action will usually result in a cleanly reversible migration (when you run it up and then down, your schema.rb is unchanged relative to your main branch). This is what I call a “stable migration.”
The problem is, it’s relatively easy to make mistakes that lead to an unstable migration. For example, if you remove a column in your change, when the migration is run in reverse the column could be added in the wrong order. Or if you manually modify your local database, create a new migration, and then run it, your schema.rb will reflect your manual modifications as well as your migration. And because no one else will have those changes, if anyone runs your migration, they’ll generate a different schema.rb.
And in a company with over a hundred engineers, many of whom work on One Medical’s Rails monolith, even small pains like a constantly-changing schema file quickly get magnified. So given that it’s recommended by Rails to check in your schema.rb (it does have a functional purpose, plus it’s literally written into the file itself), how else can we prevent thrashing in this file?
Most Rails projects use CI already, and it turns out CI is great for exactly this. We just need to run a check against every PR that requires migrations to generate stable schemas.
For reference, let’s walk through the check I’ve written up (complete code is at the bottom of this post):
Step 1: Find out if you even need to check migrations
To figure out whether you need to check migrations, you’ll need to find the common ancestor between your current branch and the main branch. If you just check against origin/main without finding the common ancestor, your check will return false positives if the test branch is behind the main branch. There’s also an extra paragraph in here that’s about letting developers manually skip this check; if a committer adds [skip schema checks] anywhere in any of their commit messages, then these checks are skipped. Every now and then someone makes a legitimate change to a migration file that doesn’t really make sense to verify in this manner, so a hook to skip this step comes in handy.
Step 2: Ensure rolling migrations back results in no schema changes
Again, you’re checking that there are no schema changes relative to the common ancestor, not origin/main. This step assumes that an earlier step in your CI pipeline has already migrated you forward.
First, find all of the version numbers for migration files. Then migrate them backwards and make sure they execute successfully. Finally, diff schema.rb against the version in the common ancestor with main and make sure there are no changes:
Step 3: Make sure running migrations forwards is also clean
If we don’t test running the migration forwards and we trust the initial state, then we won’t catch issues where a developer manually modified their local DB and then unwittingly committed that change into their schema.rb.
While Rails’ migrations may seem straightforward, it’s important to take all the necessary steps to ensure they are clean and stable. Following the steps here will save other engineers the trouble of wondering why their schema.rb file is changing and whether or not the primary branch needs to be fixed.
