Breaking changes: creating big changes.

To guarantee the success of the new product version you have to plan ahead and think about how are you delivering it to your users, here are a few tips that I’ve learned.

7 min readMay 10, 2022

Acknowledgments

This article was only possible thanks to the support and reviews of my wonderful friends André Cavalcanti and Lucas Schlee. Thanks❤️

I’ve worked as a full-stack developer in an enterprise e-commerce platform for almost four years and as a platform developer, you have to take extra care when you deploy because others are relying on your solution to build their products, meaning that you need to keep a solid API, that they can integrate while evolving it to accommodate the ever-evolving business needs. That’s the context that makes you very mindful regarding changes and I want to share some learnings that I had along the way.

Changes are the heart of software development, we either are: coding new ones, reviewing someone else’s, or having a meeting to discuss what/how and when. Even if you think in terms of tools and processes, for example, version control systems, pull requests, SEMVER, CI/CD pipelines, etc. They are devoted to making it feasible to deliver these changes to our end users and for us to have some quality sleep.

All of that to avoid breaking expectations, but unfortunately, sometimes we have to, either because the requirements changed, a better solution was found or we screwed up during the initial implementation, that’s what we call breaking change and they can make your life as maintainer of two solutions miserable. Imagine you having to fix the same bug twice, being there and I can tell you it ain’t fun. 😕

And by API I mean not only REST endpoints and function signatures, but also the entire user experience of your page. If you move a button/form from one page to the other you are creating a breaking change, because your user doesn't know how to locate this feature anymore, like if it was a Cypress test case and now it is failing, that's why you have to treat this change like such.

I've decided to write my experience dealing with breaking changes into two articles, the first, will be focused on the development part and the other on the migration of the users.

The idea is not to give you a recipe on how-to, but rather to show some tools that might be useful.

So, my approach to implementing and deploying big changes is really connected to the product lifecycle itself, like if you’re building a whole new product.

Intro

1. Why / Raison d’être

Like a new product, you need a good reason to justify the time invested by your users to adequate to this new version. Imagine that you are a salesman and your job is to sell a new version of an existing product to a customer that already has it, what would be your arguments?

Humans are hard-wired to resist change

so if you want to get some adoption on this new version you need to give your users a good reason to, this is critical for the next stages, otherwise, you will be building a bicycle for a fish, meaning a totally functioning product that no one will want to use. It can be a new feature, or improvements in speed, either way, fixing typos, or updating frameworks aren't good enough for a reason. If you are building a platform, think that your user is another developer and they will need to sell this to their PM and EM as well. Bonus point, if you could get some metric related to money to back up your proposition, for example reducing hosting costs or improving conversion rates, this will make your argument sell like fresh bread.

2. Rollout planning

Before starting to develop think about how you are planning to put this into production. For me, the ideal way is that you can do a progressive rollout using some strategy like feature toggling and with an easy rollback. Perhaps, if your change is too big, you can break it into different phases and roll out these phases progressively. Once I was working on an app that had basically four different pages and my strategy was deploying a page per time rather than the full migrated app at once, if your page is too complex you can even break it into sections. The important here is to do small iterations where you feel comfortable deploying each phase and still evolve towards your goal.

3. Observability

Sharpen your observability tools, make sure you have that set up on the previous version and on the new one as well. By observability, I mean error logs and metrics that you can use to assess your environment behavior in runtime. If you don’t have it yet, it is a good thing to invest some time adding it to the old version, even if you intend to kill it right after, because otherwise, you will end up with no baseline to compare what is expected and what not. Examples could be perhaps some timeouts that only happen in production or errors in specific devices, they might be already happening but if you are not aware, will make you roll back even if it has nothing to do with your change. Also, it is important to have a dashboard showing the adoption rate of this new solution and if you promised some metric, it's better to keep track of that as well. The Net Promoter Score (NPS) is very common when thinking about evaluating products and might be worth keeping track to compare a before and after, although I personally prefer the Likert scale, they are both very good to evaluate changes. Showing your progress is important for your stakeholders as it is for your morale, this will cheer you up in the future 😊

Some tools that I’ve used before for this matter:

Logrocket: error logs
Splunk: error logs, alerts, and dashboards
Grafana: Metrics, alerts, and dashboards
Honeycomb: Metrics, alerts, and dashboards
Google Analytics: Metrics, dashboards
Amplitude: Metrics and dashboards

4. Development

Some industry standards/concepts that are worth knowing about in this context are:

Changelog → A markdown file that contains a curated, chronologically ordered list of notable changes for each version of a project. Vide
Semantic Versioning (SEMVER) → Strategy of versioning your product where each increment in version will have a specific meaning. Vide
Deprecation → Discouragement of use of some feature. When you mark something as deprecated you're telling your users that this feature shouldn't be used and it might get removed on the next versions.

While developing is really personal there are a few tips that I’ve used before and worked like a charm for me:

Don’t open too many branches → Try to keep the new code as close as possible to the code in the master branch, try to separate it into different folders instead, perhaps even a monorepo approach. If you open a different branch and still keep working on the previous version, with time the branches will diverge to a point that it will be very painful to merge them together.
Make small Pull Requests → No one likes to review big pull requests (+1000 lines), they take too much time and make bugs easier to slip by. Merging changes directly into the master branch will also help with this.
Black-box testing → To validate the accuracy of the new version you can capture some inputs/outcomes from the production environment and run it against the new version to evaluate your changes, making the necessary adaptations accordingly.
Deprecate → Marking features as deprecated is a good practice to prepare your users for a future version and also will serve as a TODO list for you while developing the new major.

Conclusion

In this article, I've tried to bring consciousness to the challenge of implementing a new version of existing software, and some strategies that you can use to reduce this burden. Make sure you give your users good reasons to migrate and also a way for this migration to occur as smoothly as possible. In the next one, I will articulate more about how to get some traction and migrate your user base to this new piece of software, so you can move on with no strings attached to the past.