Sometimes a Big Bang Release is Necessary (and how we got faster because of it)
Axial is an online marketplace that connects private companies to capital. Our goal, like many startup teams, is to rapidly evolve our features to provide more value. But rapid development does not come easy. Our code-base has been through years of iterations with a few incomplete attempts at rewrites. We’ve introduced framework on top of framework, yet leaving legacy code behind. Business rules have been littered in several pockets of code. At times, our best solution was to bandage certain areas creating more convoluted code.
Going through years of designs on top of designs with the notion of cleaning up later on, we’ve accrued a substantial amount of technical debt. Our code complexities and dependencies have caused a severe slowness in our development efforts. Enhancements to a feature would take weeks instead of days.
Decision for a big bang release
Our business drove changes to underlying rules of how we connected the two sides of our market. The modifications were drastic enough to warrant a large-scale data migration. As for modifying the code, it would have been difficult to ship pieces to production without breaking other pieces due to dependencies. Careful transitions would have been brittle and would have taken too long to plan and execute. So we decided on a clean solution. We’ve rebuilt parts of the application (related to the business changes) from scratch, and redesigned the data schema (serving the new business needs). All this, while laying out a new foundation for quicker feature updates and quicker building of new features. This was our big bang release.
Planning, management and testing
The release, through months in the making, presented a huge risk on application integrity. From day one of this project, we placed a major effort on project planning and management. Our teams were kept informed of the project status and unforeseen issues. Integration and testing happened often and early on. Devops were heavily involved in resolving infrastructure issues that developers were facing. We also had set up a recovery plan for this release, which we made use of (it took us two attempts releasing to production).
The new road ahead
A lot of coordination was done to make this release successful. It mostly went according to plan, and was quite smooth uncorrelated to the magnitude of this release. Although we expected bugs being reported, the total amount was much less than expected. High priority bugs were fixed and patched the same day as they were found. We are now shipping multiple times per week on feature updates. Although we still have a sizable tech debt, we are armed with a new foundation to efficiently tackle those challenges. The big bang release was a huge decision, and we are now seeing the benefits in rapid development.
