Mind over (techie) matter: the importance of mindset shifts in any technical change process
In Koble, we recently improved our CI/CD pipeline to reach a nearly Continuous Deployment (CD) approach. We followed a lean approach to implement a lot of interesting technical improvements that ranged from increasing test speed and reliability to migrating the pipeline to GitHub actions but when we look back and reflect, everything really started with a shift in mindset that enabled all of the change to happen.
No matter how simple or complicated the actions needed to implement a change are, if the people involved are not aligned or engaged, chances are you will most likely fail.
This short article outlines how the Koble team made the shift from a painful to a delightful release process.
Identify the Big Opportunity
Let’s start from the beginning. Earlier this year we had reached a point where there was a long time between releases, which impacted us negatively in several aspects. Mainly, the releases were too big, with a lot of tickets, some of them with dependencies and modifications required for the same parts of the application. This situation ended with buggy releases that created a lack of trust from our stakeholders and customers. Everyone became nervous and anxious on release day, which often turned into long days and nights for all involved parties.
So then the question becomes: how did we end up there? I believe there were actually several reasons, for example, we always delivered the different modules at the same time, causing even more issues that would be difficult to untangle once things went sideways. But even with fewer modules, our development lifecycle itself was not prepared to handle more frequent releases.
Form a joint vision
Once we had a clear understanding that we needed to improve, we took advantage of some of our existing agile practices (book club, BE, and FE chapter meetings) to brainstorm and discuss possible approaches as a team. At that moment we were reading the book Accelerate. The study done as part of that book showed that continuous delivery impacts directly on software delivery performance, organizational culture, and other outcome measures, such as team burnout and deployment pain.
Then, again as a team, we agreed that we wanted to fix that issue by having a CD pipeline, and only then we started to apply changes towards our goal.
Generate short-term wins
Humans are chronically impatient so trying to keep a team aligned and focused on a distant goal is beyond hard (Discounted Utility, Kahnemann, etc.). Therefore we sliced the distant goal, CD, into smaller attainable chunks. First, we put a maximum time of two weeks between releases and established a fixed calendar. With this ‘less distant’ goal (nothing technical to see here!) we started to improve, and we also realized that we also needed to adjust a number of things across the entire process to become more efficient.
Subsequently, we also revamped our ways of working, to reflect in a better way the current development lifecycle and start moving it to CD. We expanded our workflow to reflect better the current development lifecycle and we made explicit the definitions of ready and done that needed to be matched between transitions. With this exercise, all the stakeholders (not only development) had a common understanding of what every status means and what needs to be done to reach the next one.
Sustained acceleration
Week by week we continued to implement tactics and technical changes and in only 6 weeks we transformed a painful ordeal into a 20 minute transparent and (mostly!) painless process. For example, we migrated our pipeline to Github Actions, and we improved our testing suite to start using stubs instead of real third-party integrations.
The important thing is that once your target is clear and shared with everyone, that’s the moment when the magic happens. We continued to implement ideas and changes that got us closer to the CD target in every sprint, which also allows you to adjust and adapt to your specific needs as a team and company. A clear example of this last point was when we added real time alerts to detect possible performance or errors in production. That sounds like a great idea, right? But at that point, there was no clear ownership of who needs to react to the alerts, sometimes several members of the team were looking at the same time, sometimes no one. Then we implemented “Foxy of the Day,” a rotating role that is in charge of all the interruptions that day. This ensures the issues are being looked at but it also minimizes the impact on the team at large.
Communicate effectively and with transparency
During all of this we continued to make our case with key stakeholders: we spoke with passion, we explained the benefits, were patient, and at some point, they started to see the benefits of the new approach (fewer bugs, super fast reaction to issues, and releases that are 100% transparent).
Again, communicating a consistent message from all team members will not happen if you are not on the same page. Understanding the opportunity and standing united behind a joint goal can work wonders when you are aiming to drive lasting organizational and behavioral change.
Institute change
Lastly, investing and driving change towards a better way of doing things if the new processes and behaviors ‘stick’. At Koble, we were able to see how the mindset change had really embedded itself into the DNA of the team; driving all people’s actions, not only some specific tickets or features.
Indeed, CD has really had an impact on everything that happens in Koble and enforced our culture as a product and customer obsessed team: releases are now anticipated and celebrated, the way they should be!
We don’t pretend to have all the answers, but we are happy to be part of the wider conversation about how technology and engineering can contribute to innovation in the insurance sector.
Learn more about us on koble.io.
