Continuous Delivery Is About Failing Faster!
If releasing to customers once a month, once a week, and even several times a day were easy — everyone would be doing it. Getting IT assets that generate revenue released faster is clearly an advantage. However, the biggest return on investment of Continuous Delivery is a little more subtle.
If you build it, they don’t always come
When a business has an idea for an investment in IT, they do market research, calculate costs and return, forecast customer adoption and get stakeholder buy-in — but often wear rose colored glasses without even knowing it. This isn’t any different from a start-up that is pitching its first business plan. Even with conservative estimates, many assumptions are made and there’s really no way to know how the investment is going to pay off until it gets delivered. There are no sure bets in product management.
As a product manager or business owner several of the ideas you have for your customer will turn out to be useless to them.
Over the 19 years that I’ve been working with companies to deliver IT products and services, I’ve never been satisfied with the waste I see in the processes that many organizations use in bringing ideas to their customer. So much time is spent prioritizing, evaluating costs, scheduling, and tweaking the efficiency of resources but a healthy dose of humbleness is often the key ingredient missing from the equation — and the most costly to ignore. When sure bets turn into failures, the cost of the investment and decisions of engineering are often where the finger is initially pointed. The reality is that as a product manager or business owner, several of the best ideas you have worked out with your customer will turn out to be useless to your larger audience. Customers may even tell you it’s the best thing they have ever seen, sign up to be early adopters, and give you positive feedback on what you’re thinking of charging. But by the time you release it to them, you’ve spent heaps of capital and it can cost you your job (or your equity) to find out you were wrong.
Taking risks in the market is necessary for a business to thrive. Innovation in small businesses drive the economy of the world, despite what larger corporations would have you believe. When a tiny idea explodes to have a big impact, we all benefit as new opportunities are created that result in exceptional economic growth. But even large organizations can have innovative, disruptive ideas flourish as pockets of entrepreneurial spirit win internal support to try something new. The barrier that must be overcome is risk aversion. With the biggest opportunities for growth requiring us to take some chances, why would anyone want to go about delivering their IT offerings using a process that penalizes change and the risk of unexpected outcomes?
Stop the bleeding
“Mr Corleone is a man who insists on hearing bad news at once” — The Godfather
The good news is that it doesn’t have to be this way. When a product that requires physical manufacturing is getting ready for market, everything has to be solid in the design and any changes along the way can bankrupt the idea. If a design change is needed in a new mold for carbon fiber parts that costs millions and the first one has been manufactured, the entire cost of the mold is lost. IT assets aren’t at all like this, though companies continue to think in this mindset. The inventory in an IT asset is information, and information is cheap and easy to change.
Despite the low cost of information as inventory, the time invested to create and change that information is still a concern. If an IT asset is delivered to the customer and found to not have market fit, the more that was invested in that idea, the more that is lost. Knowing this, we should seek delivery processes that let us know as soon as possible that what we’re delivering is not on target and enable us to make changes with the lowest possible loss. Many organizations employ usability studies and surveys to give them validation that an idea is viable before delivering it, and this is certainly a great practice. However I’ve seen customers rave about a design on paper, or a prototype — and then hate the product once it was delivered to them. You really need to deliver an actual, working idea to measure adoption in the market and get feedback.
To deliver IT assets more frequently requires organizational change, relentless communication, investing in some technology, and the ability to handle the increased volume of customer feedback that comes with doing so. This isn’t something that can be done all at once, and a challenge I deal with when helping clients is to balance keeping momentum going with the changes needed and not being too disruptive to existing business. When I’ve seen teams attempt to do this without outside help, the politics and lack of experience with cross-functional collaboration can doom it to being looked at as just another pet project of an ambitious employee who didn’t understand “our culture”.
So many ways to fail (and be profitable)
When a team uses Continuous Delivery to move towards frequent releases for their customers, they invest in the creation of a deployment pipeline — which is essentially an automated release process. This is more than an automated build, which may compile some code and deploy it somewhere. Does someone ping the web servers every time you do a release, to make sure things are up? Automate it. Does someone change a setting in a configuration file to publish a mobile app to the right store? Automate it. Do a group of three people have to approve a build before it goes into production? Automate moving the build further downstream with their electronic approval.
With the pipeline in place, there are dedicated stages that each build of your IT assets go through prior to being released to customers. Along the way, an increased level of automated scrutiny is placed on it to fail as soon as a problem indicating that this version is not ready for prime time is found. Here are just some of the ways a delivery pipeline can help you be more profitable by causing failures to occur earlier in the release process.
Failure: Defect found in production.
Cost: Must rollback system to good state, interrupt new work, and repair customer confidence.
Remedy: Automate acceptance tests in deployment pipeline that run in environments prior to production
Failure: Flawed design pattern identified.
Cost: All assets that used that pattern must be changed and re-tested.
Remedy: Initially release a small number of assets that use the pattern to customers.
Failure: Reduced desire for feature.
Cost: Lost investment in what was built.
Remedy: Invest only in a minimum viable feature before releasing to customers.
Failure: Slower output from team than expected.
Cost: Must adjust forecast delivery dates, must improve capability of team.
Remedy: Don’t commit to dates for sprints far in advance, measure team output more frequently, get training or make hires.
Failure: Insufficient performance at capacity.
Cost: Must purchase new hardware, optimize IT code or assets.
Remedy: Automate performance acceptance tests in a capacity testing environment.
These are just a few examples of how a delivery pipeline, and a team that has been organized to continuously deliver IT assets while aligned with their customer, can save money by failing faster. You can see here that the cost of finding a defect prior to production is less than if it were. The cost to change a design pattern that has low penetration in the overall architecture is lower than if it were identified after applied broadly. The cost to find or train resources to better support the needs of a team is less than not knowing they need this help and utilizing their low throughput over the lifetime of a product’s development. All of these early identifications of failure lead up to dramatic savings in the overall cost of finding that critical market fit, and working in a sustainable fashion to support the needs of the business.
By reducing the cost of failure, the ability to handle risk goes up and the opportunities for efficiency are numerous. If your organization is not taking steps towards a more fluid release process, it is only a matter of time until your competition’s agility will enable them to blow past any market advantage you currently have.
Originally published at Jayme Edwards.