Why continuous delivery is not enough?

Continuous Delivery has been treated as the best practice for almost a decade. I’ve seen a lot of teams who are doing it well. They have different types of tests to cover the business logic from UI to a database. They follow the test pyramid quite well to have minimal effort with maximal coverage. They are able to do a release with only one click if their product owners ask.

However, when a system-wide refactor (such as changing to a better json library or a big version upgrade to a dependency) needs to be done, they show caution instead of confidence. Why is that? Shouldn’t they do the change, push to CI, then deploy to production if it’s green? If your team can have the confidence to do this, congratulations, you can remove the button before the production deployment and go one step ahead: Continuous Deployment.

Continuous Delivery vs Continuous Deployment

More than confidence

• Real automation & 100% coverage — there would be no manual test allowed in any cases after pushing a commit. This force teams to automated really everything including visual testing for long term benefits.
• Better quality assurance — quality doesn’t require any manual tests, there would be no human mistakes
• Easier to return tech debts — Fully test coverage help teams do any change without considering the testing effort. This can reduce the cost of a system-wide refactor to less than half. And more importantly, it changes the way to do a big refactor from a team delivering activity to a purely developing activity.
• Less waste — teams have no chance to implement anything without deploying to production. This would force the team and PO to work more closely to take the responsibility of releases together because the PO and the team need to find out the client impact/value of any change before the team push the commits. However, continuous delivery still opens the chance to the PO to change his/her mind at the last minute.

Feedback before implementation and during implementation

Even with these benefits, some product people might still concern about releasing functions to production directly without a check. They want to make sure that the new feature is what they ask for and it fits into the existing functionalities well. To solve the concern, a common way is putting every feature behind a toggle (called feature toggles), which creates a lot of additional effort for developers. But is it necessary?

What we are actually looking for is confidence in the value of the feature (is it what the users want?) and the quality of the feature (can it fit into the existing functionalities well?). And we can increase the confidence by getting:

• before implementation: we can easily create mock-ups with business flows to get early feedback on a design at a very low cost
• during implementation: the PO and the team worked closely and do tiny showcases frequently during the development phase (can be multiple times even when the feature is partially finished)

In this case, we don’t need most feature toggles (only if there are legal requirements or for the purpose of A/B testing).

How

So, are you ready to remove the one-click button? Do the following practices:

1. 100% business coverage — make sure you follow TDD all the time, even the css changes should also be covered by visual tests. And don’t forget to automate all the non-functional requirements as well.
2. Zero downtime deployment — Most teams who achieved Continuous Delivery have already implemented this. If you haven’t, do it because you will deploy many times each day and your users shouldn’t feel that.
3. Monitoring & Alerting — do live check and health check for not just the service, also the dependencies to make sure you are notified if anything goes wrong

These practices give you the confidence to say a green CI build means that the application deployed to production with high quality and no customer impact, otherwise, you should get an alert from your on-call system immediately.

Summary

In summary, Continuous Delivery is good but not enough to be excellent, you can improve your team’s productivity by moving forward to Continuous Deployment.