How often should you really deploy your code?

It seems that since continuous deployment became a thing companies have been deploying code to production more and more frequently seemingly without any limits.

Here, a company (one of many) proudly proclaims they deploy code 300 hundred times a day. Have they reached nirvana yet? I’m pretty sure some companies would deploy a build on every key-stroke if only they compiled.

I wonder if people consider what the optimal deployment frequency might be for their project. For many years, more frequently certainly was better. Every month is certainly better than every year, but is every day better than every week and what benefits do we get for even more frequent deployments than that?

Firstly, I should say that being able deploy 300 times day is a good indicator that your build and deployment pipe line is well engineered. In order to do this, all manual steps must have been automated with most code covered comprehensively by tests so it doesn’t particularly surprise me that people share their deployment frequency as a way of communicating that they’ve adhered to good, established processes.

So why might you want to slow the frequency down?

You care about your users’ bandwidth and time

The main motivation for this post was the fact that I’ve seen so many sites that bundle large amounts of javascript together (for example, using webpack) and invalidate their users’ cache of that javascript every time they deploy. Whether you use a build number or a content hash as a cache buster, it’s not fair to your users to force them to download your entire app’s front-end source code each time they refresh the page.

I am aware that there are techniques to mitigate this problem to some extent (for example, splitting code) so maybe you have this solved another way, but if you’re making frequent changes that require your users to re-download static assets too often, maybe slow down the frequency of your deployments

You cache stuff on the server

Most times code is re-deployed caches are invalidated. This is because data-structures may no-longer deserialize cleanly into possibly updated data-structures, or the cache may be part of the web-server’s process memory.

Chances are, if you’re caching things you’re doing it for a reason. Maybe you’re solving a performance issue with it. The more times you deploy, the more times you have to repopulate those caches, and suffer the performance hit of having a cold cache.

Your code’s performance improves after profiling

Many programming languages optimize the performance of code as it runs. Particularly programming languages designed to work well on the server.

Techniques such as JIT compilation or adaptive optimization employed by runtimes such as Java’s hotspot VM allow code to run faster after enough information has been gathered to profile the running code.

It is rare for VMs to be able to use data from a similar but different build (I don’t know of any that do, but I’m not ruling it out)

If you deploy as frequently as possible, you are giving up many of the advantages of this kind of profiling

You don’t want to wait for preliminary metrics

A common way to determine whether code is bug free is to deploy it to a small number of users first (e.g. 1%). As long as the number of users is statistically significant you can check metrics to see if performance has regressed, or any other metrics (such as signups, or payments) have changed for the worse.

For this (pretty common) technique to be of any use you need to wait enough time for the results to come in. That alone limits the number of deployments you can do and wouldn’t this be more useful than focusing on what is probably only a vanity metric anyway?

Conclusion

Being able to deploy at a high frequency demonstrates a great dev-ops achievement, but just because you can do something doesn’t mean you should.

I’d certainly not recommend being in a position where you couldn’t deploy an emergency fix when required. Serious bug fixes should always take precedence over the points I listed above, but hopefully not every commit is a bug-fix, so does it really need to be deployed immediately?