The Problem with Change Freezes
I write this on the advent of our IT department’s second change freeze of the summer; coinciding with critical periods for the business of the organisation. On the face of it, it sounds like a reasonable thing to keep IT systems in a stable state during important activity. After all, we’ve often joked (usually before going on leave) that “things only break when we touch them!”
That’s a fallacy though; things break whether we touch them or not. Sometimes it’s because someone used them in a way that they weren’t intended to work, or sometimes there’s a 3rd party involved, like a vendor, or perhaps an environmental issue. Sometimes somebody might be digging a hole near the data centre and trip the power (the estate around the building not being subject to the change freeze — perhaps it should be!).
In IT, change is about the only constant, and there are (at least) three problems with trying to freeze it.
The first one is most obvious; that for the duration of the change freeze you have people being less productive than normal. If you aim for continual improvement of your systems, then that’s a chunk of time when you aren’t continually improving. Yes, you can be planning improvements for after the freeze, but where small incremental improvements result in smaller, less risky changes, bigger, bundled-up improvements can result in bigger, more risky changes when you do implement them.
The other two problems with change freezes are simply that they start, and they end.
When approaching the start of a change freeze, there’s the temptation, and sometimes encouragement from management, to rush changes through before everything is locked down. In recent times, I’ve been aware of an instance when changes being made in response to a major incident were rushed through ahead of a change freeze and actually caused two further major incidents.
If you didn’t have a change freeze, you would most likely approach that work with far less haste. More haste; less speed, as they say.
The problem at the end of the change freeze is covered beautifully in “The Phoenix Project: A Novel About IT, DevOps, and Helping Your Business Win”, which everyone working in IT, or managing IT, should read. With a number of changes built up during the freeze, how do you release them? What takes priority? Are any of the changes going to clash if you just say “the freeze is over, change away!”?
What you need to avoid these problems, and indeed to avoid change freezes altogether, is effective Change Management. Both in terms of systems and processes.
Her Majesty’s Revenue & Customs (HMRC) talked at DevOps Enterprise Summit 2016 about their adoption of DevOps processes and how they were able to go from having a change freeze two months long over the peak period for tax self-assessment in 2015, to business as usual during the same period in 2016, when their systems were far busier, yet experienced less downtime and fewer incidents.
We are nowhere near mature enough in our change processes to be able to do this, in terms of planning, testing, deploying or communicating, but we are moving in the right direction. Hopefully one summer before too long, we’ll be able to do away with freezes and find out what happens to solid water when it gets warm.
