Agile was the best thing to ever happen to Ops

Who here remembers this method of application deployment?

1. “Code freeze” — everybody checks in what they’re finished with and doesn’t check in anything until deployment is complete.
2. Developers build a deployment package, along with any special instructions for that build.
3. Deploy the package to a testing environment, have developers test their contributions. This step is often done by the developers themselves.
4. Deploy the package to QA and have the QA department run a smoke test (a short series of tests that’s run on every build, such as logging into the site, searching for a specific term, etc). In theory, this environment should match production in terms of OS, patches, builds, etc.
5. QA tests bug fixes and functionality for that release and confirms that the results match the testing environment results.
6. If you can afford it, deploy the application to staging and/or performance environments, which are architecturally identical environments to production (albeit often scaled down). In theory, this environment should match production just like QA.
7. Get approval from everyone impacted. This can take a long time, and it’s often easier to just reserve weekly or monthly windows for deployments.
8. Perform good luck rituals and deploy to production. Wait for QA to test.
9. (Optional): have users find small bugs, fix those directly in production and then tell yourself you’ll fix them in the other environments later.

Us unfortunate Ops engineers were at the mercy of all the above entities (development, QA, and stakeholders) as well as the executive “blackout” periods in which we could only deploy emergency fixes (which almost always got fixed directly in production anyway).

So why is Agile the best thing to ever happen to Ops? Simply put, it made the old deployment method unsustainable. For teams on a weekly release cycle, it wasn’t realistic for us to have so many manual steps. As Ops engineers, we were forced (and, more importantly, empowered) to make the following changes:

1. Builds are packaged, deployed, and rolled back the same way, every time. This doesn’t have to be across the board for the organization, but it does have to be consistent for an application.
2. Many (ideally all) build-specific tests are added to a test harness that’s automatically run post-deployment on all environments.

Those of us with passion for automation (a.k.a. “lazy”) immediately recognized the opportunity to automate the above processes. Those of us who wanted to sleep instead of perform deployments (a.k.a. “really lazy”) took it a step further and created the ability for developers to perform these actions themselves (my personal preference was a CGI web page, but there were a lot of ways to do it).

This was, of course, just the beginning. Afterwards came the official term “DevOps” as an extension of this mentality, and then the addition of AWS took it to a whole new level. More on that in subsequent articles.

So, I’m curious — how does this resonate with you? Is your reaction “wow, we’re going through something similar right now!”, or is it more like “um, yeah, we did that 5 years ago, tell me something new!” (or, perhaps “wow, was this before or after you stopped using punch cards?”) I’d love to get a discussion rolling on this. Comment below or contact me at info@runascloud.com.