How to get started with development & deployment automation tomorrow
Automating manual work, such as deploying your application or running tests, can be a daunting task. Most teams agree that automation is important. But not knowing where to start with this ‘huge and important thing’, they keep pushing it into the future. In this post I offer a practical approach to get started with automation tomorrow.
Automation is one of the cornerstones of Agile software development. The idea is that manual work is tedious, error-prone and therefore something you’re likely to do as infrequently as possible. This includes tasks such as testing your application and releasing a new version of your platform. But by doing this infrequently you lose a wonderful opportunity to rapidly gather important feedback on questions like “Are my components still integrating?” or “Can our codebase still be deployed to our server-environment, or has configuration changed?”. In short, you want to automate manual work to learn faster about what works and what doesn’t.
Most teams agree with this wholeheartedly. They can’t wait to start. But the level of automation in their development process remains minimal nonetheless. The reason why is usually a variation of:
“Yes, we know that automation is important. But we’re too busy now”.
And I can’t blame them. Automation can feel like a daunting task. Especially for monolithic legacy applications. In the mind of a team, it tends to become this huge ball of tangled wires. Not knowing where to start, and not having the luxury of time, they keep pushing it into the future (“We’ll hopefully have some time after this big project”).
What can you automate?
Pretty much everything you do manually now as part of the development process and requires little cognitive effort. This includes, but is certainly not limited to:
- Copying a new version of your app through an FTP-client;
- Minifying CSS and JS-files into compressed bundles;
- Checking if your application is still running after a deployment;
- Calculating metrics to ascertain the quality of your codebase;
- Running scripts on your database (on any environment) to migrate it to the most recent version;
- Changing parameters in the configuration of your application per environment (like a connection string or SMTP-settings);
- Setting up a new virtual machine in one of your environments;
- Clicking through some parts of the application after it has been deployed to make sure it still works;
- F5-ing the web application on your production environment to make sure that it’s still running;
- Running your web application in a number of browsers to check for compatibility;
- Creating a backup of the current application in production before deploying a new one;
- Running a build on your most recent codebase to see if it still compiles;
- F5-ing your local browser to check the impact of your recent change in code;
These are just some of things that I’ve done manually at some point, and that has been automated at some later point. Pure developer happiness!
Start untangling (with a workshop)
Automation can be hard. But that’s no reason to postpone the journey, or not start at all. Like every journey, it takes only one step to start. So make the first step small and manageable, and move onward from there. A good way to identify the first small step in a fun and dynamic way is with an ‘Untangling Workshop’. I’ll show you how to organize one yourself below:
1. Preparation
Schedule a meeting room for about 3 to 4 hours and invite your team. Or a self-selected group of representatives if you have several teams. Make sure that the room is large enough for people to move around.
2. Purpose
Begin the session by explaining the purpose of the workshop; identify the first (or next) steps in automating manual work (like deployment and testing) that we can start with tomorrow.
3. Energizer
Start the workshop with a creative energizer to get people going. Plunging straight into the subject matter generally doesn’t work well. A fun energizer is to ask people to pair up and visualize the challenge of automation in a couple of minutes. The visualization can be done with LEGO, by drawing on a big piece of paper or some other creative means. When the pairs are done, review the creations with the entire group and identify recurring themes (and have a few laughs). Take around 20 minutes for this.
4. Inventarize manual work
Ask the group to silently write down all the manual tasks that they (themselves) do as part of the current development and deployment process. Write down one task per post-it. You can offer the examples from above to give inspire people. After about 10 minutes, ask everyone to put their post-its on a flip-over. Give people a brief moment to clarify their post-its. Cluster the post-its by affinity or similarity and remove duplicates. This helps us to quickly identify the tasks that are done by more than one person. Take around 15 minutes for this.
5. Estimate cost of manual work (optional)
Starting with the tasks that are done by more people, ask the group to estimate how much time this task takes (in minutes) and how often it is done per sprint (frequency). Write these numbers down on the post-it and multiply them ([number of people] x [time] x [frequency]) to arrive at the ‘cost’ of the task (also write that down). We’re not looking for scientific accuracy here, so keep the pace up. This is a great exercise to quantify the cost of manual work and identify tasks that would greatly benefit from automation but aren’t top-of-mind. Take about 30 minutes for this.
6. Plot manual work on a pain/effort matrix
Create a large pain/cost-matrix on the wall or on a big flipover (this can be done during a break). Briefly explain the meaning of the two axes: ‘pain’ represents a combination of annoyance, (the previously calculated) cost and error-proneness. ‘effort’ is a combination of the time, research and resources required to automate this particular task. Don’t spend more than 5 minutes per item. Again, scientific accuracy isn’t needed here. We’re quickly trying to zero in on what we should automate;
7. Identify steps to automate tasks
Beginning with the manual tasks in the quadrant high pain/low effort, ask the group to identify what needs to be done to automate this starting tomorrow (or next sprint). We often find it helpful to ask for ‘15% solutions’ to avoid getting stuck in finding the ‘100% solution’ (the perfect one). If manually checking a critical path of the application following a code change takes a long time, a ‘15% solution’ might be to create at least one manually-triggered UI-test with Selenium that does this for you. Triggering it automatically on a buildserver after every commit could be a next step.
Depending on the number of tasks in the high pain/low effort quadrant, you can do this with the entire group or with smaller break-out groups that focus on a selection of items. Start with the low-hanging fruit (high pain/low effort), and move to low effort/low pain from there. Try to stay away from the ‘high effort’ tasks, unless the team really wants to also tackle some ‘bigger steps’ over the previous smaller ones. Limit this exercise to 60 minutes to keep it focused and tidy.
8. Make improvements transparent
Formulate the actions identified in the previous exercise into items that can be put on the Product Backlog (e.g. ‘Create per-environment transforms for configuration’, ‘Set-up CircleCI to integrate the code on every commit’ or ‘Setup Webpack to automatically bundle stylesheets and scripts’). Estimate them in story points (or whatever you use to estimate) and make sure they are INVEST. Make sure to discuss the improvements with the Product Owner, as he/she is ultimately responsible for what goes on the Product Backlog. If you’ve calculated the cost of every task as part of the workshop, you also have a pretty good business case.
Concluding thoughts
This workshop helps to break down automation into smaller, more manageable steps. Instead of working from a preferred future state, it starts from the things that hurt the most right now. By automating theses tasks first, the team saves time on the one hand and creates the foundation on which further automation can take place on the other. Let me know if you’ve done this workshop (or a variation of it) and made the first (or next) steps in automating parts of your development/deployment process.
