Agile and DevOps with Mainframe teams. Throw the book away!
OK so throwing books away is maybe a little extreme, but so is implementing everything a book or an expert says, word for word, process for process, especially when the book or expert have no knowledge of your team or organisation!
Agile and DevOps are two of the biggest buzz words in the industry and are synonymous with digital transformation, if an organisation does not already practice Agile and DevOps or is not transforming itself to adopt them it is viewed by many as a big problem ….. and that is where the problems start, as more often than not this leads to an organisation-wide transformation programme, aka check box exercise, of implementing Agile and DevOps processes and tools in a uniform manner rather than focusing on the outcomes that these practices aim to deliver by selecting and adapting processes and tools to fit the needs of the teams and organisation.
What’s in it for me?
Large scale change initiatives often fail and the reason often given for this is that human beings are naturally resistant to change, but we humans make and accept changes in our lives every day, so how can we be naturally resistant to change? Much more likely the reason is that many of these initiatives fail to address the key concerns and motivators of individual employees and teams. Most often only the benefits to the organisation are clearly communicated. For every person in the organisation that needs to be involved in the change they need to have the answer to the question: what’s in it for me?
A 2016 Coleman Parkes study of 1,770 senior enterprise IT and business executives found that adding DevOps to Agile practices improved new business growth by 63%, advanced DevOps users see a 42% improvement in speed to market, and > 80% of enterprises that have adopted Agile and DevOps practices see an improvement in customer experience. These are the types of statistics and reasons often used to describe why an organisation is embarking on an Agile or DevOps transformation, but this does not tell individual employees or teams whether or not the change will be better for them or even what will change, they are not involved in the change, it is being imposed on them.
Focus on outcomes over methodologies and processes
The next step and error that most organisations make is to select the methodologies that all teams will use, thus assuming that one size fits all. When CA Technologies (now Broadcom) embarked on an Agile transformation in our Mainframe Business Unit back in 2013 we made this mistake too. We mandated that all teams adopt Scrum with 2 week sprints and then later we also mandated that all Value Streams or Business Units adopt SAFe in order to solve the challenge of scaling Agile. We also had a central quality initiative that mandated all teams to have quality targets such as MTTR (mean time to resolution), which needed to improve every quarter. We did see benefits from this, but we also had quite a number of issues, we could do better, we could improve.
With an aim to improve we gathered a group of people (managers and individual contributors) representing the organisation and held a 2-day retrospective. We learned 3 key things from the mistakes we had made in the past.
· Focus on achieving outcomes not on implementing processes.
· One size does not fit all!
· We need to provide teams with enough time and support to make improvements.
Then we identified 3 key outcomes we wanted to focus on: delighted customers; happy and engaged teams; and fast, quality delivery.
Communicate, listen, adjust, repeat
Then all we needed to figure out was a way to help our people understand why these outcomes are important for the organisation, for them as individuals, and for our customers and at the same time ensuring we sought feedback and were prepared to change our plans if needed. We also needed to figure out how to enable our teams to deliver these outcomes without telling them exactly what to do or how to do it and at the same time ensuring they have the support, time, and resources needed to be successful …. Wow does that sound like a fine balancing act or what?!
We communicated these plans to everyone and we listened to feedback. We told our product teams that we expected them to complete at least 1 story per quarter that would help them to improve with respect to 1 or more of the 3 outcomes we are aiming for and that they must measure the improvement. Outside of this exactly what teams choose to work on and how is up to them.
We did have one specific requirement for every team, they had to do a process map (fig 1) to describe the steps and time taken to go from code check-in to being ready to deliver to production/customer. We believe from experience that this is absolutely necessary in order to understand where the bottlenecks are in the continuous integration process prior to working on improvements. It also enables a team to be able to articulate the return on investment from the work they do, e.g. if manual regression testing currently takes 1 week of effort and elapsed time for 1 developer/tester and needs to be run once per month and the average salary of a developer/tester is $100k, then the annual cost is 12x(100,000/52)=$23,077. If this can be automated with 2 weeks of effort for 1 developer then the savings will be $19,231 in the first year and $23,077 each year thereafter (not accounting for a small amount of effort required to maintain test cases). This in turn will help management to prioritise this against other work in the backlog.
Remove Waste
One of the biggest complaints we had from engineers about SAFe was the number of meetings they needed to attend that were not relevant to them, so we reduced this to only 2 per quarter, a business context meeting to help everyone understand our achievements from the past quarter and goals for the upcoming quarter and a final readout of each teams objectives aligned to our epics, so that each team can see how their objectives fit the big picture and what other teams are doing to contribute to this. Total time per quarter = approx. 2.5 hours. Teams now plan on their own and we review the output with a much smaller group to identify any dependencies, set priorities, and business value.
Feedback is the goal, not sprint length
We also had complaints from some teams that 2-week sprints were too short, as writing code in lower level languages like assembler, metal C, Cobol, and PL/I, often takes longer than higher level languages like Java and C++. This resulted in sprint planning, reviews, and retrospectives not providing value, as work from the previous sprint was not completed and was just carried over to the next sprint. We gave teams the flexibility to choose their own sprint length of 1, 2, or 3 weeks. If they wanted longer than that we would also allow it, but they need to explain why it is needed and what the expected benefits will be. One of the key goals of Agile is to get feedback from end users on working software as early and often as possible. This ensures that you do not work on features that are not needed or implemented in the desired way. Doing this significantly reduces waste and thereby increases productivity, allowing Agile teams to deliver value faster than Waterfall teams. Agile teams don’t write code faster, they just spend more time writing code that is needed. In our case there was no point in having all of our teams stick to shorter sprints, as many did not have working features to be able to demo and get feedback on.
How do you know if it’s working?
The feedback from our teams on these changes has been hugely positive and anecdotally we are starting to see increased efficiency and engagement, both internally and with customers. Are we finished, certainly not. We continue to seek feedback and look for opportunities to improve. We have also asked all our teams to collect 3 qualitative metrics (1–5 rating) twice per year: team happiness, customer engagement, and product quality.
While the changes we have made to our Agile and DevOps practices are starting to bear fruit, we have a way to go before we can call it a great success and we will never be done, we will need to adapt to changes and improve continuously. What we have done may or may not work in your organisation, but if you focus on the outcomes you want to achieve and choose and adapt the processes and tools from Agile and DevOps to help you then I firmly believe you will set yourself on a path to success and continuous improvement.
What’s next and where can I get help?
There are many topics related to Agile and DevOps with Mainframe teams that I have not covered in this blog, such as culture, structure and collocation of ops and dev teams, specific challenges facing Mainframe teams re. adoption of DevOps and Agile, application architecture, and tools. I will endeavour to address these topics and share the progress shown by the metrics we are collecting in future blogs. If there is anything you would like more details on, please leave a comment.
