Assuring Agile Delivery
Assurance and Agile — two words not commonly seen together, and for good reason. The early and iterative delivery of product increments using Agile, constantly reviewed and inspected, provides vastly more oversight than any traditional assurance approach could ever hope to achieve. Inspection and Adaptation is at the heart of Agile and ensures the myth that Agile is about cutting corners on quality, is just that — a myth. There is no place shoddy work can hide in an agile approach — a focus on transparency ensures that.
So what’s the problem?
Organisational assurance processes typically mirror their traditional delivery counterparts, namely that are gated and sequential. However, whilst Agile’s inbuilt inspection processes are a step change over traditional assurance approaches, a number of problems remain:
- They don’t provide an out of the box approach for independently (outside) technical review of the deliverables or the processes,
- Many company’s traditional “tried and tested” assurance processes have yet to catchup or full acknowledge that Agile has arrived, even though the application of the approach is now mainstream and has been for at least a decade.
In some business domains the above are significant concerns, for example where there are Safety or Regulatory considerations. Notwithstanding outside requirements, an independent review culture is widely considered highly desirable for a range of important reasons, including:
- Ensuring corporate knowledge of previously learned lessons are applied
- Removing group bias that can get baked into a team
- Spreading knowledge
- Wider cultural benefits (e.g. developing more junior staff, promoting technical excellence, fostering curiosity etc).
Approaches to assuring traditional delivery and Agile have to be different. The premise of assurance in a traditional world is typically based on phase gates and/or product maturity. Both present problems for Agile:
- Phase Gates — the concept of phases/stages is central to traditional development lifecycles, and enshrines the key issue with traditional approaches, namely they result in right shifting of feedback and risk. They also bake in inflexibility and they create a culture where learning and rework is considered negatively (assurance events are to be feared). Furthermore, they foster a lack of transparency and in many cases a lack of honesty. For these reasons, utilising phase gates for Agile would not be a good direction to take and would be counter productive
- Product Maturity — Product maturity often gets confused with Phase Gates, but there is an important distinction which is best illustrated by an example. A team may only ever aim to deliver a product that is of low maturity e.g. a prototype. The level of assurance therefore is not so much around phases/stages, rather it is tailored to the expectations around the deliverable. You would not expect a model of a rocket to carry the same level of assurance, as the real thing. The main challenges with product maturity based assurance in Agile is twofold. Firstly, the pace that Agile works (measured in weeks), and secondly that many Agile teams now service multiple products — operating more in a software as a service model — think ITIL rather than Prince2.
In my experience, traditional assurance processes frequently struggled to keep pace with agile delivery, or simply don’t deliver value to the teams leading to a value disconnect between the two systems. Without all parties seeing the benefits of assurance, the true value of it can never be attained.
Don’t forget Complexity
The pride company’s take in their assurance processes has always been something of a mystery to me. Whilst companies often see them as differentiators, the reality is that the key business value doesn’t come from the assurance process, rather it comes from the skill and expertise of the assurers/reviewers and having receptive receivers for the advice. The focus of assurance should be to add value, but frequently the emphasis seems more focused on providing a superficial due dilgience excuse for middle management.
Many coorporate assurance regimes are backed with cavanas and carefully crafted manuals and handbooks, complemented with checklists to simply things. There is clearly some value in these (e.g. as demonstrated by the WHO medical checklist (ref 1)), but in software assurance I suspect less than you might imagine. The reason for this is that Agile is popular in environments which have high degrees of uncertainty and variability. Dave Snowden’s Cynefin (ref 2) describes these domains as “Complex” and this is the domain of emergent practice. Novelty and flexibility of delivery is often the differentiator, therefore applying a rigid checklist just results in many considerations being recorded as not applicable (n/a) with massive checklists that cover every possible delivery scenario.
This is also ignoring the obvious fact that the products themselves could have very different considerations dependent on the market they exist in. For example, assuring a medical device is likely to require a very different set of expertise than assuring an aviation product. This takes us full circle back to the real assurance asset, namely “Individuals (reviewers — suitably qualified, experienced and knowledgable) and interactions (between the reviewers and the delivery teams), over processes and tools”. Sounds familiar? If should, as it is the first line from the Agile manifesto.
Agile Assurance Requirements
There are three key considerations around understanding the success of Agile deliveries:
- The success of Agile is heavily dependent on having appropriately skilled, empowered and motivated teams. Team’s will have micro cultures, and these cultures will be heavily influenced by the wider environment,
- Iterative (Agile) process are by their nature, highly repetitive and repeatable, but they must retain the ability to be adaptive — so change should be expected. It is distinctly unagile to prevent process adaptations (including sometimes quite radical ones). There is value in assuring that customisation/optimisation of working practices is taking place,
- The products being delivered by an Agile team are highly predicated on interactions between the development team and the product owner, and the context that they are operating in.
What does this tell us about assurance of agile deliveries? Well, the key conclusion is that we need a multi-pronged approach:
- Requirement 1 — We need to understand the health of the team and the environment within which it operates— the best teams will proactively do this, but a team health check is likely to be a key leading indicator of how well things are going
- Requirement 2 — With a healthy team, we need to check that the deliver processes are suitably robust and free to inspect and adapt to meet the delivery challenges. Processes should be empirical.
- Requirement 3 — As context is critical in understanding the products being delivered by an Agile team, any independent assurance of them has to be seen as delivering domain specific high value insights. This value has to be recognised by the wider team including the business stakeholders (e.g. the product owner) and utilise the standard mechanism for work prioritisation (e.g. the backlog)
Delivering Requirement 1 — Health check
Team Health is a relatively simple thing to gauge (there are numerous health check tools on the market — although many also stray into Requirement 2); a competent Scrum Master should be able to provide evidence of this. However, a mature organisational culture of respect and trust will be required if actually team health results are to be more widely shared and made transparent across a large organisation.
Delivering Requirement 2 — Applying an Agile Framework/Approach
Armed with the output of the health check. checking that an agile mindset and processes are being utilised is also straightforward activity. Subject matter knowledge of Agile is required, but knowledge of the delivery domain is not. Independent assurance is likely to take the form of conversations with Scrum Masters (who themselves act as a kind of independent agile assurance capacity), observing key events, and looking at evidence such as retrospective write ups, metrics etc. Also, checking DevOps style metrics around things such as cycle time will shine a light on how much automation has been embraced. Agile Coaching functions are typically effective at providing this assurance capability.
Delivering Requirement 3 — Advise on the Product
Having assured the environment and the processes, the job of assuring the products being produced, should be straightforward. Reflecting an iterative process, the Scaled Agile Framework recommends this is done at regular intervals — in the case of SAFe at the end of the Program Increment (ref 3). Whilst this synchronisation point is unique to SAFe, all Agile methods operate to a very predicable cadence. Therefore slotting in a reoccurring assurance review every n-sprint is a simple scheduling problem — much simpler to accomplish than in the much more chaotic and unpredictable world of traditional sequential (waterfall) deliveries.
The frequency of the assurance events will need to take into account:
- the batch size of what will be reviewed (too much content to review may result in more shallow assurance), together with
- the implications of delayed feedback (will it be practical to address certain comments if “too much water has gone under the bridge”).
Critically, the value and frequency of assurance events is also likely to be governed by a number of other important factors. A quick prototype with limited quality aspirations may never fully benefit from independent assurance, where as a safety critical medical device deserves much more oversight.
However, this grossly simplifies the number of relevant factors to consider. The criticality of a product may also be influenced by how much money is being invested in it (e.g. it could be make or break for an organisation), it could be that the regulatory environment the product is operating in has some harsh sanctions for failure, or it could simply be that the risk to reputation or relationships is too great for an organisation to jeopardise. Politics and money are important considerations. Alistair Cockburn, an Agile Manifesto signatory, recognised this an in his Crystal framework in which he presented a tailored set of processes based using what he described as “Criticality” (ref 4).
Therefore, to get value from product assurance, careful upfront assessment is required to establish who would be appropriate assurers (reviewers — suitably qualified, experienced and knowledgable) and how frequent these assurance events should be. Traditionally assurance functions have been good at the latter, but poor at the former.
The ultimate destination
With the above being said, I see assurance, even as described above as a sticking plaster for ubiquitous corporate failings. These failing are a lack of curiosity within the organisation and its workforce, and a suboptimal ability to share experience and knowledge. In a utopian world, teams would have easy access to experts to advise them, and teams would have the thirst for feedback by engaging experts at appropriate points in time. In this utopian would, assurers would be proactive, and armed with a highly transparent organisation, would sprinkle their magic where it delivered most value. Communities of practice would organically emerge and develop/offer guidance on best practice to teams.
This utopian view aligns with Teal organisational design (ref 5), which would be considered a novel concept and an unrealistic end state for most current executives. Hierarchies, politics, self interest, economics, compounded by limitations on human social networks (Dunbar’s number — ref 6) ensure that progress towards that level of empowerment is slow at best. For the time being, we must operate in a hybrid world — a world where Agile and traditional organisational design has to work together. Assurance is only one cross cutting concern, but an important one. Assurance in an Agile world is not as difficult or as challenging as many seem to believe it is — and tracking team health and nuturing an Agile Coaching function are on the path to become a more dynamic organisations.
In most respects Agile makes assurance much simpler and predicable. But to get value from assurance processes in the modern world, we must refresh our approach and shake off the last remaining vestiges of traditional delivery thinking. At the very least, we must consign the notion of phase gates to history.
The insidious institutionalisation of “water-Scrum-fall”
What the hell is Water-Scrum-Fall and why you should not go near it …
References/Follow on reading: