Agile primer for non technical teams
Whenever I’m coaching teams I give a talk about agile. This article is based on that presentation.
The talk is not aimed necessarily at development teams (although I’ve had good feedback from them). It’s a talk I give to entire companies or divisions because people in sales, marketing, communications, finance and senior management have an important part to play in the development of good software and often they don’t really understand agile or they have incorrect notions.
This article covers a lot of material and naturally when I give this talk to people new to agile, I’m there to answer questions. Therefore, if anything isn’t clear, please chat to me by leaving a comment, I’m always delighted to talk about agile.
PO in a Nutshell
I always start my talk with Kniberg’s excellent video “PO in a nutshell”, it’s been viewed nearly 2 million times. Not bad for a short video on agile.
Kniberg focuses on the role of Product Owner, but I find that the diagrams and explanations offer an excellent and succinct introduction to an agile development process. If you’ve not seen it please take a look now, I’ll be referencing this video throughout this article.
It’s worth noting that Kniberg does a 2 day workshop on this subject so there are a lot of gaps, and it’s here in these spaces that I want to dwell.
The first question is why we use Scrum at all. In the video it’s taken as a given. There are two parts to this question that I want to briefly touch on.
Firstly, Scrum is a framework that promotes agility, and I think it’s really important to emphasise that agile is an adjective.
That is, we want to “be agile” not “do agile” (a term that overly prioritises process).
When we say we want to be agile, we mean use data & focused communications within short iteration cycles to achieve better business outcomes.
If we’re not focused on better outcomes, why are we trying to be agile?
Let’s take a quick diversion into waterfall.
As you know there are two common approaches to project delivery, waterfall approaches and methods like Scrum that promote agility.
If we look at waterfall, we work in stages to flesh out a set of steps that will lead to a big outcome. Let’s say for example that we’re moving into new offices, we can create a plan that details 2 months of work, and of course, 2 months later we know that we’re only halfway done!
And why is that truism, true?
Because with many types of projects, particularly for software development, we learn as we do, which impacts what we do next, changing scope. This concept of learning while making is central to why we use short iteration cycles in agile. It maximises our ability to improve thinking as we build.
Therefore, Scrum is often represented as in the diagram on the left.
But this diagram of Scrum annoys me because it looks like a production line, and viewed this way, it’s easy for Scrum to degrade into a series of mini-waterfall projects. We must avoid falling into the trap of seeing Scrum as a boring repetitive cycle because then it loses most of its value.
It’s only when we add data, entering the system through constant communications with customers and stakeholders, that we see that short iterations allow us to accelerate delivery of customer value.
In short, our understanding of the business domain improves through testing and developing. This leads to scope changes.
Waterfall doesn’t cope well with scope changes because in waterfall scope is fixed.
But in Scrum, scope is variable - we adjust it each sprint.
This ability to deal with change is at the heart of why we use Scrum, and was summed up nicely by Mary Poppendieck when she said:
Late scope change is a competitive advantage.
And I believe it was Martin Fowler who said:
A good predictive project may go according to plan, but through incorporation of late scope change, an effective agile team will build something different and better than the original plan foresaw.
What’s more, by constantly releasing working code (a key requirement of modern agility), we ensure a flow of new data that drives further refinement.
This is why development teams use Scrum or frameworks like it.
Focused communications
The centrepiece of the process is, as Kniberg says, focused communication. The critical word here is focused, we use user testing, models, drawings, flow diagram, designs, whatever works best to ensure focus and shared understanding.
Because communications are critical, Scrum puts a number of rituals in place specifically to ensure there’s space for it.
We have standup, review, retrospective, and sprint planning. These meetings are important. They can be short. In order to deliver value they must be managed by someone that understands, deeply, why we have them. Otherwise they degrade and are often abandoned.
If you’re in a Scrum team and you’re not deriving value from these meetings, then reassess, get some coaching, work out why not. Because the simple fact is that something is wrong, and in my experience it’s generally related to poor use of data leading to unfocused communications (which quite rightly people find frustrating).
Perhaps more subtly, effective Scrum requires compactness, i.e. small teams, doing small bodies of work in short time frames with constant iteration because for one thing, communications don’t scale well. If we keep things compact, they’re far more efficient.
However, there’s something interesting here: Kniberg presents communications as a round table with the PO, dev team and stakeholders in direct contact. Indeed in his opinion the PO is orchestrating.
Yet Scrum often presents the PO as somewhat of a gatekeeper.
The implication is that all access to the stakeholders & customers is via the PO, and sadly I find that is often the way organisations use Scrum. There are many reasons for this. The PO is seen as the product CEO and so external parties naturally interact with the product via that role, dev team members want to focus and are happy to let the PO do the talking, perhaps because it’s hierarchical and human beings just seem to like hierarchy. Whatever the reason, it’s not optimal: constant, short & focused communications with all parts leads to better results.
I believe part of the reason for the discrepancy is historical. While we use Scrum as a framework, modern software companies also try to uphold the values of the agile manifesto, and confusion about the difference causes problems.
Let’s take a quick look at the history.
Scrum was first defined in either the late 80s, or early 90s depending on who you talk to, and is a framework initially used for iterative process improvement within manufacturing.
Conversely, the agile manifesto was created by a group of developers in 2001, trying to improve software delivery. Modern software companies have married the two together.
In the agile manifesto we find 4 core values (shown in the image left) that underpin agility.
The idea is that while there is value in items on the right, we value the left more.
For software development, we use Scrum as the framework within which we embed the values and principles from the agile manifesto.
In addition to the values there are a further 12 principles that set out a guiding philosophy.
I’m not going to elaborate on each of them, they’re fairly self explanatory. The point is, being agile is a mindset, it’s not a process. A central facet of being agile is focused communications around small, easily defined bodies of work (point 4).
And this is why Kniberg has all parties including stakeholders in close contact, because we’ve learnt that it’s essential if we’re to create the best software.
But here’s the rub, if we’re not generating a constant stream of data through release of small testable iterations, then we have to find other sources of data to review or agility will slow or stop and we can wander off course.
A useful analogy is to think of a tightrope walker. Their muscles are constantly making minute adjustments based on input from their senses to keep them on the rope. If they’re making large corrections, something is probably wrong. The same is true for agile: data from small testable iterations of work allows for constant small course corrections, large corrections indicate a problem.
For data, end user testing is preferred, but it’s augmented and supplemented through use of models, designs, and user personas because while ideally the PO should always have access to target users, sometimes this can be difficult. For example, if you’re designing education software, it can be hard to test in a classroom.
The dev team
Let’s look at the dev team a little, because Scrum is vague here.
Naturally within the dev team there’s a set of developers, but what about other roles? For example, designers are not specifically mentioned by Scrum and I find are often badly integrated.
Designers, according to Scrum, are part of the dev team. However design must occur prior to execution of many stories, and also designers are often a shared resource. The PO is grooming stories late to take advantage of best intelligence but designers need time to do good designs. This means designers often work a sprint ahead of the developers.
All this can squeeze design in difficult ways, and I’ve found that design delay is a constant source of velocity degradation.
There’s no magic bullet to fix this, but I find it helps to have designers engage with all Scrum rituals because:
- It’s very useful if designers are present for standup so developers can know where they’re currently focused and can adjust their own work if delays are likely.
- Designers should review front end changes with the PO before release in the sprint review.
- They’re also an important bridge between end users and developers, so are critical in story grooming.
- Finally because they don’t fit neatly into Scrum we often adjust how we work with them in the retrospectives.
Another problem is that designers can find it hard to design to stories, they often prefer to work at a broader, full-system level which lends itself more to waterfall projects.
The best way I’ve seen to overcome this problem is to get front end developers and designers working very closely together so they’re almost paired.
If you can achieve it, the marriage of a good designer with front end developers leads to story aligned designs. By that I mean smaller designs that mature through iteration. If your designer and front end developer are remote from each other, then work to ensure they have a set of tools that allow effective sharing of designs, conversely it should be very easy for a developer to spin up an ephemeral build to showcase their latest work for the designer.
Customer value
Efficient increase in customer value through faster learning is the aim of agile.
Therefore we must know the customer. It’s pretty hard to work out what’s valuable to someone without drilling into their motivations.
We construct customer personas by working with the stakeholders, the PO and UX designer and, of course, the customer themselves.
The better engineering know the customer via personas or direct contact the better the resulting software, because our focused communications are greatly improved and there are fewer misunderstandings.
If we return to Kniberg’s customer value graph:
It’s clear that his “knowledge focus” stage happens early on (often pre launch of a new product or feature where data is relatively scarce) and therefore gaining knowledge to build on requires R&D (that is spending time coding options and working out how to approach things).
It’s very hard to estimate this early R&D effort, and, at the same time, companies can feel rushed during this stage. There may be very real commercial or investor pressures driving to launch, however if we don’t do the R&D, growth in customer value is retarded by potentially costly mistakes.
How do we balance costly R&D vs launch pressure?
Again we use data wherever we can find it. Real data from real users allows us to spot areas that need attention early so that we expend effort in the right places. For example, we try to check small sections of design work with real end users or stakeholders before writing code.
Looking at the graph, Kniberg notes that customer value increases slowly during the knowledge phase and the rate of customer value accelerates as the teams gel and knowledge is applied more efficiently.
The corollary to this is that communications load is high during the knowledge phase and there’s an equivalent decrease in comms load over time. This is critical to understand. Developers need more time with people from other parts of the business and customers, early, to develop great software.
It’s during the knowledge phase that we make many of the important architecture decisions that are difficult to change down the line. And so it’s critical to spend more time ensuring shared understanding in this phase to avoid costly mistakes that retard the growth phase.
Note that this can be a difficult time for new teams and products. People are working things out, and there’s often language confusion. Managers must be prepared to defuse tensions and enable communications.
Forecasting
Naturally the business wants to know “when will it be done”?
To make a forecast the PO needs data, specifically:
- A complete(ish) backlog, i.e. a list of activities that when completed give the product.
- The backlog needs to be estimated.
- We need to know the team velocity (how much work they do per sprint).
In the early stages of product development, we don’t have this information.
The backlog is loosely defined, velocity varies widely because we’re in the knowledge phase, and the team is bad at estimating.
It can take many months to resolve this, but note, neither would we have this data using waterfall. The difference is that waterfall provides a start to finish narrative that management likes. Agile more honestly shrugs and says “hmm, tricky…” which really gets up the noses of people unfamiliar with the limitations of forecasting.
This is the velocity graph for a recent project over 6 months. The variance isn’t bad for this phase of the project, but still we’re jumping from below 40 to above 80 story points per sprint. Still you could guesstimate it at around 60.
However, during this period we added 2 team members, suffered a very distracting trade show, Christmas, and unexpected R&D efforts, so actually it’s hard to derive much value from this for forecasting.
Velocity is only one out of three variable factors that are required for forecasting. Estimating backlog items is equally hard (during the knowledge phase).
Kniberg draws reasonably close lines with a small amount of variance when looking at forecasting which makes it seem useful, however I find that because of the uncertainty with estimating and velocity, forecasts become fairly meaningless within a couple of months. At that point the spread is so great that there’s little value in making them.
However, businesses need forecasts so instead I find it common for POs to create roadmaps and design pyramids to define general targets, then adjust sprints to deliver essential functions.
Is this bad?
I think it’s brutally honest. Forecasts are forecasts and they degrade the further out you look.
This is why a core value of the agile manifesto is working software. Each sprint should deliver more working software that people can use.
The cultural problem here is that a degree of faith is required from management. They need to believe that a good agile project will deliver something better than waterfall could achieve. This can cause problems unless there is strong understanding of agile in general. This leads on to the:
Scrum event horizon
This is a discordant fringe that annoyingly circles agile teams. It’s the boundary between those people who understand why we want to be agile and those that don’t.
This is where I find Scrum can lose a lot of its effectiveness. Because it’s at this boundary that we find people who think Scrum is a methodology used by developers, rather than a mindset adopted by the company for competitive advantage.
To be agile we want the company to embrace an agile mindset, otherwise it degrades into a method for process improvement (which was its original aim after all).
Getting rid of the Scrum event horizon is the job of the Scrum Master, and they will do it through education and of course, sharing this article :).
Build the right thing, build the thing right
Last but not least we come to this lovely image showing the tension across Scrum roles.
This diagram reinforces the idea that in Scrum, roles are not about authority, but about scope and mindset.
Each part of the Scrum team is adding knowledge from their perspective to allow us to balance these three things.
This subject is so important that I’m going to devote another article to it soon…
The Agile mindset
And this takes us full circle:
we don’t do agile, we are agile.
The agile mindset, described in the agile manifesto, underpins agility.
If you find yourself thinking in terms of repetitive process, then I ask you to visit the agile manifesto and think again.
This is especially true for stakeholders, and everyone who is not a developer or PO, because of the rich domain knowledge and customer access they bring.
I know I’ve belaboured this point, but, at the risk of flogging the proverbial horse, let’s do it one more time.
The data that we need to drive agility for competitive advantage comes from outside engineering. It comes from talking to customers, from conversations with sales and marketing, through interactions with personas created by the business.
By the time developers have stories to code, the agile part is done. All the thinking that gives a competitive advantage is embedded in the tickets (via story grooming).
Summary
This article is primarily written for people who need to understand agile because they may interact with Scrum teams. These people will be in product, marketing, sales, or on the management team. I hope to help them see that their interactions with the Scrum teams are essential to extracting competitive advantage from agility.
Finally I write articles to encourage debate. Please comment politely if you disagree with anything I’ve said, and share freely.
Thanks
