The fundamental misunderstanding that is destroying agile
Let me provide some context with the following imaginary conversation.
Chief: We want to build product X and hope make 5 coins out of it. So development of the product can cost up to 4 coins. 2 coins for marketing and sales and 2 coins for R&D.
Chief: Devs can you build our product for 2 coins within 1 year?
Devs: Hold on Chief. That’s 26 sprints and from the past we know that we can complete about 5 story points per sprint. That’s 130 story points. Just to be save, let’s say 100 story points. You’re probably going to change your requirements again.
Devs: But Chief, don’t try to convert story points into coins. The agile people say that only stupid people do that.
Chief: Okay, how do I know if my requirements can be completed with 100 story points?
Devs: You write your requirements in the form of User Stories, please make sure to stick to your DOR and DOD definitions. Then we refine the stories together and assign story points to each User Story during planning poker.
Chief: We require some external help to sharpen our requirements. Getting all requirements sorted out could easily take 2 months. Your refinement and story point poker game will take another month. Are you saying that I have to invest 3 months to even start? I thought agile would make us faster?
Devs: Hold on Chief! If 3 months go into planning, we don’t have 100 story points but maybe around 75. And please remember that we are talking about a completely new product. You have to expect the first 3 months of estimations to be completely off.
Chief: ???
Can you relate to this conversation?
What’s the issue here?
The chief wants a somewhat reliable estimation for the devs to decide on whether to pursue the new product idea.
The devs want to be reliable partner and understand the details to come up with an estimation.
I’m not sure how you feel about this, but I think both sides act seemingly reasonably. What do we do? 3 months of requirements engineering before building anything. Sounds an awful lot like waterfall if you ask me.
Let’s dig a little deeper and try to find the underlying problem.
The issue
Let’s say we did invest those 3 months to write all the User Stories, assign the user stories and add up the story points. Let’s even say we would come up with 70 story points. What would we have gained?
We’d have an estimation (70 story points) for efforts that would probably fit into the estimation for the product cost (2 coins) that would be worth it, if we can make the estimated earnings (5 coins). There are still loads of if’s in there.
All projects deal with huge amounts of uncertainty. We can’t make it go away. We need to recognise uncertainty and manage it by continually evaluating our options and priorities and adjust accordingly.
If business people and developers recognise this and really collaborate, then overspills in sprints and changing requirements are expected and an opportunity to adjust, instead of a trigger for blaming.
The goal shouldn’t be for the developers to stick to the 2 coin budget, but for business people and developers to maximise profitability together.
Not recognising uncertainty leads to unrealistic expectations. The expectation to get a reliable estimation is unrealistic. It creates an environment where developers will react defensively and invent complexity to make sure that estimations are always really high in order to escape the blame.
This creates an environment of pressure and fear.
Ever tried to be creative under pressure and fear? It doesn’t work
In the research on developer productivity, Gene Kim identified 5 ideals:
- Locality and Simplicity
- Focus, Flow, and Joy
- Improvement of Daily Work
- Psychological Safety
- Customer Focus
Let’s start with focus, flow and joy. Software engineering means to fully understand the business domain in detail, while incorporating performance, reliability, maintainability, security and cost optimisation.
Hence, in order to write good software a developer has to be in a state of undisturbed focus to find a solution to bring all these elements together. Being in that state, when typing these thoughts down as program code feels natural and easy, we have flow. This “figuring it out” and getting it done, essentially creating something from nothing, is very rewarding in the sense of feeling an accomplishment and joy.
I think it’s very clear that fear and pressure would make it hard to work like this. The other side of the coin here is the 4th ideal psychological safety. When there is pressure and fear, people tend to hide problems or cut corners just to avoid the blame.
All these cut corners and hidden problems have the potential to blow up and come back to haunt us. Murphy and his law, right?
The solution
I already mentioned it, but recognising the uncertainty and facing it head on collaboratively is the right way to go here.
Moving from commitments to validation.
Subconsciously or even consciously we all know that we deal with uncertainty. That’s the reason behind the demand for commitments to efforts timelines and costs. But really, we need to stop this blind trust in someone’s estimation.
The answer agile gives to uncertainty is to have quick feedback loops. That’s why we have iterations. Iterations are to validate and adjust course not to meet committed stories of a sprint. This is not just true for the development time and efforts but also for the business side.
How will we know if customers are going to like and buy our product? We need feedback and we need it quickly, if we’re going to take it seriously and use it to adjust our product strategy.
Instead of spending 3 months guessing efforts and cost, we should instead get ideas and prototypes into the hands of customers and use their feedback to validate if we’re on the right track with the idea in the first place.
That’s agile. Agile means nimble to change, being able to move in the sense of changing direction. We can achieve this with the right mindset and short feedback cycle.
The fundamental misunderstanding that kills agile is believing one can enforce certainty.
The problem here is that when we believe there is certainty we think the timelines, budgets, cost and quality constraints are realistic and any deviation is due to poor performance. This in turn creates pressure and fear which leads to poorer performance.
The core idea of agile is the answer. Work in small increments to get feedback quickly and use that feedback to adjust your course. We can make long term plans and goals but they should be treated as hypotheses who’s validities should be validated frequently.
The mindset of the team should be to maximise the business outcome, instead of meeting a deadline.
