“…our estimating techniques fallaciously confuse effort with progress, hiding the assumption that men and months are interchangeable.” ~ Frederick P. Brooks, Jr.
Software development estimates are hard, and they are unreliable. This is not a new problem; we’ve struggled with estimates for decades. Yet, we still insist on them — they provide a (largely false) sense of security.
The problem lies in the nature of the work that we do. It often requires reasoning and problem-solving; we need to sit down and wrestle with a problem until we can “figure it out”. Our estimates attempt to bring some degree of predictability to work that is inherently unpredictable.
To illustrate — let’s say that you are familiar with the rules of chess. If I put a chess board in front of you, with a game in progress, and I tell you that white can mate in three moves — how long will it take you to figure out what those three moves are? Remember, you know the rules, you know how the pieces move and you know what the board currently looks like. In other words, you have all the information that one would imagine you’d need.
Can’t estimate it? Well… software development is pretty much the same thing.
Why estimates are difficult to get right
For software projects, we have to consider a number of elements — including technology, people, requirements, and external dependencies. Variability exists in each of these.
Technology in itself is complex — there are a multitude of programming languages, libraries, and tools. Unexpected issues often manifest themselves when we put all of these things together. A supposed five-minute change to upgrade a library version can turn into two days’ worth of effort because the new library version is incompatible with a different library somewhere, which means that needs to be changed as well. In turn, that requires additional testing. A typo in a line of code can result in hours’ worth of debugging.
People aren’t always equally productive either — and estimates assume that we’re all equally competent and that we’ll work at a constant pace, regardless of whether we’re tired, frustrated, or ill. It doesn’t work like that in the real world. Interruptions happen, and sometimes someone needs to spend an hour or two helping a colleague (and that’s not a bad thing).
Requirements can be unclear, or they can change as new information comes to light (and they should). If a developer starts implementing a new requirement and discovers that something doesn’t quite make sense, they need to get hold of someone who can clarify it for them. But what if that person is on leave, or stuck in meetings for the rest of the day? The “wait time” results in a delay overall — which means that one single question around a requirement can invalidate an estimate.
External dependencies are harder to control than requirements changes — if we move into a world where we depend on external vendors, we are subject to their SLAs to address issues. If the external dependency is an API, it means that we have introduced multiple additional points of failure — any issues with our servers, the vendor’s servers, or any of the network components in between will introduce delays.
Estimates can also lead one to believe that work is divisible — if a feature is estimated to take 10 days to develop, does that mean that two developers can do it in five days? Or 10 developers in a day? Unfortunately, nine women cannot make a baby in a month. In fact, adding more developers to try and speed up the process only serves to add communication overhead, which is likely to make things worse.
Why stakeholders want estimates
It’s not difficult to understand why our stakeholders want estimates. Ultimately, someone is paying for the work that we do. They have budgets, they have deadlines, they have customers to satisfy, and they have ROI to calculate. If we can tell them what to expect and when, that process becomes easier.
In some other industries, tasks can be sufficiently well-defined that we can reliably estimate them. If we run a production line assembling widgets, and we know that it takes exactly 10 minutes to assemble one widget, we know that we can assemble six widgets in an hour. However, our widgets (unlike our software) are standardized — so unless the production line shuts down, it’s unlikely that we’ll deviate from the norm. The same thinking that works in other industries cannot be extended to software. Every innovative solution that we build involves solving a new, novel problem. We can draw on past experience, but we’re not running a production line.
Traditional approaches to estimation
Hours
This is self-explanatory — we estimate that a particular task will take a certain number of hours to complete. In practice, human beings are terrible at this and we usually get it very wrong. This is also dangerous in hourly estimates vary across different team members — a two-hour task for a developer with 10 years’ worth of Java experience is likely not going to be a two-hour task for a graduate who started learning Java a month ago.
Story Points
This is the approach proposed by Scrum.
Story points are a relative unit of measure that compares tasks to each other. In other words, we’ll start with a proxy task such as “add a screen with five fields, an API endpoint, and a database table behind it” and allocate an arbitrary value (usually based on the Fibonacci sequence) to it. If our proxy task is allocated five story points, and we estimate our next task — “add a screen with ten fields, two API endpoints, and a database table behind it” — we might decide that the task is a bit more complex and allocate eight story points to it. Story points do not represent days, hours, or any other time-specific metric — they are merely a way to compare different tasks.
We then calculate the number of story points that a team completes over the course of a sprint and use that to determine the team’s “velocity”, which is a metric that gives us an indication of how many story points we can complete during a sprint. This metric is based on the team as a whole, rather than individuals.
The process of estimating story points is called “planning poker”, and it happens during backlog grooming sessions that include the whole development team. There’s a lot of opposition to story points, but the process of assigning story points is immensely valuable as it allows a whole team to debate and understand a task. I personally believe that it’s still a much better approach than trying to estimate hours.
Alternative approaches
We use estimation to create a sense of predictability. While estimation is difficult and unreliable, there are other ways to create predictability.
Predictability can come from regular delivery cadence — in other words, we deliver something to production on a regular basis (i.e. a deployment after every two-week sprint). While we can’t reliably predict what will go into production, we can almost guarantee that something will be deployed every two weeks. Since we define stories as units of work that add actual business value, it means that we are continually progressing towards a better state.
While development work is often not divisible, we can streamline it by allocating more brainpower to a problem — i.e. pair programming or mob programming. This isn’t a case of dividing work, but rather collaborating on solving the same problem. This can speed up development, since the actual work is problem-solving rather than typing — two heads are better at solving a problem.
Transparency is key — unexpected surprises are par for the course, but as a development team, make sure that your stakeholders know about your progress and the issues that you run into.
Break tasks down into small, granular units. Small pieces of code are easier to develop, easier to test, and more predictable because of their limited scope.
If you have to provide an estimate, be clear on what the preconditions of your estimate are. As an example, your estimate of three days to implement a feature might assume that requirements won’t change, that there won’t be any library conflicts, and that all external API’s will be be available at all times. This might not be realistic, but by being explicit, you also help your stakeholders to understand the underlying complexity.
In conclusion — estimation is difficult. Use it as a guideline and a target, but remember that your estimates are likely to be wrong. Instead of trying to get your estimates 100% correct, focus on understanding tasks, incrementally deliver small pieces of work, delivering value frequently and being transparent with your stakeholders.
P.S. Also check out the #noestimates movement.
Disclaimer: As always, the views expressed in this post are my own.