FROM THE ARCHIVES OF PRAGPUB MAGAZINE April 2013
Estimation: The Best We Can Do
By Ron Jeffries
Two months ago in these pages, Ron Jeffries told us estimation is evil. Now he’s back to tell us it’s a necessary evil — and that, done right, it isn’t even evil.
Yes, there are many abuses that organizations perpetrate around the notion of estimation. Most developers have been injured by these abuses more than once in their careers.
Unfortunately, this leads to a common naïve view among would-be Agile practitioners whose learning is at an early stage: they want to abolish estimation entirely. This is probably not possible and it is certainly not ideal. We do have the ability to estimate how rapidly we can do some things, and the business can make better decisions if we share what we know.
Development isn’t just a smooth-running machine that cranks out features rapidly. I know it’s comfortable to think we have no responsibility for the business’s concerns of money, time, and dates. It’s just not true. What we build is not solely the purview of some Product Overlord who decides what we’ll do and takes all the risk. The business people may make final calls, but many of the decisions are best made by the team — not just how to do things, but what to do. We do have a responsibility to help guide the project, using our creativity and our special knowledge.
The Agile Manifesto says, “The best architectures, requirements, and designs emerge from self-organizing teams,” and that’s what we meant (emphasis mine). Agile developers often have a good sense of how long things will take, and this is a valuable component of selecting and refining requirements. Developers need to step up and talk about the probable cost of what they are asked to do.
Concerns with Estimation
There really are a number of serious problems relating to the estimation of software work. In a previous article, “Estimation is Evil”, I described some of them. Let’s review:
- Predefined backlogs of work reduce creativity and inhibit steering the project for success.
- Demanding delivery of “everything” by some fixed date is a trick that never works.
- Treating estimates as promises leads to conservative teams and disappointing results.
- Trying to improve the “quality” of estimates is attention that could be better paid to improving the quality of the product.
- And so on.
In that article, I argue that the result of this focus is “weak Agile.” In essence, teams focused on estimation are generally working the short end of the lever of cost vs. value. Better teams focus more on value and less on cost. They still consider cost, because cost is one important component of delivering value. But value is their primary interest.
It’s true that many top teams do not estimate in any real sense of the term. They work on small stories, small enough that to get a sense of progress you can just count them. They work on the most valuable things first — using whatever notion of value they have. They adjust their outlook frequently and adjust what they work on next in accord with what is happening. They work to produce a continuous flow of valuable software, and they deploy it as frequently as possible, often daily.
Since estimation is often associated with dysfunctional or weak teams, and since great teams seem to work without estimation, many Agile practitioners strongly resist estimation of any kind. And it’s good to build up a team — and an organization — that is strong enough to work without estimation. But this is an advanced way to work, not the way to start. It’s not good to refuse to estimate, or even to resent it, in an organization where it’s needed. On the contrary, we need to learn to do it well, in a way that influences the organization positively.
Budgeting a New Project
Now, let’s look at just one legitimate need, helping the organization decide whether to go ahead with a project.
When something goes wrong with our house or our car, and we don’t have all the money in the world, we probably get estimates on fixing things. We probably consider alternatives: if our trunk lid is scratched, should we repaint the whole car for a perfect color match, just spray the trunk lid, try to polish out the scratches, or just live with a scratched trunk? We pick an option that seems to be the best use of our money.
Business people budgeting projects have the same problem of allocating funds. They need to consider how to deliver as much value as possible, within budget. In a small organization trying to create some amazing new product, the concern is even more pressing. We need to start generating a positive cash flow before we run out of money!
In situations like these, we really need some estimation. We need to know whether it’s wise to undertake this new project. Business people have some estimates in mind already: How many people will be interested? What percentage of prospects will turn into buyers? How much will buyers pay for the product?
But they need to decide: Can we begin to bring in enough money to stay alive before our cash runs out?
The decision maker knows that all these estimates are estimates, and uses his best judgment to put value to them. But there’s a key area he doesn’t know much about: How much of this product can we get done, by what date, for how much money? This is a technical question and it needs to be answered by technical people.
Yes, of course, we don’t know either. We really don’t know how long it will take. But let’s face it, we have a better idea of how long things take than our poor entrepreneur has, and he needs our help.
I believe that we can make a good guess at whether it’s a week, a month, a quarter, or a year before we start getting a good sense of how fast we can progress. And that guess, wild though it may be, is a better guess than the business person can make.
Can’t we just say “Try it?” It seems we should be able to say something like this:
Our team costs ten thousand dollars a week. Let’s start working on the most important, most risky, most informative parts of your product, and build up a sense of how hard it is and how long things take. You’ll see us building what you ask for. You’ll see it working. You’ll be able to decide whether to keep investing or to stop.”
It seems to make sense to say that — unless it’s your personal Ten Large rolling out every week. Then you have more questions. “Am I going to know within a week? Will I have to wait a month to know? Will I know by June? Holy cow, guys, there’s over a hundred grand between now and June. That’s big money! Cut me a break here, will you? How much is this thing going to cost?
We need to step up to this kind of challenge. We need to give as solid an answer as we can, even when we don’t have an absolutely concrete answer.
We’re standing in sand. How do we get concrete?
The question is, “How long will it take, and how much money, to get to a product people might buy?”
We’d like to say, “Well, start spending money with us, and if you get bored, stop,” but that’s not helpful. Can we turn this idea around a bit? Let’s begin with some very broad estimation just to get a sense of where we are.
We probably have some sense of whether it’ll take a week, a month, or six months to make something people can use. If so, let’s talk about it. Our investor has some amount in mind that he can spend. Let’s find out how much that is, or get a sense of it in conversation.
Suppose we kind of think that maybe we could get a rudimentary but usable product in six months, and the investor is willing to invest up to about a million dollars to get to positive cash flow. A million dollars is about a year of our time, and if it took us nine months rather than six, we’d need to bring in a million dollars in three months to get positive. Let’s talk about that. How rapidly does he expect revenue to grow with a first release? Suppose he thinks revenue will grow that rapidly so that if we do take nine months, we’ll be OK. That sounds risky to me, but it sounds like this project is at least worth exploring.
Remember how Agile works. We want to get our business people fully engaged in making live decisions about spending money based on what they see. So let’s get to thinking about feature selection, not just dates.
Estimation You Can Live With
OK, intuitively, we think we could get an initial release in six months, and you think if it even took nine months, the project would be viable. That looks good, but it still seems risky. We’d hate to see you spend that much and get nothing, and these are still wild guesses.
So let’s do this: Let’s take two weeks to produce a more concrete answer. Two weeks will cost you only $20,000, which is a lot less than a million. In that two weeks, you’ll work actively with us on the most important aspects of the product. Some will be things like look and feel, and some will be driven by other business or technical risks that we can foresee now. At the end of the two weeks, we decide whether to go forward. Our job is to show you, concretely, what we can build that convinces us all whether we should go ahead. Your job is to guide us by describing what you want and to work with us to identify risks and concerns.
If after two weeks, things look bad, we’ll know it and we’ll recommend that you stop. If things look good so far, we’ll decide together what the next major decision point will be. It might be a month out, or three months out. Frankly, we’re perfectly happy to do this every two weeks.
Yes, that’s right. Every two weeks we’ll talk with you about what to do next. We’ll do it and show it to you. If it’s good enough, we’ll continue. If not, we’ll stop.
That way, your risk is never more than another $20,000. You can decide whether to use that money to get more information, to redirect our efforts, or that it would better be spent in some other way.
Can we work like that?
If he says yes, let’s get started. If he says no, we need to figure out why, and whether this project is for us.
This is definitely estimation, and it’s a kind of estimation that we can do in concert with our business-side people rather than in conflict with them.
We estimate, with some certainty, that we can produce useful information in two weeks. If we think we can’t do it, we’d better suggest four or six or eight weeks.
Furthermore, we estimate that a viable product is probably possible in six to nine months. If we have no idea, or worse yet, we sincerely doubt it, then we cannot legitimately invite him to find out — at least not in the terms above. We owe it to him to say, “Frankly, this sounds like a two- or three-year project to us. We could be wrong: here’s what we’d have to learn to find out. Here’s what it would cost to find out, and our best guess right now is that the answer would be bad news.”
Sometimes we have a good idea of what we can do. Sometimes we know less. Either way by estimating what we know, we can frame experiments — investments — that improve our ability to decide what to do.
When it comes to spending other people’s money, I think we owe it to them to do that.
This is Estimation, Not Negotiation!
An estimate doesn’t have to be: “This project will be done on Tuesday, May 14th, at 2:35 p.m.” An estimate is supposed to express a range of possibilities. It could be perfectly okay to say something like, “We see no way to get this done in April. If everything went perfectly, we might be ready in mid-May. Based on our experience, we’re thinking June, maybe July.”
Whether we say that or not, odds are we know it or suspect it. And if that’s what we think, the project needs that information. However, if we put it that way, we’re going to get pushback. Someone is going to challenge us to prove beyond a shadow of a doubt that we can’t get done by April 30th, and if somehow we do that, they’ll say, “What about May 5th?” Bah, we hate when they do that. So how can we give the project the information it needs without playing this game?
Move from Date Estimation to Velocity Estimation
Let’s use the Five Card method from the previous article, not to estimate a date, but to break down the whole project into sub-stories small enough that our team can do, say, five of them in two weeks. Of course, we might only get three done. We might get six but we doubt it. So we come back with an estimate in terms of velocity:
We have broken down this project into small stories. We believe that on stories of this size, we’ll be able to do between three and five every two weeks. And at the end of each two week period, everything we’ve done will be visible, tested, integrated, and working. So whether it’s five or six or two per week, we’ll know, and you’ll know.
If you can remove or defer some of these stories and still have a viable product, you will bring the product in sooner. If you can simplify them, you can bring the product in sooner. If you add stories, or make things more difficult, you’ll push the product out.
It’s up to you. We can split things down to this size, we can guess our speed now at about three items per two weeks, and you’ll know every two weeks what’s really happening. You can use that information to decide what to do, and what to defer, so that you can get the best possible product for any date you pick.
Help Management Learn to Use Velocity
Now management may still want to negotiate this with us. If they add up the figures to get dates, and then start trying to adjust the dates, don’t negotiate: turn the conversation back to the rate of progress. “The date is up to you. If you choose more to do, it will be later. If you choose less, it will be sooner. We think we’ll do between three and five items every two weeks. We could be wrong: if we are, you’ll know as soon as we will.”
Someone might try to push the velocity. “Can’t you do seven every two weeks? That way you could get done on time.”
Remember how Agile works: developers work at an optimal sustainable pace; developers aren’t in charge of dates; the business side owns the dates, and owns the responsibility to meet the dates.
The developers’ job is to deliver at the best possible pace:
We think we can do three to five. If we happen to go faster, you’ll know. If we go slower, you’ll know that too. When we get done depends on that, but it depends even more on how much, or how little, you give us to do. What’s put in by the date is up to you. You should plan on three to five items per two weeks, and keep an eye on what actually happens.
They may push more than once for faster. But we have the facts at our disposal, such as they are.
Based on our experience, we’ll get between three and five done. It would be unwise to assume anything higher than that, because it’s not likely to happen. If something happens that lets us go faster, we’ll all know it and you can select more items into the release.
All this, of course, is more about communication, not much about estimation, and emphatically not about negotiation. The estimation was trivial: we did it way up there when we split the cards. Now we’re communicating what we know, what we guess, what we believe.
We give management our best estimates of our production rate, and we stick to them, improving them only as we learn. We continue to make it clear that the business people control the date by choosing how much work they ask for.
Optimizing the Product
Whether you’re all for estimates or not, you should know that Agile is about building the product incrementally, in good condition from day one to the very end, and getting value from it as soon as possible. This requires great technical skill, but technical skill is not enough. The people paying for our work need to allocate their resources. They need to manage their cash, and they need to coordinate our work with activities and stakeholders outside our team.
To do this effectively, they need information on how long things take, which translates into how much things cost, and into how long it takes before we start getting our money back. They need information that we have to do this job well. The fact that our information is vague, flawed, and uncertain is not a reason to hold it back. We need to get the information into their hands and do it in a way that gives them the best chance of using it well.
Refusing to estimate is not the right way to do that. Estimation is necessary, and effective communication of estimation is necessary as well.
The best way to do things is to develop small features at the best possible sustainable pace and to use the information generated to help the business people decide what to ask for, and what to defer. To do this, we must do estimation. We slice stories to an estimated small size, and we estimate our rate of delivering them. This kind of estimation isn’t bad at all: it’s incredibly useful and it’s what the best agile teams do.
You May Also Be Interested In …
About the Author
Ron Jeffries has been developing software longer than most people have been alive. He holds advanced degrees in mathematics and computer science, both earned before negative integers had been invented. His teams have built operating systems, compilers, relational database systems, and a large range of applications. Ron’s software products have produced revenue of over half a billion dollars, and he wonders why he didn’t get any of it.
Pick up his book, The Nature of Software Development, from The Pragmatic Bookshelf:
