#noestimates — What Are We Estimating Anyway?

A debate is raging right now in the twitterverse.

The #noestimates debate revolves around our ability to predict how much software will be output from your team. Jeff Patton has talked at length about how worrying about output is missing the point. We should really be concerned with solving the user’s real problem, and creating some outcome. Maximize outcome, minimize output.

If output != outcome, then measuring storypoints isn’t focusing on what we’re really looking for. We could be delivering consistently for several months and be “on schedule”, but still deliver something that is totally useless.

How do you measure outcome? Possible ideas I have, but I think they’re all flawed:

  • Number of Customer Complaints — Really a measure of how angry off your clients are
  • Customer survey (aka Customer Delight) — Hard to get good numbers. Response rate is very low. Mostly people who aren’t happy.
  • Usage statistics (Are they actually using it) — People can be using your software and still not be happy with it

If you could create 2 parallel universes, one where you used estimates and one where you didn’t, and both produced the exact same burn-up chart, there is no guarantee that the pieces of software that came out the other end would be at all the same.

I have a hypothesis: Estimates are a self-fulfilling prophecy. They’re a bit like a gas. The work expands to fill the space it occupies. This is why even when you triple or quintuple your estimates, you still go over. I think this also causes people to lose sight of the real decisions they need to make around delivering value to the customer, and they instead focus on delivering on the estimates.

I present to you the Observer Effect applied to Software Development

The act of estimating software will inherently change the outcome

Estimates may also trigger poor behaviors. What if we delivered on our estimates but acquired significant technical debt in the process? Will this have a long term impact on the viability of the software? Does this even matter if we delivered something of value in the short term?

The real question that #noestimates needs to answer is: Does not estimating increase the desired outcome even if we lose the ability to make certain decisions.

I’m not sure this question is answerable, but I think it’s where we need to start.