Estimating Like A Capitalist
Let’s play with some ideas about crazy incentives and penalties to see if it changes how we think about estimating.
Here are the rules of my thought experiment:
- Charge developers $20/hour every hour they are short on an estimate.
- Charge developers $40/hour every hour they are long on an estimate.
- Pay developers $10,000/quarter — $1000 for every 5% they are off on estimates for the entire project.
- To change an activity that has begun, the company will need to pay the developers involved $20/hour for every hour of work completed against the activity at the time of the changed request. Introducing new activities is free, but requires a project reassessment (as it always does).
On most projects, time and effort setup an economy with aspects that are fairly easy to compare to capitalism. Capitalism works best when the payer is charged the full cost and the seller is paid a fair price. We have all seen examples of capitalism going off the rails when costs are hidden or externalized.
One of the strangest part of any project is the confusion that results from estimating the time for activities. Nobody seems to value accurate estimates until a project gets in trouble and then, it’s too late. Let’s look at what might happen if we introduce some incentives and penalties before the project is in trouble.
In some respect, the estimators and the product owners are both “playing” with the house’s money. When an estimate is inaccurate, it’s unusual that either estimators or product owners will receive changes in compensation (though management will often get a bonus based on “success”). Sure, the product owners often care about “time to market” but it’s rare that this is tied to their compensation in any way that relates to the estimates.
One could argue that it’s in the best interest of a contractor to estimate in a way that gets them the work, regardless of the time to complete an activity, and certainly there are contractors that estimate this way (this is especially a problem with the modern RFP system). But long term, contractors that practice their craft this way tend to lose repeat business and have a tendency toward lower quality standards (largely due to focusing on the wrong thing).
No, it’s in everyone’s best interest to estimate accurately. So why are estimates such an area of contention on projects? There are many reasons, but let’s try a thought experiment for just one of those reasons — lack of feedback through penalties and incentives. We know from basic economics that penalties and incentives have strong effects, but that these effects diminish quickly over time. This makes them perfect for our thought experiment because we don’t need them long term.
Here’s the thought experiment. Let’s pretend we are a developer given one activity — some collection of tasks. Let’s say that unknown to us the activity can reasonably be accomplished by a qualified person in 2 weeks and that the risk is low. We would want our estimate to be 2 weeks plus or minus the smallest variation we can accomplish.
I can already see you thumbing through the tasks, adding up the hours for each and zeroing in on an total estimate for the activity. Good, now lets add that most capitalist of penalties and incentives — money. Let’s assume this is a 3 to 6 month project and that the numbers scale appropriately for longer projects. And let’s consider all the activities.
For every hour you come in under your estimate you have to pay the company $20 dollars. I’m assuming your rate is well north of this amount. For every hour over your estimate, you have to pay the company $40 dollars. At the end of the project, you get paid a $10,000 bonus minus 1,000 for every 5% off the actual effort is from your estimate over the entire project. If on average you estimated to within 10% you’ll get a $9,000 bonus (we round the penalty down). By making the bonus for the entire project, it’s hard to game the bonus during any one activity.
Further, any change to an activity by the product owner that comes after an estimate is established will pay the estimator $20 per hour for every hour already spent on the activity. That is to say the activity can be changed, but there’s a cost of $20 per hour for every hour completed on the activity when the change is requested and the estimator’s original estimate is thrown out and 5% is added in favor of the estimator on the bonus calculation to be used at the end of the project. If there are enough post-estimate changes the estimator will almost be assured the $10K bonus. And the estimator will be paid a $1,000 for every excess 5% credit they may have earned. Change crazy product owners will have to pay!
It’s natural to try and game systems like this. In fact, it’s guaranteed. I can imagine that in a system like this the developers would immediately begin to refuse to accept changes. Wouldn't you have to? Changes are often cited as the number one reason for estimates being off, especially “off-book” changes or side-loaded features.
I can imagine that with the $20 per hour penalty for changing an activity after it’s been estimated, the product owners will want to introduce needed changes via new activities rather than changing existing ones. Changes that must be made after the initial estimate will be very well thought out in order to justify the added expense.
I can imagine that developers would quit experimenting with the latest and greatest technology and use tried and true tools that had proven themselves in the past, after all — the second most common reason for estimates being off is surprises in the technology.
If new technology were being used, I can imagine that the developers would lobby for smaller and shorter activities that help them learn the technology without committing to a big and risky estimate.
I can imagine that the estimators would begin to lobby to have each activity estimated as close to time of development as possible. In other words, make the estimate at the time you know the most about the activity, the environment, and the tools. The estimators would also want to estimate one activity at a time, do the work, and then estimate the next activity. In this way, they could apply the learning from previous estimates to the next round or iteration.
I can imagine that the estimators would lobby for the units of estimation to be as large as possible — weeks or days, not hours and minutes. It’s easier to be accurate to the nearest week than to the nearest minute.
This system is a fantasy, a thought experiment. I offer it up to get both the product owners and the estimators to think about the systems in place today. Mostly I see product owners and managers putting pressure on developers for shorter estimates (really for no reason). I see developers trying to make management happy by agreeing to shorter estimates than they would otherwise be willing to commit. Who benefits from accurate estimates? No one. These pressures happen even on Agile teams.
There is currently a misguided belief among managers that estimates that are slightly too short represent a “stretch goal” and that this motivates developers. The idea that an estimate represents a goal is so wrong headed it needn't be discussed here.
On the other hand, developers are misguided to think that estimating short will gain them favor. How much favor will estimating 10% short gain you? Will 20% short gain you more or less favor? Eventually those same people you’re trying to curry favor from will be furious that your estimate was off by 50%. Even 10% or 20% grows tiresome over time (here comes the management “padding” of the schedule).
Yet most estimator’s initial estimates will be attacked and subtle (or not so subtle) pressure applied for shorter estimates. I've seen it maybe hundreds of times myself on dozens of different teams. The means of attack may vary, but the pattern of pressuring estimators for shorter estimates and then complaining about the inability to estimate accurately is repeated again and again. It’s almost as if the people requesting the estimate think they are requesting the duration of the work and not a guess of how long that will be. Strange.
This thought experiment makes it so that both sides pay for inaccurate estimates and both sides should have incentive to work together to limit post-estimate changes and reward accurate estimating. The estimator can earn a bonus with accurate estimates, and the product owner gets reliable estimates for forecasting, budgeting, and release planning.
No one will ever do this thought experiment for real. But never ever feel bad for a shitty estimate again, and don’t ever think an estimate has any reason to be accurate under the current scheme of pressure / wrong / pressure / wrong. Why should any different? There’s no reason. Whether the an estimate is short or long, today the estimator is just badgered to make it shorter. With the way we do things on most projects, why would it be any different.
This thought-plan also lacks sufficient bite to keep the product owner honest. It could be argued that a corporation has an interest in the estimates being accurate, but not the managers. There is no penalty for them if they pressure the estimator for shorter estimates and they get to report shorter times up the management chain. The system outlined in the thought experiment makes it expensive for the estimator to succumb to the pressure. For this and a thousand other reasons this system is not practical, it’s just a thought experiment. But it shines a light on the current system.
One last point. What about Agile? Well, I know of no mechanism within Agile that encourages more accurate estimates. Too few teams that say they are doing Agile use the estimates and velocity to predict and manage the project. This is largely because most Agile teams are not supported by Agile governance. I have yet to work on an Agile team without a timeline and a budget. That’s not Agile’s fault, but it is the norm.
I've been in dozens of retrospectives where the accuracy of the estimates was brought up as an issue. On occasion this even resulted in some ideas to try and make it better. Those projects always had something more important to focus on than the accuracy of the estimates and it fell through the cracks. How could it be any different? On an Agile team what penalty is paid for shitty estimates? Velocity stablizes just the same for purpetually shitty estimates as it does for highly accurate ones. What’s lost is the ability to forecast and working under a budget and timeline it’s always “burn at the maximum rate until we ship.” Agile loses.
If your only use for an estimate is to lie to upper management about timetables and to create false “stretch goals” (rather than real stretch goals), you’re about as far from the capitalist estimate as I can imagine.
Is there such a thing as the fascist estimating?