Software Estimation is like Short Selling a Stock

Lately I’ve been working on a large software project along with ten other engineers. We’ve been given a deadline for this project. Furthermore, we’ve been told that some of our reputation as a group is on the line in terms of hitting that deadline. Our estimates have consistently come in under that deadline, but I can’t help but feel that there is some pain on the horizon.

I frequently discuss the progress and future of the large software project with my manager. He has remained consistently optimistic, whereas I can’t help but feel consistently paranoid. “We’re screwed!” I’ll exclaim. “Why?” he’ll ask. I typically can’t answer that. For the most part I’ve had trouble figuring out why my instincts are telling me there is going to be a problem, but it consistently comes down to the estimates versus the deadline.

Who takes the on risk for a software estimate, and what are the limits of that risk? I work at Bloomberg L.P. and finance terminology surrounds us. The best analogy I can think of is that when you estimate software, and your reputation is tied to that estimate, it’s like you are short selling a stock.

When you buy a stock, your liability is the value when you purchase it. If you pay ten dollars for it, but it tanks, then you have to sell it for one dollar, you’ve lost nine dollars. You are never going to lose more than ten dollars in that situation.

But when you short sell a stock, you are selling first, and promising to buy later. You must purchase that stock at a later time to complete the short sell contract. Therefore if you sell the stock for ten first, but then buy it back when it‘s one dollar, you’ve just made a nice nine dollar profit. But if you sell the stock for ten, and the value goes to one thousand, you have to eventually have to complete the contract, and lose nine hundred ninety dollars.

So when you estimate a task at ten days, if it takes two days, you’ve gained eight days. That’s your “profit”. If you estimate a task at ten days, and it takes six weeks, you lose four weeks! This is one of the many reasons why software is always late. The risk is always bigger than the reward when you are estimating, and the liability for being wrong is effectively infinite.

Why Does This Happen?

There are reasons why software engineers chronically underestimate, which I can get into in another post, but right now lets take a look at some reasons why a single estimate can be very very wrong.

Unfamiliar Platform

Engineers need to grow and learn new technologies. New platforms come along that are faster, more effective, and solve more of the problems that we commonly face. Moving to an unfamiliar platform should be something that your engineering team does every once in a while. If you aren’t then your product and your team might be stagnating. Doing this has risks though. Unfamiliarity means that you can encounter problems you have no experience solving, and you need to build new experience. This is valuable in the long term, but if it is a sticking issue it can easily double or triple the amount of time you expected to perform a task.

New (Unstable) Platform

It’s not enough that you move to unfamiliar places, you should also try to keep up with the forefront of technological progress. This can mean using a platform that is still developing. Invariably you can encounter bugs or missing features that your work depends on. Suddenly you need to spend time either contributing to the project if it’s open source (it should be) or developing a work around if it’s not (you should avoid this if you can). Contributing to an open source project can be a valuable learning experience! But chances are you didn’t expect it in your estimate.

New Team Members

If you are working on a large software project at a large company that has a deadline, then it’s likely that someone will say “Let’s just throw more people at it.” And that brings me to one of my favorite quotes from Fred Brooks:

nine women can’t make a baby in one month

It’s not perfectly applicable to this situation, but I love it so much I needed to put it in.

There are multiple risks with bringing new people into a project. One of the biggest risks in terms of individual tasks is that in their first month or two, they are still training into the conventions of the group. This can dramatically increase code review times for both the reviewer and the reviewee, as they are asked to start doing their work “our way.”

Are there ways to deal with these?

Yes, here’s a few ideas:

Unfamiliar Platform? Read the docs/source

One of the biggest time holes a good developer can find themselves in is in that mode where you think “Let me just try this and maybe it will work”. This is known as programming by coincidence. Instead you should think “Let me reread the documentation and source code around this issue”

I’ve found that I rarely understand things 100% on the first read. I get an idea, and a sense of what it’s like, but that doesn’t mean I understand every detail. Always go back to the docs first. If you have the source, then read that too.

New Platform? Better Be Open Source

If you can’t look at the source, and it’s potentially unstable, then you probably shouldn’t be using it. End of story. If you are being forced to use it by someone else, give the caveat that dealing with issues in closed source libraries has the potential to be extremely expensive.

New Team Members? Don’t Lower Your Standards

Every time I’ve been pressured into a hire, and I thought to myself “Eh they’ll probably be fine” I’ve been burned. You should be excited to work with every new hire that comes onto your team. If they are temporary, then they better be ready to provide merge-able code on day one, or you are wasting your time.

But what about the meta problem?

In spite of our attempts to fix these smaller issues that bigger issue still remains. We as the estimators are holding the short sale, and the risk can still be huge under the wrong conditions. I might research to find good solutions to that, but does anyone else have suggestions?