On project estimation or how long does it take to solve a sudoku

I enjoy doing sudoku puzzles, I sometimes do 2–3 puzzles before I go to sleep. It relaxes me. I’m an average player, I usually solve hard puzzles (evil setting in Websudoku) in under 10 minutes, without pencil markers as I like to exercise my memory. My record time is 4:36 minutes, but I sometimes struggle very much and I’ve had puzzles that took 40 minutes to solve. My average is around 12 minutes.

Some puzzles are harder to figure out than others even with the same difficulty level, but my time also depends on how I feel at that particular moment, if I’m tired, etc. Sometimes all the pieces seem to come easily one after the other, and sometimes I just can’t figure it out.

I’m probably not the first person to say that doing a sudoku can be compared to coding, in that it’s not typing the code or numbers that is difficult, but figuring out what to type. I also find that ballparking a software project is very much like estimating the time it will take to solve a puzzle.

If you asked me to estimate how long it takes me to solve a hard sudoku, I’d say about 10 minutes, even if this is lower than my average. Most of the time, I solve in under 10. The particular puzzle you ask me to solve might be an outlier, in one direction or the other, I wouldn’t know. Despite my experience, I can’t evaluate by looking at a puzzle for a few seconds how long it will take, but I can safely say that it will lie somewhere below the average (outliers in the upper direction are much farther away, and skew the average up).

Similarly, I have enough experience to gauge the scope of a project just by having general information about it, but just like a sudoku, I won’t know precisely how long it will take until I am very well into it. Even if I know all the rules and I’ve done it a thousand times.

Why then when estimating a project is it tempting to go with the lower figure instead of the average? Perhaps part of the answer lies in the fact that we don’t see project estimation the same as puzzle estimation. Project estimation is not based on stats, we think, it is solely based on our talent and knowledge.

A strong incentive is that the lower figure pleases the client because it is what they hope, so your chances are better to win the bid. We wish to be better than our competitors, to do a better job in less time. But that wishful thinking should not be the largest influence in our estimation of the project. It should push us higher of course, but not passed the breaking point.

There’s also always the possibility that all will go well in the project, and that’s often what we gamble on. After all, we can’t say that it’s impossible to deliver under x hours, if it has already been done for a project with a similar scope. As for the similar projects that took longer (sometimes waaaay longer), we convince ourselves that we’ve learned from our errors and won’t repeat them, or that the context was different.

We tend to assume that it is only up to us as developers to keep the project on track, and that we always have the power to control how it goes. If we can’t do it in the lower-end budget, it’s probably because we’done mistakes, we don’t have enough experience, etc. We figure that it’s our responsability and the client does not have to pay for that.

We have to take into account though that no, it is not entirely up to us. Technology is hard, and we cannot take responsability for all the difficulties we will encounter along the way. An incompatible library, a misunderstanding, a server outage, whatever. Complications that, by the way, our competitors would stumble on as well.

Just like sudoku puzzles, some projects will have more of these problems than others, and will be inherently harder even if they look the same at first glance. Some problems will be our fault, of course, but most are out of our control and are just normal things that happen in a normal project.

Any project will probably have an average share of difficulties, and we have to consider that fact when estimating to get a realistic figure. We can’t assume that this particular project is an outlier in the right direction. It’s a difficult position to be in though, because if we estimate with the mean instead of the lower figure, we have to justify our pessimism… as that is how it is perceived.

Why won’t this project be going perfectly? The client always assures us that he’s not difficult to work with, that we will have everything we need in due time, etc. And we are good at our job! So we seldom have arguments. Citing the Mythical man month, the planning fallacy or Hofstadter’s law won’t do. We can’t say what part will cause the project to lag, yet we are asked to justify our estimation with hard facts. Neither statistics nor experience are enough to convince the optimists.

But in the end, chances are that this project, this puzzle, will regress to the mean. I can’t tell you the exact reasons at this point, but it will, trust me. Math is on my side. I just can’t convince anybody.