Dates for Software Development Are Absolutely Necessary and Completely Impossible
The Question, and the Bad Answer
The Question
“When’s it going to be ready?” Oh, it’s a hard one. In previous incarnations of my career I have been asked that by founders, CEOs, nervous leadership groups and a couple of the more illustrious VCs in Silicon Valley (John Doerr was particularly direct).
I figured out fairly quickly that there are no great answers, several pretty good answers, and one bad one.
The bad answer, I discovered recently to my amazement, is still around in the software business. I’ve seen impassioned arguments, in mid-2020, supporting it.
The bad answer is “I don’t know”.
I’ve had founders ask me “should my VP Engineering know when the software will be done? They say they don’t know”. I’ve had CTOs ask me why their CEO gets really kind of irritated when they respond to the Question with “well, it’s hard to tell — software is complicated and there’s this thing and that thing, and well, we’re designing for extensibility and…”.
It’s still happening. Hence this post.
The Problem With The Bad Answer
The real problem with the bad answer is that it’s more or less true.
We don’t know when a piece of software will be ready. We’re talking about a number, sometimes a large number, of very creative people making stuff up. They’re taking a (sometimes, usually) rather ill-defined Thing and building an approximation of it in language. They don’t know exactly what the Thing will look like until the Thing begins to show itself out of the work they are doing. Oh, and often many people, sometimes highly opinionated and important people, have new thoughts about the Thing as the Thing is being built.
But, on the other hand, unless we’re building software for fun, somebody at some point needs to sell it, and somebody needs to buy it. This is as true for software as it is for more or less any creative act: music, writing, painting, juggling. And in order to play the game of capitalism in which we are (mostly) all heavily engaged, we need to be able to predict when that very special moment is. Because that’s when the money comes in, to replace the money we spent building the Thing.
The capitalist system likes certainty. It likes “we beat the number by $0.01 quarter after quarter”. It likes to know that money is safely going to grow and fatten under your capable stewardship. It really doesn’t like “I have no idea, come back next quarter and we’ll tell you how it all panned out”.
So, bottom line (as we say): if you continue to say “I don’t know” in answer to the Question, it’s unlikely (not impossible, but unlikely) that you will have a long and storied career in software management.
By the way, a variation of the Bad Answer is something like “well, September 15th, but we have a very low confidence in that date”. Love that one.
Better Answers
Better answers are a date range and a percentage likelihood, as long as the date range is within the bounds of what will work for the business, and the percentage likelihood is high.
“Second half of 2021, um, probably”. Not so great. “Q1 2021, 90% likelihood”. A lot better. “By March 15 2021, 95% likelihood”. Very good.
Now. These answers are only good if, over time, you are often correct. Everybody knows predicting software is hard, things change, blah blah blah, so a miss or two and you’re forgiven. Wrong most of them time? Sorry, you lose the game, and this particular iteration of it will continue without you.
Which brings us to How to Be More Or Less Right
Doing the Impossible: How To Be More Or Less Right
So your job, is to deal with the impossibility of taking something inherently chaotic, unknown and unpredictable and making it happen inside a structure of time and budget.
And the first important step is to acknowledge what you’re doing: you’re guiding a soup of instincts, craft, talent, personality, insight and sheer grind. It’s not predictable, but you have to predict it. That’s why Engineering Management is a difficult, highly paid job. If it was easy, it wouldn’t pay so much. You’re leading people. That means connecting with them, understanding them, motivating them and providing them with good reasons as to why they are doing what they’re doing. All this is tricky and takes experience and a desire to find the best in people.
And then second, you lean on the basic principles of project management, which, over time, have remained remarkably constant:
- Break it into pieces. Keep the pieces small. See where you are after each piece. Adjust.
- Match the work to the people you have. Give architecture to the architecturally inclined. Give bits that have to go fast to the speed freaks. Seems obvious, but, hey, all of this is obvious at some level.
- Do the hard parts first. Oh, human nature: “that’s probably difficult, we’ll do it when we have some momentum”. No. You will keep putting it off until times are squeezed and pressure is high and then it’ll bite you. Do it now.
- Shine light on the unknowns, remorselessly. “We don’t know” is a red flag, a blaring alarm. The unknown is your black grail, your shadow. Find it, dispense with hit. “We’ll find that out later!”. How much later? How? etc.
That’s kind of it. You’ll find many blog posts on Jira, and Kanban Boards and a thousand and one iterations of the Gospel of Agile (which, I must add, in its pure form, I support — the more baroque later incarnations leave me cold). Sure, get into it. But the four points above will get you a long way.
Supplemental Questions Which Will Come Up
Are the engineers working hard enough? A: almost certainly yes. But you would know, you’re talking to them every day. Aren’t you?
Are you still certain of your date? A: percentage, and timeframe.
Can we speed up the project by adding people? A: No.
Can we speed up the project by buying something (a library, a company)? A: maybe, it depends.
We should rewrite the whole thing, it’s a mess. OK? A: ummm, be very, very careful.
I was talking to somebody at a Silicon Valley party on the weekend and they said we should do it this way. What do you think? A: be polite. It’s irritating, but listen, there’s a 10% chance it’ll be useful. (John Doerr told my CEO once we should rewrite everything in Java, which would have actually sunk the company, the same way it did Netscape. That was a hard one to be polite about).
Good Luck
I think the delicate act of balancing form and content, structure and chaos, creativity and business is endlessly fascinating, which is why I’m still at least peripherally engaged in it after three+ decades.
The fact that the mechanics of team software development remain mysterious and unpredictable after six decades of intense focus and growth is kind of wonderful. The more so because the entire world now rests on the results of this alchemic, half-understood process.
It’s an art, a craft, a magic trick, a merging of deep rationality and an emotional understanding of people. And it pays well. What could be better?
