Estimation and unknowns for Product Managers and Engineers
Figuring out how long things take isn’t easy. In the worst circumstances it can lead to conflict and bad feeling between people.
Let’s have a look at an anti-pattern…
The Product Manager
“What in the name of all that is spiced festival wine do you mean you have no idea how long this will take? You’re a technical genius…”
Some Product Manager, recently.
The ancient problem. A question people have been asking since the dawn of the cybernet. Not an entirely unreasonable one either.
“How long will this take”
For a Product Manager, it seems like those nerds should be able to answer such a question.
They’ve done this before. That’s why they have a job and well they should be glad of it. Amazing technical people are ten-a-penny these days. I’ve had enough of these experts. I’ll browbeat estimates out of them if it kills me.
In this scenario we have a frustrated Product Manager who feels like getting an answer is like pulling teeth.
The Engineer
“By the ravens of the dale, why must we participate in this infernal circus of lies once more”
A Software Engineer, just now.
“How long will this take”
For an Engineer sometimes the initial reaction to this can be to categorise it as a totally ludicrous question.
We’ve never done this before, they keep asking the same questions over and over, and dear lord of mercy, how do they not get this is actually hard? I’ll just chuck some random crap at them so they go away. Again.
In this scenario, we have a embattled engineer, getting dragged away from what is thought to be useful work, to do ‘management’ nonsense.
Clearly the above are ridiculous distillations of reality, but useful to highlight the divergence between two people, who should be on the same team, but with very different ideas on what is actually a perfectly reasonable and answerable question.
“How long will this take”
Now I said perfectly reasonable and answerable, but that demands reasonable behaviour from everyone involved in trying to answer it.
Framing the Problem
Here is a suggestion of how to go about it. I’ll give you the perspective of the Product Manager (which I am) and the Engineer (which I am).
To start with getting some common ground to work from is really useful, and a gentle reminder here: you are making something as a team.
The Product Manager
Context is always good here. Make sure you share with people helping you do your estimating what the point of this exercise is. Maybe you need some rough costs, maybe you need to figure out how long this piece of string might be. Share that. It really helps.
The other context that is useful is about the work you are estimating. What is it, and why are we considering this? If you are doing good things, you will be talking about user research that led to this exercise. If you are doing great things this won’t be a surprise.
Be prepared for questions. The more you can answer the better answers you will get.
The Engineer
Bring your experience with you.
More importantly bring your patience and your listening ears. Ask for context, ask dumb questions about why, as an engineer I like to know I’m not wasting my time on some management boondoggle.
If you are working with a good Product Person, those ‘dumb why’ questions are gold dust. You know the value of rubber ducking a problem, sometimes being the duck is massively helpful.
Both
Establish the ground rules. I suggest any and all estimates include things to avoid epic failures, the dreaded non functional failure. Be clear about what that actually means. Have a ‘definition of done’.
Accept you do not, cannot and never will know everything. This is reality.
Anti-Patterns
- Your answer to the question ‘Why are we doing this’ is:
- The last estimates were unacceptable
- The last estimates are wrong
- Management didn’t like the estimates
- We need to make the estimates smaller
Woah there Nelly. You are on the express train to Crapsville and your cargo is a whole pile o’ horse manure.
Sounds to me like maybe there are problems somewhere else in this town that may need to be fixed. I’ll cover these during the process outlined below.
2. You have no agreement on what ‘Done’ is. Bad times. You need to sort this out. You are just asking for trouble if you don’t have an agreement on this. People will be driving all sorts of tragic buses through the holes in your estimations, and ultimately your product’s success is at stake.
3. You do not accept unknowns exist. Please check yourself before you wreck yourself.
Map the things
This is the fun bit.
Start with two axes….take ten paces. Turn and throw.
Actually wait, that’s a different thing. I mean axes in terms of a graph…like so (use a wall / whiteboard / large piece of paper / window / whatever):
On the Vertical (arrow going up) Axis (I always confuse these) you are mapping things you know a lot about up to things you know nothing about.
On the Horizontal (arrow going right) Axis you are mapping the things you think are super dangerous, or will take a ton of effort. Or Both.
Once you have this done, start putting things on there. You can do this at a very high level to start off a new thing, or in more detail where you identify crazy unknowns (more on that soon).
Here is a made up example:
This is where context and experience matter. The point here is not to get into very long debates about the why’s and why not’s of the thing. Get on with getting on. Trust each other. If something is not clear, ask questions. If it remains unclear…amplify the unknown and risk/effort. This will help later.
DO NOT put numbers on ANYTHING. Seriously don’t. As soon as someone starts talking about days, or weeks or any measure of time, award yourself a mark and agree at the end of the session whoever has the most marks provides some sort of atonement (be it snacks, books, comics or otherwise).
We will get to the numbers game later.
Let’s look at some example classifications, and why they might end up where they do and what to do about it.
Well understood, minimum risk/effort
These are things we really understand, and have all the control in the world over. Changing a label on a screen is easy. Maybe adding a new mapping feature to something we already did some great work implementing iteratively is also nice and easy.
You know what? You can probably go ahead and do your nice small user story and get on with building it.
Anti-pattern
- Everything ends up in the bottom left of the graph. Wow. Either you are in the business of re-labelling forms or you are in some cloud cuckoo fantasy land. Or you are a team of rockstars from the Matrix. This is probably a sign of Optimism Bias. Maybe get a second opinion.
- You are not including everything you need to do to ‘go live’. I may be a cynic, but i’d prefer to include everything rather than being the eejit explaining to everyone that on reflection I forgot to include the testing needed to go live in my estimation session.
Tricky, risky, less well understood things
This is where you earn your stripes as a Product Manager. You learn about the things where you need to make some brave decisions. JFDI as they say in the states. Or perhaps not.
These are tricky bits. We think we know a reasonable amount about them, but actually, there could be some spiders in there.Perhaps we have less control over these things.
In my imagined examples, we know that our APIs were really good in Node.js and we wrote a lot of tests. We really don’t know lots about Go though. So this is dicey but not entirely unknown.
What about the new Authentication mechanism? We finally realised nobody understands the ExampleOrgSSO thing we deployed and want to simplify things a bit, go for a fairly solid thing built by someone else. There is a bunch of work in here, but we reckon we think it is worthwhile. Google looks good, and you, oh sage Engineer, have some experience with it. Still though. ExampleOrgSSO resulted in some weird decisions in our architecture. Dicey.
You can of course take a punt here, go for a high five Eiffel Tower and implement the things…but it’s probably best to Spike some work on this. Get some better answers.
(I’ll write about Spiking next and link to it, I promise, stay tuned!)
Anti-Pattern
Beware Optimism.
Your Engineer sense tingles. I am equal to this task. It is a mere walk in the park. NO! The Park is full of snakes. That fly around on bats. Be realistic.
Beware pushing for positive (smaller) estimation
Don’t be a buzz killington and start questioning everything to try to deflate the risk or unknown elements. You’re shooting yourself in the foot. Perhaps you are secretly a genius who has somehow managed to straddle the pillars of Product and Technology like some digital Colossus of Rhodes. Fair play.
Terrifying dangerous things
The realm of nightmare and confusion. Maybe you are more optimistic, crisitunity abounds.
Either way, these are things that it would be irresponsible for you to claim any certainty about. Lets consider my contrived examples.
Zero downtime deploys straight to production. In this imagined world this is halfway through a project and someone decided to try do this in the middle of everything else. This is going to be a massive undertaking. We have no idea how to do this. We can’t just do it. We might destroy everything. More than likely we’ll have to rewrite our application code, rewrite our deployment, probably do some infrastructure work, do a lot of failover testing. In reality this is new product territory.
What about upgrading ExampleOrgSSO, our third party tool. We don’t own the code, we have no idea what their definition of done is. Maybe it is really dodgy and they have a great marketing department. The last one was OK, but the one before that went on fire for a week. The release notes are super sketchy, and massively inconsistent. Lots of risk, lots of unknown. Are we cool taking the latest version? Are we at risk of the early adopters curse? We need to try this first. Spike o’clock. Capture what you do know, list the questions you need to resolve where you don’t. This is OK. Nobody knows everything.
I’ll write about Spikes in a follow up article.
Anti-Pattern
- Everything is in the top right. You really don’t know what’s going on. Super bad times. Do a lot more work on defining things. Or you know lots but know its a huge effort. Start breaking things apart into smaller Products. If you can’t, then you best get out the money wheelbarrows and flak jacket, may the force be with you. You do have the option of cancelling the thing. This is not a bad plan sometimes.
Size the things
Once you’ve done your assessment of how well you understand the problem, and what you believe the rough effort/risk is you can start shuffling stuff around a bit.
The point here is to sense check what you’ve done. On reflection now that you’ve talked through everything, is the effort/risk/certainty still about right, or should it maybe move around a bit.
Stick RELATIVE sizings on things. Stay away from numbers still. There are lots of good reasons to not go into numbers. You’ll get lost in detail and loose the value of what you get out of this.
In the above example I went for boats. Changing a label is rowing boat sized. That Zero downtime thing? That is a battleship. Maybe that crazy flying one S.H.I.E.L.D have.
Either way, doing relative sizing lets you be a bit more honest in discussions, and stick to the real thing you are doing when you ask for estimates, which is increasing certainty where you can, and reducing unknowns.
Anti-Pattern
- Demanding real numbers at this point. Avoid the temptation, you’re not getting value yet. Be patient.
Ring the Alarm
This is where this pays off loads. Those things in the upper right of the graph? There be dragons. This is where you get to test how honest an environment it is that you work in. You do not know enough to give numbers that can be turned into promises. It is wrong to do so. You need to do more investigative work and not promise delivery of it until you have done so.
The other things? Fill your boots. You should be in a great position now to actually stick some numbers on ‘em. Use ranges. It is sensible. Worst and best case planning and all that. Really, you can get properly scientific here, but sadly not everyone is a statistician or data analyst. Either way, you can, with some confidence make some promises here.
But that upper right hand side…if you must, use wide ranges, and low resolution timeframes: weeks, months.
Anti-Pattern
- Refusing to accept unknowns and demanding high resolution estimates. If this is your approach and you really want to push for answers on which to build your plans, castles made of sand fall in the sea, eventually. Instead, buy yourself more certainty by Spiking some work around this. Additionally, you are in a position where for whatever reason, honesty isn’t allowed. This is the ruin of many a working relationship and the doom of a product. Worse, it can lead to a product continuing a lot longer than it ever should, wasting everyones time, good-will and money.
One more thing
There you have it. Try it out, you might find it useful. I certainly have.
One last thing. Don’t feel like you need to limit this to two people. The more the merrier, especially in getting a true representation of where things should sit on the graph…especially don’t make the mistake of including only those from the ‘tower’ (i.e. all ideas and zero implementation types) because you will get massively skewed results and ultimately bad estimates.
Have fun!
Thanks to: Will Hamill Peter Campbell Jennifer Hanratty for the proof reads and suggestions for making this better!
