Why I Stopped Using Planning Poker

The simple alternative to Planning Poker that deals with relative estimations in a proper way

Estimations is a hot topic in software development. There are articles written, talks given, and books published. When to estimate, how to estimate, and whether to estimate in the first place — all these are not easy questions but I’m going to let you explore them on your own. Let’s assume though, you’ve weighed all the pros and cons and decided to estimate in story points. Chance are, you are looking at Planning Poker. It is probably the most popular estimation technique in the Agile community. I had played numerous sessions with my teams before it gradually started falling out of my favor. And I had reasons for that. But before we get into detail about why you shouldn’t use Planning Poker, let’s take a look at why you actually might want to use it.


What I Like About Planning Poker

Planning Poker was created by James Grenning in 2002 and seems to be a simplified version of another popular estimation method — Wideband Deplhi (although Grenning himself claims he would have forgotten Wideband Delphi by that time [1]). There were two problems he was trying to solve. First, estimates were taking too much time. Second, existing techniques failed to involve the entire team [2]. Planning Poker wasn’t very successful in solving the first problem, as you will see later, but it did well in solving the second one.

Incentivizing Discussion

One of the key elements of Planning Poker is conversation. Participants are expected to discuss the items until a consensus is reached in the estimate. A session facilitator usually makes sure the discussions happen if the group members turn out to be too inert to do this on their own. Sometimes it leads to valuable discoveries.

Protecting From External Influence

It seems to be a side effect of Planning Poker but it’s exactly what Wideband Deplhi was aiming at. There is some scientific evidence demonstrating that in their estimates people tend to gravitate towards the number they’ve been exposed to before the estimation started. This happens even if the number is totally random or incongruous. This effect is a cognitive bias and is called anchoring [3]. So, whenever you involve more people in an estimation process, you don’t want them to know each other’s estimates lest they are not anchored. Another reason for anonymous estimation is that participants tend go along with the most authoritative opinion in the room. You want to avoid that too. Even simpler than that, people may be just too lazy to make up their own opinion. In any case, forcing each participants estimate on their own effectively removes the effect.

What I Don’t Like About Planning Poker

At the same time there are a lot of things that don’t work in Planning Poker as well as you would expect them to.

A typical Planning Poker session. Sort of…

Faster, Still not Fast Enough

While Planning Poker seemingly has achieved some success in speeding up the estimation process, it could do much better in this field. During the Poker sessions a lot of time is still wasted on figuring out how exactly to play the game, especially in inexperienced groups. Turns out, the process is not entirely intuitive and people need to be thoroughly instructed before they fully understand the rules and the dynamics of the game. What’s worse, it doesn’t seem to improve a lot over time. After a team gets the hang of the rules numerous discussion keep popping up about how exactly the rules should be interpreted. Should we go for another round? Why don’t we just take the bigger number? No, you can’t just average out, we need to come to a consensus… And it goes on, and on, and on.

In defense of Planning Poker, Grenning in his original paper suggested not to spend too much time on trying to reach consensus. If you can’t reach it quickly, don’t sweat it. [4] So, with the right approach, the issue can and should be alleviated.

Not Truly Relative

But Planning Poker has a much more serious flaw: it actually makes it harder to use story points properly. In Planning Poker items are estimated one by one and a group is supposed to compare each item to a baseline item (typically a ‘1’). It’s exactly where lies the main problem. In their heads participants have to figure out how many times bigger the item is and it looks like human brain is pretty bad with this kind of operation. It tries to simplify the task by creating an image of each size on the scale and unconsciously translates it to time intervals (e.g. 2–3 days for size ‘2’). While the entire idea of relative estimations is to compare items against each other, people start comparing them to fixed time intervals. This is not much different from estimating in hours or days. After all, hours and days are fixed time intervals. Planning Poker, designed as a relative estimation technique, turns into absolute estimations without you noticing. Have you ever seen during a poker session somebody saying ‘it feels more like a 3 to me’? This is a dead giveaway the effect is taking place.

There‘s a way to get around this problem by triangulating with two or more values used as baselines [5]. It helps, but unlikely removes the problem entirely. There is another method that essentially does the same thing, but does it naturally and is faster and easier to understand.

Magic Estimation as a Fast and Effective Alternative

So, is there really a better alternative to Planning Poker? I believe there is. It’s a family of estimation techniques that share some common characteristics:

  • they are visual;
  • the estimation is done by moving cards around on a white board;
  • the goal is to align the cards in groups left to right by size from small to large.
An example of an online Magic Estimation board from Kiryl’s Facilitation Toolkit.

You may have heard about them under various names: Silent Grouping, Magic Estimation, Affinity Estimation, Swimlanes Sizing. I will call them Magic Estimation (indeed, it feels like magic the first time you experience it). The differences in these methods are minor and which one exactly to use is a matter of taste.

So, what Magic Estimation can offer that Planning Poker can’t?

Simple

The rules of the technique are extremely simple. You basically only need to draw the estimation board. Once people see the board, it takes a couple of minutes to explain the gist of the game.

A typical board setup of Magic Estimation. Size values are optional, the number of columns may vary.

Lightning Fast (With a Caveat)

In all versions of this technique communication is limited in some way to speed up the exercise. But even in fully unconstrained versions conversations are triggered mostly when there’s a disagreement between players. Procedural overhead is very little and no discussion occurs if it’s not needed.

Gains in speed come with a caveat, though. You can’t expect participant to estimate something they have no clue about, which means they still have to spend time discussing the items they estimate. It would be a lie to say that you can shrink a one-hour Planning Poker session down to 5 minutes, because in Planning Poker discussions are an integral part of the game and in Magic Estimation you need to hold these discussions beforehand as a part of preparation. However, there’s still much less overhead since the conversation format is not mandated. I’ve seen real teams doing two-hour sessions to discuss three months worth of work followed by a 5-minutes estimation exercise. Imagine estimating several dozen items using Planning Poker.

One may argue that while Magic Estimation saves you a lot of time, you might be losing in accuracy as opposed to Planning Poker. It’s a valid concern but here’s the paradox. Conventional wisdom has it that estimating large backlogs with Planning Poker is slow as hell and thus impractical, so nobody does this anyway. Surprisingly, estimating iteration backlogs with Planning Poker is no more practical. On the scale of two weeks the error margin is a few of user stories at the very worst. When so little is at stake, there’s no rational reason for perfect estimation accuracy. Instead, it’s better to turn your focus and energy to breaking work down into smaller shippable pieces and making the team deliver truly working software at the end of each sprint — that is, actively learning and reducing risks.

Prepackaged with Relative Sizing

The part that I love about Magic Estimation the most is its visual nature. It makes it almost impossible to avoid estimating relative. You are literally forced to compare items against each other by having to place them on the board either left or right. That’s the best way I’ve found so far to quickly make people understand relative estimations.

Magic Estimation forces you to think relative.

There’s a a reason why I think it works. It’s typical to use the (slightly modified) Fibonacci sequence in relative sizing. Human brain is much better suited to comparing on a logarithmic scale [6] and the Fibonacci sequence is an exponential function that makes one (strictly speaking, it becomes one if interpolated to a continuous function) [7][8]. Magic Estimation is in fact a heuristic that leverages this fact. The idea is simple. If one can spot a difference in size between two adjacent items, it is likely that those items are approximately in relation to each other as the corresponding Fibonacci sequence numbers. Because if they weren’t, the difference wouldn’t be noticed and the items would end up being the same size.

Conclusion

Planning Poker is a technique that still has its use and is preferred by many. In my own practice I found something else that provides me with more value and it is Magic Estimation. Here’s a quick comparison between the two.

What Planning Poker is good at:

  • facilitating deeper discussion and risk identification.

Where Magic Estimation trumps Planning Poker:

  • saving time,
  • producing good enough estimates at much lower cost,
  • promoting the right understanding of relative sizing.

The only time when I prefer Planning Poker over Magic Estimation is in immature teams that have a hard time starting a conversation. Planning Poker forces people to speak, so if you think they have a hard time starting a discussion, it may be a viable option for you.

About the Author

Kiryl Baranoshnik is an experienced Agile and Lean practitioner. He works as an Agile Coach for a large international software vendor and shares his knowledge publicly as an Agile trainer. If you want to learn more about Kiryl and the trainings he offers, check out his website and the schedule of the upcoming trainings. If you want to get the latest updates on Agile, Lean, and related topics, follow Kiryl on LinkedIn and Facebook.


References

[1] Grenning, J. (2002). Planning Poker — The Original Paper. Retrieved from https://wingman-sw.com/articles/planning-poker

[2] Grenning, J. (2002). Planning Poker or How to avoid analysis paralysis while release planning. Retrieved from https://wingman-sw.com/papers/PlanningPoker-v1.1.pdf, p. 1.

[3] Thaler, R. and Sunstein, C. (2009). Nudge: Improving Decisions About Health, Wealth, and Happiness, Kindle Edition. New York, New York: Penguin Publishing Group, p. 23.

[4] Grenning, J. (2002). Planning Poker or How to avoid analysis paralysis while release planning. Retrieved from https://wingman-sw.com/papers/PlanningPoker-v1.1.pdf, p. 1.

[5] Cohn, M. (2016). The Best Way to Establish a Baseline When Playing Planning Poker. Retrieved from https://www.mountaingoatsoftware.com/blog/the-best-way-to-establish-a-baseline-when-playing-planning-poker

[6] Vsauce (2018). 1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35… Retrieved from https://www.youtube.com/watch?v=Pxb5lSPLy9c

[7] If the Fibonacci sequence is plotted as a curve — is it an exponential curve? (2017). Retrieved from https://www.freemathhelp.com/forum/threads/107174-If-the-Fibonacci-sequence-is-plotted-as-a-curve-is-it-an-exponential-curve. Accessed on: August 9, 2018.

[8] Fibonacci number. Retrieved from https://en.wikipedia.org/wiki/Fibonacci_number#Relation_to_the_golden_ratio. Accessed on: August 9, 2018.