What’s Wrong with Dot Voting Exercises

Or, what NOT TO DO (and what to watch out for) next time you decide to use dot voting

Stephen P. Anderson
Apr 23, 2019 · 13 min read
Source: Flickr

I used to love dot voting.

I don’t recall the first time I participated in a dot voting exercise, but I’m sure I felt a sense of wonder at how a skilled facilitator was able to take a jumble of recently minted ideas and get us all to coalesce around just a few of these…

Alas, I rarely feel that way anymore.

All but two of the dot voting exercises I’ve participated in over the last few years have left me feeling — more than any other emotion — frustrated.


To be clear about what I am critiquing, it’s the flavor of dot voting popularized within design thinking circles, where groups will:

  1. Generate a bunch of ideas —“One idea per sticky!”
  2. Cluster these ideas into groups or themes (sadly, this step is sometimes skipped).
  3. Dot-vote on these clusters/individual stickies. Everyone is typically given three sticky dots with which to vote (you’d think there’s something magical about this ‘three dots per person’ pattern, as groups rarely stray from this default).
A typical dot voting process, a la Design Thinking

There’s nothing wrong with this per se. As with most things, it’s in the details where things break down. Tools are tools. Some tools or more or less suited to a problem. A good tool can be poorly or expertly used. And we can always invent and modify the tools we use.

If I’m being honest, part of my frustration is that I see really great tools — such as dot voting — being wielded poorly, with lackluster results. This eventually leads to faulty conclusions about the value of such tools. This is true of tools as well as the entire practice of design. In the democratization of design methods and mindsets, I see a lot of adolescent fumbling with new things, by non-skilled practitioners . This is good and to be expected. But, this calls for those who are skilled to step into coaching roles, and model best practices. Or, as is the intention of this article, to be more explicit about things might have come naturally and never have been articulated. I have zero interest policing what people do or trying to tell people the “right” way to do something (we’re all learning and figuring things out!), but there are some things to share that might level-up usage of this tool for all of us.

Bottom line: Despite the negative nature of this title, look at this is as a provocation to make things better.


So, back to my frustrations. Why am I frustrated? My issues are mostly with the quality of objects being voted on, though there are also issues with the act of dot voting itself, including the human biases inherently built into this activity. All of these issues are solvable. Let’s look at these in turn…

Problems With ‘What’ is Being Voted On

I think of these as information architecture problems with Dot Voting, because, well… the ideas, concepts, proposals and other things generated on the fly often have little to no thought about how they relate to each other. Let me explain with some exaggerated examples. And while these are exaggerated and easy to laugh at, these are the very problems that I repeatedly see in everything from product brainstorms to team off-sites.

Bad when there isn’t parity between things being voted on

Sadly, I don’t see this kind of clarity very often. More often, I see something resembling this:

We get four kinds of cookies and the fruit tray to decide between. Because there are so many variations of one general idea, and one representative of another general direction, this creates a “split-vote problem” which leads to entirely different voting outcomes.

Bad when basic expectations are mixed in with differentiators

Pick three!

In software design, authentication isn’t very sexy and won’t differentiate your app. But, you probably need some kind of login, right—right!? So why is this “voted on” alongside differentiating or value-add features? They’re not in the same class. If you’re trying to decide where time and effort should should go, at the very least cull out the critical, non-negotiables, first.

Bad for pieces in a system

You can only vote for three ingredients!

Good luck making that chocolate chip cookie with only the three ingredients you just agreed to prioritize.

This one is harder to spot, but usually shows up across many of the ideas generated. A team might decide to focus on recruitment, when that should be considered as one of the six stages of the employee lifecycle (attraction, recruitment, onboarding, development, retention, and separation). Or, to improve team performance, a group may decide to focus solely tools and processes, when we also need to talk about things like the organizational structure and psychological safety.

Bad for pace layered priorities

Years ago, I started thinking about product and feature prioritization through the lens of pace layering. For the uninitiated, ‘pace layering’ is essentially a way to discuss different layers in a system, and how each layer changes at a different pace, from the fastest layers to the slowest layers in the system. Pace layering is often shown like this:

“Pace Layers” diagram from Stewart Brand. See: https://jods.mitpress.mit.edu/pub/issue3-brand

Pace layering is also explained using a house metaphor, where rearranging furniture (changing “Stuff”) is far easier than adding an extra bedroom (changing the “Structure”).

This metaphor is especially relevant for software design, where we have technology stacks and deep — and slow changing — layers of infrastructure sitting behind things that are far easier to change. Editing a misspelled word is far easier to change the the underling technical architecture everyone has committed to. If we think of user interface changes through this lens, it’s a good way to prioritize things according to where they sit in the stack.

Bad when there’s a relationship between things being voted on:

It’d be awkward to walk out of such a meeting having prioritize B and C, knowing A is dependency for both items.

Example: A team identifies a dozen or more frustrating problems. However, one of these may be the root cause of these frustrations and the best thing to tackle. Sadly, if this isn’t discussed, then you end up with a scattershot of votes, mostly addressing cascading problems, without getting to the core, triggering issue.

Bad when solutions are mixed in with problems (and outcomes)

Start a Friday culture club

and…

Fix problems with employee morale

While they both start with a verb, and may seem like similar things, the first is a proposed solution to the other, broader articulation of a problem.

  • If you just generated possible solutions to a narrowly defined problem, that might be okay.
  • If you were only brainstorming problems (or desired outcomes), that might be okay.

Dot voting on a mix of problems and solutions is very bad. ‘I want to fix that problem, but I don’t think that’s a viable solution, but that other thing over there is a solution I do like to the problem, but I have limited votes, do I vote on the the solution I like or the problem, knowing we might or might not test that solution… Yikes!

Bad when we mix easy and hard options.

Quick, cast your vote! We’re forced to vote on things with very different levels of effort. Or unknown levels of effort. Or unknown value. Maybe we shouldn’t be dot voting on these at all, but seeking to understand them better by placing them into a simple matrix or canvas?

A simple prioritization matrix

Of course, if you must persist with voting, I’ve seen this mixed-difficulty problem solved: Classify your votes!

A Good Dot-Voting Example #1:

A good dot voting example.

In this example, you see six dots in total, two for each type of vote.

Sidenote: In nearly every dot voting exercise I’ve been in, it’s always three dots, and they always signal the same thing. I love that this simple idea examines the dot as our voting unit, and uses color coding to introduce a classification. Moreover, this exercise broke the implicit “three votes” rule. Call the dot voting police! But seriously, think of all the things we could do with our votes:

  • Play with the number of votes people get
  • Use color coding to classify the nature of vote
  • Use color coding for traceability (the engineers, product managers, and designers all get different colors)
  • Use differently sized sticky dots to signal intensity
  • Use two differently colored dots as votes ‘for’ and ‘against’
  • And so on…

While we’re at it, voting need not be limited to dots, as John Willshire suggests:

Bad when there are redundancies and overlaps

Not sure where to even begin! At least they’re all animals. Oh, wait.

Unfortunately, this kind of muddy mix — bulldog, tiger, cats, canines, pets, poodle, clown, etc. — is all too common. Especially, especially, where the brainstorming topic was very broad.

Before voting, examine the things that have been generated. Are some the same? Is the distinction between things clear? Is there an overlap in some things? Like I stated at the outset, most of my frustrations have been with the lack of information architecture in the things we vote on. This mixed information is a classic problem example.

Bad when there is a poor understanding of proposed concepts

Bad for Bold, Visionary Ideas

Exception: Unconventional ideas do get support when the person advocating for this idea is really compelling. Which gets to my next set of issues: The Human Element.

Human Problems with Dot Voting.

Let’s the get the biggest, most obvious issue out of the way first:

Voting is Not Blind

Or, the opposite.

That idea that’s really important, it’s not getting any votes — quick, I’ll cast my vote, and maybe others will follow.

The live nature of dot voting also means votes are easily skewed (and often are!).

Fortunately, there are ways around this.

  • If the things being voted on are numbered, you can have people number their votes (privately), before getting up to make their votes public.
  • Tools such as Feedback Frames keep all votes hidden, until ready to be revealed.
  • And recently, I witnessed another great (and simple) example of this of making the voting blind…

A Good Dot Voting Example #2:

Several teams — each with a competing concept — needed to align behind one concept. Much of the afternoon was spent helping each team actively listen to the other, to fully understand what the other was proposing. That said, the day needed to end with some consensus building. For the voting exercise, the facilitator did several things really well:

  1. Ahead of time, she prepared a structured, objective rubric that would be used to assess five different considerations.
  2. For each consideration, each person could only vote for one of the three concepts.
  3. Rather than live vote, voting was done blind, on a handout. It took a bit of extra time to ‘tally’ the votes, but the results were much more useful.
Blind voting within pre-defined criterion

In this, the facilitator addressed both the structural issues (mentioned above) and the issues with live voting. Best of all, when a decision was made, there was alignment. Everyone agreed with the decision, and (most critically) felt they had been listened to —even where ‘their’ idea wasn’t chosen. This is powerful.

Penmanship and clarity skews votes

Where do your eyes go? What do they gloss over?

You can skew votes by throwing all of your dots on one thing


Conclusion? The best or right idea isn’t necessarily the one that gets the votes. In fact, the idea that gets the most votes is — often, not always — the idea most advocated for (‘squeaky wheel’) or the idea most easily understood (familiarity bias).

While many of the things listed above are tactical, and can be fixed, let’s get to the heart of the matter: How people feel in the end?

Dot Voting Alienates & Isolates:

I love how Austin Govella articulated this:

I never use [dot voting]. You spend all that time aligning everyone, and then turn around and marginalize a bunch of people.

That idea you were so ardently advocating for, no one voted for it. No one. But you still believe it’ s important. Where does this leave you at the end of it all?

I’ve found the value of dot voting primarily as a discussion tool, not a tool for making decisions. Let me repeat that: The value of dot voting is as a discussion tool, not a tool for making decisions.

Unless you take some time beforehand to prepare a tight structure, and you carefully manage for human biases, I’d be cautious about using dot-voting to make a final call or cull out ideas.

A Final Thought:

While writing this, I felt called out — in a good way — by my friend Kate Rutter. She challenged me to:

“Focus on the helpful/good parts and briefly address the limitations. Kinda sad at all the “wrong” being hyped right now.”

True.

There’s way too much “wrong” being hyped right now. In writing this, it was born out of years of mounting frustrations, but tearing down is not my intent. Rather I’d love see everyone build up, develop, and elevate their own abilities. Dot voting can be a great and useful tool, if used thoughtfully.

Stephen P. Anderson

Written by

Speaker, educator, and design leader. On a mission to make learning the hard stuff fun, by creating ‘things to think with’ and ‘spaces’ for generative play.

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade