A KJ activity in progress in a conference room.

Reaching team consensus quickly: Online KJ activity, using Google Slides

Priorities are hard. Getting there in a distributed team can make getting there even more challenging.

From Morgan Miller and Dana Chisnell

KJ activities are great for getting teams to consensus priorities democratically and very quickly. Doing them in person is energizing and can be fun. Although they weren’t designed for it, originally, they’re a great tool in the toolbox of collaborative synthesis for user research and usability testing.

But there are times when you can’t meet in person for a KJ. We’re writing this in the middle of a pandemic, where everyone on the teams we’re working with is remote from one another. Teams that have long been remote may already have discovered the approach we are about to describe. Most of the people Dana talked with about how to do a KJ online went with electronic whiteboards like Miro or Mural.y.

While online whiteboards seem ideal for mimicking real life for an activity where people are using sticky notes and moving them around, it turns out that those tools are difficult for visitors to use. There are interactions and affordances that are unfamiliar. It is not like being physically present and co-located because you can’t see the other bodies moving around. So having multiple people moving things around at the same time just looks like stuff flying around in the frame of a screen. It can be frustrating and disturbing, especially for new users. It’s easy for people who are more familiar with the user interface to take over.

In addition, access is sometimes problematic for teams. Some of the tools demand setting up accounts, which some users are barred from doing by security rules of their organizations.

But if you’re working in a remote team, what do you do? We looked for something that was lighter weight, with interactions that people might find similar to tools they already use. Nearly everyone has interacted with software for making presentation slides. Why not try Google Slides as a KJ collaboration tool?

The team we worked with were allowed to use Google Slides as long as they didn’t have to log in or download anything to their computers. Hurdle averted there. We conducted the meeting in a Zoom session, but this would work through Google Meet or Webex, or whatever video conferencing your team uses.

Short version: It took some setting up ahead of time and there are refinements to make for next time, but overall, it worked pretty well.

The general structure is the same as you might do in person:

1. Identify a focus question
2. Bring in all the people who are part of decision making
3. Individually capture observations or assertions related to the focus questions
4. Post them randomly so everyone can see what everyone captured
5. Sort into categories
6. Label the categories
7. Identify the top three categories important to answering the focus question
8. Vote, using the labels that best describe the the priority categories
9. Count the votes
10. Tada! Priorities. Discussion.

Doing a KJ in a real conference room goes quickly (mainly because there is not talking until step 10). No matter how many people (we’ve done this with small teams and very large teams) or how many sticky notes people make with their individual observations or assertions, we’ve never had it take longer than 90 minutes to get through to counted votes (step 9). Online, it took somewhat longer — about 2 and a half hours. But still, not too bad, considering. After this team has done it once, it’s likely they’ll go faster next time.

This was an experiment born out of new constraints. We’ve done exactly 1 of these this way. There are some kinks to work out. But this is how we did it (and includes some things we might do differently next time.)

Setting up the deck

1 Open up a Google Slide deck and give it a name. Choose the most generic slide template you can find. We set a custom size for ours (File / Page Setup / Custom) of 20 x 15 inches. Create a title slide and a bunch of blank slides. Here’s a template based on what we did.

2 Choose an appropriate focus question. The team we were working with had just completed several user research / usability testing sessions. So, our question was similar to “What do we need to change to better meet members’ needs?”

3 On the title slide, enter the focus question. (You might want to put it in the footer of each slide as a reminder. Or in the speaker notes.)

4 Create a slide with a couple dozen text boxes. Space them out around the slide. Include some dots for voting. (We put ours close to the margin on the right side of each slide for each participant. If folks are using smaller screens, you might want to make the voting dots a bit bigger to make them easier to grab. Fitts’ law and all that.) Ours looked like this:

Slide with text boxes for observations (including an example), and dots for voting later.

5 Create a copy of that slide for each person who will participate in the KJ. We had 5 people who each got their own slide. We also chose a box color (border) for each person on the team. (If you were using sticky notes in analog space, you probably all would use the same color, so up to you.) Label each of the observations slides with a person’s name. Like this:

Labeling each personal slide with a name of a team member.

6 Create a slide for people to move their observations onto and sort them into categories. We wanted to give plenty of space for clustering, so we had 2 large gray areas (seems right) to copy and paste observations into. Add text boxes for labeling the categories. Make enough labels for everyone participating to label every category. Ours looked like this:

Slide template for grouping text boxes that have observations in them.

One change we would make is to add instructions on the slide or in the speaker notes on these slides for grouping observations that makes it clear that there should only be one group per gray area. If you want to break up a group, move it to an empty gray area.

7 Duplicate this slide several times. We had 9 slides that looked like this. One remained empty by the time we finished. Better to have too many than too few, but it’s easy to duplicate in the middle of the activity, if you need to. We made a lot because we wanted to signal that there was plenty of room to move things around to curate categories, so we created what we thought would be more than enough of these category slides.

8 Add one blank slide at the end for tallying votes.

9 Set up sharing for all of the participants so they can access without logging in and so everyone can edit.

10 Make sure everyone has the link to the deck.

11 Set up the meeting with Zoom or Webex or whatever you’re using, and invite everyone.

Running the session

1 Open up the deck, yourself.

2 Welcome everyone into the meeting, make sure they all have access and can see the things you need them to see.

3 Introduce the general process and present the focus question.

4 Capture observations. Explain documenting observations from the study by saying something like:

In this shared slide deck, each of you will find a slide with your name on it. On the slide you’ll find prepared text boxes.

In these boxes, please write out all of your learnings and ideas which answer our focus question. Put one observation per box from what you heard and saw during our research sessions.

You can either wait until you see people look up and there aren’t any more new boxes being filled in, or you can timebox this step.

If you decide to timebox it, at the beginning, tell your folks how much time they have. Give them a 2 minute warning, and ask if people need more time. If they indicate they do need more time, add a few minutes and tell them how many more they have.

5 Start moving observations to slides where they can be sorted into categories. Say something like this:

Now we’re going to use the slides with shaded columns to start collecting observations together. Without talking, please move your sticky notes to these affinity grouping slides. Take items that seem like they belong together and place them in a group, spatially separate from other groups and non-grouped items.

Keep going until all the text boxes with observations get moved to slides with categories. Please note we have created many more groups than we expect you to need. But you may create as many groups as you need. Just duplicate a grouping slide to get started with more categories.

The category slides might look like these from our KJ:

Example groupings in progress.

Different set of example groupings in progress.

6 After all of the text boxes have been moved to the grouping slides, now is a good time to have everyone pause and look around again. Take a minute to tidy things up. Tell participants to take a couple of minutes to review all of the observations in all of the groups. Say:

Let’s take a moment now to look around at all the slides with groupings and categories. Give me a thumbs up when you’re done.

7 Invite participants to move text boxes to groups they feel the observations are better suited to. They can also add new observations now, split or join categories. Say:

Okay, now that you’ve had a chance to review, you might have ideas for more observations. Go ahead and add those. You also probably thought of moving things around. Feel free to move items into groups other people created. Additionally, everyone is free to rearrange the items until the grouping makes sense. You don’t have to ask permission, just do it.

As you read through any given group, you may feel like the group really has two themes. Feel free to split those groups as appropriate. Similarly, you may notice that two groups really share the same theme. In that case, you can combine those groups into one.

Turn off your camera when you’re done moving things around.

8 Tell everyone to turn their cameras back on. It’s time to label the categories. Everyone labels every category. Say:

Now I want everyone to name each group with a representative theme.

See the rectangles with “Label” in them along the outside of each of the grouping areas? That’s where you’re going to put your label for that group or category.

So, read through each group and write down a name that best represents each group. The names themselves should be a string of nouns like ‘Printer Problems’. Please be as succinct as possible and refrain from full sentences. Every person should give every group a name. The only time you may skip a group is if someone else has already written a name using the exact words you would have chosen.

Here’s an example with labels filled in:

Grouped categories with proposed labels.

9 It’s time to review the focus question again and have another review of what’s the categories to prepare for voting on priorities. Here, participants narrow down the options to what they think is most important to address the focus question. Say:

Let’s review our focus question again: [Say the focus question.] With this in mind — on a piece of scrap paper that will not have to be shared, please write down the three labels of the groups you think best answer this question. If a group has more than one label, please choose the name that best represents the theme of the group for you.

Next, please rank your choices in order of importance.

10 Now everyone votes. They have a total of 6 votes. One for their third priority category, 2 for the second, and 3 for the first. Participants will have to go back to the slide with their name on it to copy the red dots to paste near labels in the slides with groupings / categories. Say:

Time to vote! For the category and label you picked as third most important to answering the focus question, paste ONE red dot on it or near it. [Wait for everyone to do that.]

For the category and label you picked as the second most important to answering the focus question, paste TWO red dots on or near it.

Finally, for your first choice, paste THREE red dots on or near the category label you picked to be most important in answering the focus question.

This is an example of what voting looks like:

Slide with grouped categories, proposed labels, and red dots showing votes from teammates.

11 Make a separate slide for tallying the votes and counting off. Ask someone who is a participant to go through the slides with grouped categories and tell you which labels have how many votes. Say:

[Name], would you please back up to the first slide with votes on it, and read off which categories have votes and how many votes there are?

Type the label and number of votes on the last, open slide.

12 Organize the labels with their votes with the top vote getter at the top, and the rest in descending order. Read off the labels and the vote counts. The top 3–5 are now the consensus priorities of the team participating.

This is what the vote tally slide might look like:

List of voted topics in order of number of votes.

Facilitating discussion of the priorities

Now we can discuss if there are any labels within those that got votes that should be combined, and why. At this point, it doesn’t really matter which groups/categories they came from. The idea now is to get to a shared understanding of the priorities because you’ve eliminated all the things that are not priorities to anyone in the room.

Participants can propose combining and give their rationale. You should probably repeat back what you believe the proposal is to the group and ask for discussion. You’re basically parliamentarian at this point, so you can call for a vote whenever you feel like the discussion is ending. To combine categories, the vote must be unanimous. If there is dissent, the categories stay separate.

To initiate discussion, you might start by saying something like this:

Ok. These are the priorities of the group. Take a minute to process that and form any questions you might have.

If you feel that any of these are actually the same things, you can nominate two groups if you think they are identical to combine them.

If one feels like a subset of another, that does NOT qualify for combining.

After you nominate groups to combine, we’ll take a quick preliminary vote. If anyone believes the groups should stay separate, we’ll have a short discussion followed by a final vote. Only groups with a unanimous vote will be combined.

Continue like this until the group is in agreement or there’s just a lot of churn and it feels like nothing new is happening. Assign someone to document the priorities and report them out.

. . .

Morgan Miller is a math enthusiast who realized she also loves people. She is a freelance User Experience Architect designing user research, strategic documentation, and site architecture.