Running prioritization workshops

How to run a prioritization workshop using the impact by method effort

Impact / Effort Matrix

Run a prioritization workshop after each watch party

We talked about watch parties in our last article, which gave us deep insight into what end users were saying about our product and how they were using it. After we ran our watch party, our team wondered, “what’s next?”.

This seemed like a perfect opportunity to introduce a lightweight workshop to prioritize feedback after a watch party or user research study.

Involve the whole team without letting consensus drive your prioritization

We wanted to involve the whole team somehow so that they wouldn’t feel like prioritization decisions are made in a silo. We had previously included only the engineering manager or tech lead to help with estimating effort, but the IC engineers felt left out when these projects eventually landed on their plates. This time around, we wanted to involve the whole engineering team so that the engineers closest to the code and actually building our features could participate.

To ensure this was worth everyone’s time, we created a structured agenda, gave everyone votes to express their own opinions, and limited the meeting to 1 hour (see Recipe for more details). At the start of the meeting, we made it clear that the PM would have the final decision on prioritization as the roadmap owner, but everyone would have a chance to share their opinion.

Select the right prioritization framework to structure your discussion

Because we wanted to invite ~10 people on the team to participate, we selected an Impact/Effort prioritization framework. Impact/Effort was the simplest concept to both explain and understand. We found that more complex ranking systems didn’t work well with large groups due to ambiguity around ranking definitions. Items in the top right of the matrix (high impact, high effort) are bigger roadmap items that need to be thoughtfully slotted in, while the top left (high impact, low effort) should likely be looked into immediately. We largely ignored low-impact items to create focus for the team.

Encourage healthy disagreement to create the best discussions

We selected 5 key issues from the watch party to discuss one by one, and the goal was to place each issue on one of the four quadrants in the Impact/Effort matrix. The team should already have context on each issue because they participated in the watch party. The goal is for team members to openly discuss why they think it’s worth it or not worth it to fix this problem. The best discussions came out of healthy disagreements between team members on impact or effort. There were some issues where we couldn’t agree on impact. We left those issues in the low-impact quadrants but created action items for us to follow up in a future research study.

Those who were more vocal actively participated in discussing each problem, whereas those who were less vocal participated through voting at the end. There were cases where no one would speak up when a new problem was introduced, and we resolved that by having the PM and design lead provide their opinion on which quadrant they felt the problem belonged to get the ball rolling.

Leave the workshop with more ownership and clarity across the team

Coming out of this workshop, we saw a few signs that showed this was working:

1. Engineers were excited by the easy-to-fix, high-impact items because they could ship something fast that they knew the whole team thought would improve our user experience significantly. They created tasks to work on those items right away.
2. Voting on items was a hit. You could now get a pulse on what was important to the team and use that as input to prioritization.
3. Engineers knew what big rocks were under consideration for the next 3–6 months, and this knowledge influenced in-progress architectural and design decisions.
4. PMs knew what product specs they needed to write next for higher impact, higher effort items.

Recipe

Preparing for the workshop

1. Identify moderators for the workshop who will help keep the agenda on track and facilitate discussion. We had 1 PM, and 1 Designer run this.
2. Create a Miro board for your workshop with the following sections: (1) Items up for discussion: Will have post-its that represent each issue the team will discuss together (2) Prioritization Matrix: Area in the board with the Impact/Effort matrix (3) Parking Lot: Area to collect ideas that need more discussion outside of this meeting. This will help you keep the meeting on track without being rude or losing track of the topic.
3. Create a strict 1-hour agenda for the meeting and invite the whole team, including user researchers, every engineer, and every designer. Only one person will need to share their screen, but everyone will need access to do voting. Here is ours: (1) Intro — 5 minutes (2) Place each theme on the Impact/ Effort matrix — Spend about 5 minutes each for 45 minutes (3) Team Voting — 5–10 minutes (4) Wrap-up — 5 minutes
4. Group together emerging themes and key issues from your watch party videos. This will take a significant chunk of time from moderators. Create a note card for each theme in the Miro Board in a separate area from your prioritization framework.
5. Within each card, write down additional data points like (1) how severe were some of these issues? and (2) how many users were impacted and under what conditions to make sure you have enough data to make prioritization decisions.
6. Go through the list of cards and select only the top 5–10 issues that would benefit from a discussion on effort or impact. You should assume it will take about 5–7 minutes to discuss each theme. There will likely be way more issues to discuss than the 1-hour meeting will allow. The PM should pick ones that are difficult to estimate for effort and need engineering input, and/or are controversial.
7. Place the remaining issues that you don’t plan to discuss with the group onto the matrix for visibility.

Running the workshop

1. Select one issue at a time to discuss and place it onto the matrix.
2. Use a timer feature to give each issue 5 minutes. To keep the discussion on track, the PM or moderator should designate another person to stop the discussion if it’s not relevant or is taking too much time to resolve. To ensure you don’t lose track of that topic, place it as a post-it in the parking lot area. Once you’ve gone through each theme, you should have 1 matrix with a theme in one of the 4 quadrants as the output.
3. Now, it’s time to vote. Give every team member 3 votes to pick the items that most resonate with them.
4. Send out a recap of action items

Ingredients

• White-boarding tool like Miro or Figjam that ideally has a built-in voting feature
• Template for a prioritization framework of your choice. We used Miro’s built-in Impact X Effort matrix.

Thanks for reading our recipe. Our goal is to share our experiences working together using a short-read format with instructions that you can try on your own — just like a cookbook. We hope you found it useful!

Katie Le & Nicolas Backal