Driving Diverse Product Feedback at Your Company
As a Product Manager, your main job is to build consensus and get input from all stakeholders on the product development processes. However, consensus building is difficult and if you are not careful you will end up in a situation where the “loudest voice wins.” In many cases the loudest voices will belong to the most privileged people in your organization, so product managers run the risk of amplifying inequalities if they are not mindful and deliberate about preventing them.
Beyond the fairness factors, your products simply won’t be very good if you don’t get a diversity of inputs.
Obviously, it is going to be hard to get a diversity of inputs if you don’t have a diverse team. For a good recent study in AI with recommendations for ensuring that you have a diverse workforce, see Discriminating Systems: Gender, Race, and Power in AI. In this article, I’ll assume that you do have diversity among your stakeholders, but they are not representative. I’ll also assume a broad definition of diversity, meaning any broad range of personalities, identities, and experiences, not just the demographics that tend to be protected classes.
There are many ways to tackle diversity and this article is focused on just one: how to ensure more feedback from diverse stakeholders as a Product Manager during the PR FAQ product process. If you aren’t familiar with PR FAQ documents and the review process, I recommend reading these articles first:
- PR FAQs for Product Documents — Everything Product Managers Need to Know
- Tactics for Leading and Optimizing PR FAQ Reviews
The most common way to elicit feedback about product documents is to hold meetings with as many stakeholders as possible. This is true for any product process, and especially true for the PR FAQ review process. A PR FAQ meeting starts with everyone reading the document and then asking questions. It is an incredibly effective process to refine products.
What biases can you inadvertently perpetuate in a product meeting?
There are three common practices in product meetings that can lead to inequality:
- You are asking people to react straight away
- You are asking for feedback in a group setting
- You might be asking people to disagree with their manager
Almost everyone will have some degree of nervousness with these three practices. In general, you don’t want to miss out on anyone’s feedback because that person is nervous in a group meeting.
Anyone can learn to be a confident and quick decision maker. It is hard (but not impossible) to become an executive without developing decisiveness. But not every stakeholder for your products will have developed the skills to make tough decisions right in the moment, and you don’t want to exclude those people from contributing. The first people to offer opinions in a meeting will often set the tone and focus for discussion, and if those people tend to be the most privileged, then it might shape the product direction away from a diversity of viewpoints.
There are strong correlations between marginalized groups and people’s individual reluctance to provide feedback in group meetings. I’ll give a few examples, with the caveat that they are by no means comprehensive. You’ll learn some mitigation strategies in the next section.
The most obvious potential inequality is language. If you are running a meeting in English, then non-fluent speakers of English are less likely to share their thoughts. Language correlates strongly with ethnicity and country of origin. In some places, like California, it is illegal to discriminate based on the language(s) that a person speaks or signs. Your product documents should be in plain enough language that second-language speakers can understand them, which helps. But your meetings need to account for feedback from less fluent speakers.
Some of the other potential inequalities are more subtle. For example, in many Western countries, boys are taught to express their opinions more quickly and more loudly than girls. There is no genetic basis for any difference, but this difference in socialization as children can carry into adulthood. This can be exaggerated by meetings where there is already a gender imbalance and is one of the causes behind which the “lean-in” movement seeks to address. So, merely by sitting everyone in a circle at a meeting and asking their opinions, you might be perpetuating gender biases.
One of the biggest myths in Silicon Valley is the egalitarianism of meetings, where everybody sits in a room are therefore equals when discussing some problem. The most extreme examples are at famously secretive companies like Apple where you might not even know the actual job of everyone that you’re meeting with. Regardless of the company and setting, people are conscious of who is more senior and who is more junior. This can shape people’s behavior, especially for someone whose manager is in the room.
This is something I can admit to: I am comfortable in almost any setting, but I get very uncomfortable if I’m put in a position of disagreeing with a more senior person in front of other people, so I try to avoid it altogether and return to the discussion privately. If someone comes to you after a meeting to argue against something that they agreed to in that product meeting, that might a cause: they weren’t comfortable disagreeing with a more senior person in your meeting or going against what felt like consensus. Try not to throw it back at them “but that’s not what you said…” even if that is what you are feeling.
There is a cultural dimension to this, too. In a lot of Western countries, people will have no problem disagreeing with their boss in front of other people if it is over a minor matter like a product decision. By contrast, in the rest of the world and especially in many countries in Asia, people will avoid disagreeing with their manager in front of other people at all cost. Again, there is no genetic basis for it: they are cultural trends. But as someone’s upbringing will correlate with ethnicity and national origin, you run the risk of perpetuating ethnic and country of origin biases.
6 Steps for how to drive diverse product feedback
The main way to mitigate these issues is to ensure that there are multiple ways for people to give feedback on your product documents and being mindful about who is in each meeting relative to their place in the company. I’ll expand on this below.
This is the main motivation behind why I run the PR FAQ review process slightly differently from the way I learned it at Amazon, which is covered in this article:
1. Elicit questions in advance and let them be anonymous
You should circulate the product document in advance and invite questions about it. Many people will be more comfortable having more time to think about the questions, rather than asking them to react straight away to the document.
Many people will also be more comfortable providing questions privately. You should let advanced questions be anonymous if the person wishes, to avoid any confrontation that might occur when the question is later discussed in the actual meeting. As the Product Manager, you should then own the question and any response to it in the actual meeting.
2. Elicit questions after the meeting
If someone hasn’t read the product document in advance and isn’t comfortable discussing it at the meeting itself, you can invite questions afterward. As I said in the article on running PR FAQs, I try to avoid this because it means not everyone in the meeting will get to consider any late-coming inputs. I prefer to include that person in another meeting in the future. Other options include having a Phase 1 and 2 of the meeting so that there is a natural gap, or to include those follow-up questions for discussion in a following meeting even if the person who posed them can’t attend.
3. Have more meetings with fewer people
No one wants more meetings, but the benefits of this should be obvious: people are more likely to be comfortable participating in meetings with fewer people. Shorter meetings can help, but this is hard with the PR FAQ process as you don’t want to rush (re)reading the PR FAQ document at the start of the meeting.
In general, smaller meetings are a better use of people’s time, too. If you have 18 stakeholders, it is better to have 3 meetings with 6 people, instead of one meeting with 18 people. This way, you are getting more information per person and more hours of feedback overall. Obviously, it takes up more of your time as a product manager, so this is a trade-off that you need to consider.
The other trade-off is that you often want a lot of different stakeholders in the same meeting, because one of the goals will be to get all those stakeholders aligned on the product direction. So, you might want to be strategic and have smaller meetings where you are strategically inviting stakeholders from different parts of the organization to each PR FAQ review. This can be more effective than having one meeting per-department, which is probably your default strategy for breaking down meetings. Another option is to have representatives from different department or functions gather feedback, but you will have to be comfortable turning over the review process to those people.
4. Avoid inviting too many people to a meeting with their manager
For the reasons outlined above, people will be more comfortable talking freely if their manager is not in the meeting.
Again, inviting everyone from one team to a review meeting, manager included, will probably be your default behavior as a Product Manager. So be mindful about your decisions on who to invite. Product Managers tend to work relatively freely up and down an organization’s hierarchy to get everyone’s buy-in on new products and features. So, be mindful that you are not missing important power dynamics that other people might be experiencing, even if you are not experiencing those power dynamics as a Product Manager. As the PR FAQ review process tends to go up the chain of management in any case, you might want to consider only inviting people at the same level for meetings: individual contributors at one meeting, managers at another, directors at another, etc..
5. Invite people directly to speak and follow-up afterward
This is good advice in general and not specific to PR FAQ reviews, or even to product meetings. If you think someone might have an opinion but they aren’t sharing it during a meeting, specifically invite that person to speak. Be especially mindful of this for people joining by phone or video conference, as they are the easiest to forget about.
Relatedly, ask for feedback and not whether people have feedback. For example, don’t ask “are there any more questions?,” ask “what other questions do you have?” There’s a lot of evidence that the latter will elicit more information. If you’re a regular public speaker or have worked in sales, you probably have this skill already. As a Product Manager you are probably aware of the likely trouble areas and if you aren’t getting feedback in those areas, you should address the people with the expertise.
Make note of the body language and facial expressions when surveying the room. You should be sensitive to when people clearly want to share information but are holding back. You will need to decide if asking them to contribute to the meeting is going to give them a voice or put them on the spot. Probably a bit of both, so be empathetic.
6. Always read the product document yourself, slowly
If your product document is in English, then make sure that any non-English speakers have time to read the document.
One of the stranger elements of a PR FAQ Review meeting for people new to the process is the 10–15 minutes of silence at the start of each meeting, when everyone reads the document. People will naturally feel like skimming it and getting the silence over with as soon as possible. It is important that everyone reads it closely so as not to skip any of the existing FAQs only to ask them again. Someone who is a slower reader, for any reason, might feel like they are holding up the meeting if they take too long.
The best way to mitigate people skimming the document is for you as the Product Manager to carefully read your document once more at the start of the meeting. Don’t sit there looking expectantly at the room until you are certain that everyone has read the document. By reading the document carefully and slowly, you are giving everyone else in the room permission to do the same. This is a great strategy for ensuring quality feedback and mitigating biases.
Conclusion: find the strategy that works for you
The situation in your organization will have its own particular quirks. Should you overcompensate? Definitely not. Your own organization will have its own potentially biases that you will need to navigate.
For example, I wouldn’t ban anyone from disagreeing with their manager in a product meeting, just because I know it will make some people uncomfortable. Instead, I note anyone who seems uncomfortable and follow up to get their input later. However, your organization may be different and you might explicitly ask people not to disagree with their managers in the meeting, if that is what is most appropriate for you.
Alternatively, you might find a middle-ground, like implementing the “yes, and” policy from improv, where people are encouraged to present new arguments as building on the previous point instead of arguing against that point. You could make this an explicit policy, or decide instead to privately coach people who you think would benefit the most from this strategy. If someone pushes back on coaching because they do not personally feel discomfort, it can help to remind them that these kinds of strategies are done as much for the comfort of people watching the discussion, as for the people disagreeing with each other.
There are many aspects to diversity and inclusion and many steps to the product development process. All are equally important, and as I said at the start, this article is focused on just one: ensuring more feedback from diverse stakeholders as a Product Manager during the PR FAQ product process. Some of these lessons will carry over to other parts of an organization and product process and some will not. I welcome comments about the suggestions here or about diversity in the product development process more broadly.
Acknowledgements:
Thank you to seasoned Product expert, Greg Chang, for detailed feedback on this article.
