Changing The Way We Listen To Product Feedback — The Proposal

Part 1 of 2: A case study of the Customer Insights Project that I ran to improve our feedback processes in our products.

At Rex I had identified some key problem areas in the way we collected and utilised customer feedback, and saw some opportunities for improving not only our design and research processes but also our relationship with our customers. This is part covers the (abbreviated) proposal I wrote to pitch this project to the business.

Read Part 2 — The Outcome here ➡️

Problem Area 1: Asking For Product Feedback

All products teams at Rex should have the means to solicit feedback from our customers (in general and at key moments), such as:

• Collecting general product sentiment/satisfaction/value scores
• Assessing satisfaction/value with particular product features/areas, or with newly released changes
• Assessing the success of customer interactions (such as onboarding)

Currently a lot of teams lack the processes and/or tools to collect this feedback. The requirements for collecting feedback from customers can also vary between products and teams — such as in-app collection vs external collection. Some real-life example problems:

• “Is there a way we could get more context on feedback from customers? It would be really helpful to be able to follow up with them.”
• “How do we measure if improvements are successful after shipping?”
• “What are the biggest pain points for our customers during their onboarding?”

🤔 How might we collect product feedback for general and specific cases across all of our products?

Problem Area 2: Collecting & Organising Product Feedback

Rex has many different sources of customer feedback — across several teams and with varying levels of relevance to other teams. Collecting or logging this feedback is irregular and more often than not the feedback is not collected, lost, or not passed on to the relevant parties.

Feedback aggregation should introduce a clear taxonomy and support easy collection and categorisation of feedback from all sources for Rex — as well as any potential new sources brought in by the insight collection milestone. A real-life example problem:

• “I’ve got a customer who has given me some feedback about a feature — how can I log this?”

🤔 How might we aggregate customer feedback from all channels?

Problem Area 3: Presenting Product Feedback

Rex has a large volume of feedback, so how can this be presented in a useful and accessible way? Useful presentation may vary between different teams — the product team may be looking for feedback around a specific feature, where as the marketing team may be looking for general product sentiment. All teams need to be able to easily access and manipulate this data. Some real-life example problems:

• “We delivered a feature and I guess it could have value, but I don’t know. Do people actually like it?”
• “Is there a trend in the feedback we get from UK churns?”
• “Who are our churn risks and what are the issues they are having?”

🤔 How might we present feedback in a useful and accessible way?

Problem Area 4: Actionable Insights

Once feedback has been properly solicited, organised, and presented — how can this be turned into actionable insights? And what does this look like for the various teams? For example:

• Flagging customers or agencies that are giving low value scores in order for customer success to step in early
• Identifying useful trends in feedback:
  — Overall value score increases or decreases
  — Top feedback categories
  — Top features mentioned in feedback — where could we improve?

🤔 How might we turn our product feedback into actionable insights?

Milestones

The project was broken up into 3 milestones. The learnings from the first milestone were used to inform the approach for the second and third milestone — for the sake of this case study I’ve included the final definitions for all milestones here.

Milestone 1: Research & Investigation

The first milestone broadly investigates the problem areas (outlined above).

1. Interview different roles from teams across Rex to learn what value can be gained by addressing the problem areas
2. Research into existing feedback processes in other companies
3. Research into feedback tools/platforms for soliciting, aggregating, and presenting feedback
4. Produce a report with problem definitions and summary of learnings
5. Use learnings to define approach for milestone 2 and 3

Milestone 2: Collecting Customer Sentiment & Feedback

This milestone aims to enable all Rex products to collect product feedback and sentiment directly from customers. There are two objectives:

🎯 1 — Enable teams to continuously measure the overall customer experience and collect general product feedback

Continuous measurement supports teams monitoring the overall health of their product, and allows unfiltered customer feedback to flow in over time. This can take the form of NPS or Value Score surveys, collected in-app or externally depending on the needs of the product.

🎯 2 - Enable teams to evaluate the success of projects by measuring satisfaction and collecting targeted feedback before and after releases

In contrast, satisfaction and feedback can be collected in a more targeted manner for both existing features and for newly released improvements. This can serve as a measure of success for a project along with other key results, and can support iteration based on the feedback that is collected.

Measures of Success

• Product teams are continuously measuring the overall customer experience (objective 1) with an average response rate of at least 12% for in-app surveys and 2.5% for email surveys
• Product teams measure customer satisfaction of a specific feature in-app (objective 2) at least five times with a response rate of at least 15% (measured over a three month period)

Milestone 3: One True Customer Feedback Source

Milestone 3 deals with the broad range of customer feedback sources that Rex has and aims to bring them together into a useful, centralised platform. There are 4 objectives:

🎯 1 — Establish a platform to house customer feedback

🎯 2 — Establish a feedback taxonomy and framework that supports categorising, searching, and analysis of feedback by the product team

🎯 3 — Funnel existing customer feedback sources into this platform

🎯 4 — Establish a process for the broader team to submit customer feedback, that allows them to easily check in on that feedback

These objectives aim to address the key problems which were identified during milestone 1:

• Submitting feedback from customers to the product team lacks process
• Following feedback items or checking the status of feedback is not really possible
• Feedback is frequently not logged or is lost
• Searching for and analysing feedback is painful
• Customers (and staff passing on customer feedback) feel like their voices are not being heard due to lack of communication

Measures of Success

• The feedback platform addresses the key problems above
• All existing sources of Rex customer feedback are feeding into the platform
• Feedback is being submitted at least five times per week by Rex operational teams (measured over a three month period)

Continued in part 2:

Changing The Way We Listen To Customers — The Outcome