Listening to user feedback
Learning how to make the most of our customers' requests and feedback.
At Nubank, we are lucky to have an engaged and passionate community of users, who submit hundreds of feedback and feature request messages every week through our support channels and social media.
This steady and loyal stream of input is like a gold mine to our lucky product teams. It is rare that such a rich and voluntary source of insight into user’s needs and pains is this readily available.
Managing this information and putting it to good use, though, has been much trickier than we’d imagine. In this article, we'll share some of the challenges we’ve faced, how we’ve dealt with them until now, and what we’re planning to do next.
Phase 1: screenshot copy and pasting
When I joined Nubank back in 2016, we were nearing the 1MM active customer mark. At this point, every feedback message we received was proactively posted by our customer support experts (xpeers) in a Slack channel called #feedbacks.
It is interesting to note that nobody demanded that support team members report customer feedbacks to the product teams. This workflow was a natural result of having a highly-trained support team that feel as much ownership of the final product as designers, PMs and engineers.
To this date, anyone in the company can join the #feedbacks channel, read through common and hot requests in real time, collaboratively discuss them in threads, propose solutions and @ mention stakeholders to get their attention on important issues.
As I write this, the #feedback channel is the 34th most popular channel out of 1454 channels in our Slack account. It has 232 members and is bubbling with real-time customer messages all day.
👍 Value added by this first simple solution:
- Centralised home for all customer feedback in Slack
- Threaded discussions about each problem/idea
- Bring visibility to important bugs or urgent problems with the app
Even though this was a great start, there were evident problems that needed addressing:
👎 Downsides of the first solution:
- Screenshots are impossible to search, parse, or analyse.
- It can only be consumed as a feed. There's no way to generate insights from an aggregate view
- A single channel for all feedback topics quickly became too noisy as we grew to a multi-product company
Phase 2: multi-channel
In order to adapt to our growth, squads naturally started creating spin-off feedback channels specific to their domain.
So beyond the main #feedback channel, which became home to more “generic” topics, we started seeing channels like #virtual-card-feedback, #chargeback-feedback, and so on, get created every now and then.
With this new approach, stakeholders now could subscribe to specific topics and understand a less-noisy subset of user’s needs.
Phase 3: Spreadsheets
The biggest change came when we decided it was time to abandon screenshots to adopt a more structured online submission form. We created a simple Google Forms and applied it to every feedback channel as a centralised submission system.
At the beginning, we worried that customer support engagement was going to drop because of the extra friction. At the same time, though, the value we'd be able to derive from structured, parsable data was definitely worth the risk.
We also made sure that when the form was submitted, a bot replicated the submission to Slack, mirroring our older model. This way, on-the-spot discussions and threads could still happen on Slack, while product teams would still be empowered with a structured database on Google Sheets.
This new workflow felt like a vast improvement, and we ran with it, unchanged, for a while. We could now categorize and rank feature requests, get in touch with specific users to understand their needs more deeply, and share more insightful reports with the company.
However, as we all know, where there’s a spreadsheet workflow, there’s an app. Very quickly we felt that reading through a long list of requests on Google Sheets didn’t offer the best analysis experience, so we started shopping for third-party apps that could help us manage customer feedback.
Phase 3: Third-party platform
The idea first came when we landed on this video from MailChimp, where their research team explains how they built a global, searchable database of insights inside Evernote.
We tried building a similar workflow, but it didn’t feel quite right to our specific needs. We shopped around for a bit until we eventually found and felt at home with ProductBoard.
By directing our Google Form submissions to Product Board, we can sort messages by product, sub-product, tags, and have discussions in context. We can export contact information for all customers that requested feature X and get a researcher, designer, or analyst talking to them.
More importantly, a specialised tool gave us the right mindset to look at feature requests. They are very opinionated about not treating feature requests as an all-important ranking that tells us what to build next, but instead as one data point among many others that we should consider to make better decisions.
For us, this dashboard has become the starting point for many projects. It’s a place we go to get a first feel of customers' needs, raise initial hypothesis for the research studies, and spread awareness about low-priority but pressing issues in our products. It has become an indispensable piece of the discovery puzzle, not merely a source of truth to roadmap decisions.
Next Steps
Our current solution gives us a much better sense of control over the information we collect, but the process is far from perfect. It is hard to estimate the cost/benefit of the enormous effort required to report, sort, and categorize all the messages we receive, and it will certainly not scale at the speed we need.
Here are some next steps we’ve been thinking about for future iterations.
Input Friction
An important step we could take involves finding ways for customers to self-report their requests without opening customer support tickets. Doing this will reduce the load created by feedback tickets that don't require support, and give us a less-biased stream of insights, which today inevitably get filtered by the agent's level of attention, squad, or even personal interest in the feature at hand.
Some companies solve this by creating a specific “feedback” entry point and flow on their apps, that is separate from “help” sections and knowledge bases. Others direct customers to an online form and ask them to fill it themselves.
Sorting
It is tough to keep up with the stream of feature requests if we have to read and sort them in categories by hand. Looking ahead, we need a model that allows customers to pre-sort their messages. This would require that we create a taxonomy that both customers can understand and that we can use internally to redirect messages to appropriate channels.
Looking even further, we might want to have machine learning algorithms do the hard, repetitive work, the same way they currently do for routing support tickets.
Community-focused
Some companies we admire, such as Monzo, have adopted an “open roadmap” model where users can request, vote, and discuss features online. It is not clear to us yet if this model would work in our context, but it is a solution that scales well, and one that gives users an incredible sense of ownership and empowerment.
In conclusion, there are many ways to deal with customer requests, and it is yet another complex design problem we love to work on and talk about here at Nubank. It might be overwhelming to deal with this kind of information at this scale, but with the right mindset, it can become one of the cheapest, most valuable source of data-points for better decision making in our product roadmap. We hope to have inspired you to look more carefully at the proactive feedback your customers are voluntarily submitting, and that you use this information wisely to build things they love.
We’d love to hear from the design and product community on this subject. Please leave your comments with any feedback, ideas, and your own experiences below ;)
