First steps for better UX research
How we’re making user research more frequent, structured, and transparent at Nubank.
Here at Nubank, it’s extremely important that we keep ourselves fine tuned to our users’ minds and emotions. After all, each aspect of their financial lives is a very personal, delicate matter, extremely rich in details and heterogeneous behavioral patterns. When building for millions of diverse users, qualitative research is the tool to help us keep an empathy window opened so we can design for realities, demographics, and pains that are different from our own.
Nonetheless, as in most companies similar in size and age, there’s no team exclusively responsible for the practice of user research at Nubank. Our design team is mostly composed of generalist, process-oriented professionals (we will write more about this in a future post, so stay tuned). As such, we believe that design is not only about problem-solving (executing), but a lot of times about facilitating problem-understanding. With this in mind, we’ve been acting on our responsibility to push good user research forward, with the ambitious goal of “making every Nubank employee an expert on the people they’re building for”.
Not long ago, our approach to research was one that may sound familiar to many people working in agencies and consultancies: Whenever a new big project came up, we’d assign one or two people to execute an extended, focused sprint of research on the topic, feature, or customer segment, crunch through the collected material and write a report detailing the methodology, producing user profiles, insights and action points.
This way of working, however, quickly got out of sync with the rhythm in which things were happening at Nubank. Consultancy-like research, compared to the agile world, is slow, expensive (in terms of time and human resources), and the final outputs were often ignored by stakeholders who still trusted their guts more than our methodologies.
So we started looking for ways of doing research that would better fit our context, and make it a more distributed, frequent practice that is consistently planned, executed, and documented, without becoming an overhead for any single individual.
1. Frequent
Our user base grows and shifts extremely fast. This means that research done 3 or 4 months ago may very well not be representative of the majority of our users anymore. We need to keep up with the speed in which the market, technology, and our users’ realities evolve.
Our solution to accelerate the pace at which sessions happen was to establish an agenda for user interviews, with an optimistic goal of having some member of the team talking to a user at least once a week. The key value of having an agenda before you even have something to test is that you get the team talking to users no matter what: if there’s no new prototypes or features to test with users, we talk to them anyway. It’s a great opportunity to have them show us their current apps and learn from what’s already in production. Also, publicly setting an expectation about frequency creates a healthy pressure that keeps us moving.
To lower the barriers for this to happen, we had to work on our recruiting process, which involves running database queries, crafting and sending invite emails, buying gift cards, scheduling and keeping in touch with users. This is always going to be time-consuming, but with the help of tools like Typeform, Doodle, and some well-meaning Business Architects, we can now run this step much faster than before.
Also, in the beginning, we had a lot of trouble booking space for research sessions: There was never a room available for long enough, which made us run around and re-setup with gear in between sessions; and business meeting rooms are not especially suited for making people comfortable talking about their lives. To account for that, we’ve built a room in our design corner of the building exclusively for research use. It’s now called Engelbart, (in honor of one of the founding fathers of human–computer interaction), and we’ve replaced tables and office chairs with a sofa, TV rack, snacks, and plants to make people feel more relaxed.
2. Engaging and transparent
Writing polished reports or presentations about our research findings turned out to have very little value to us. We tried sharing PDFs on Slack, putting insight posters up on walls, and still got no sense that people were finding the information useful. Also, having stakeholders be in touch only with the end results often felt frustrating, as it was easy for them to dismiss findings based on personal opinions or biases.
So what we decided to do was to make all the action live. Now, all our research sessions are streamed to a second room where everyone interested is invited to watch, and there are no final reports or presentations to be shared. This move was by far the most valuable of all for us, as it puts product managers, engineers, and other professionals in direct contact with the user they’re working for, helping develop empathy both for the customer and for the design team, making our processes less obscure to professionals with more analytical and rational backgrounds.
Quick note about how we actually do this: we didn’t want a two-way-mirror setup, since we think it’s too obvious and make people very uncomfortable. Our best solution was to plug cameras and microphones to a physical HDMI cable that goes up to a watching room, streaming all video and audio perfectly. We tried streaming through the internet but the quality was never perfect. Also, with streaming stakeholders started watching sessions from their desks, which does not enable for collaborative note-taking and live discussions with others.
3. Well documented
I mentioned that we didn’t do many reports and presentations on UX research anymore, and it’s true: the beauty of having stakeholders all together watching the session is that they can do the parsing and documentation themselves, while the interview is still happening. Furthermore, with more people involved in taking and comparing notes, we reduce bias in the final conclusions, insights, and action points. (see this article on Collaborative User Testing).
We give everyone access to a fresh copy of Jessica Crabb’s incredible Trello Board and instruct them to collectively participate on the sorting of quotes, insights, and action points in real time. This board is great since it helps keep all the documentation (screeners, participant list, interview guide, questionnaires, NDAs, prototypes, raw notes, insights, and action points) in one centralized place.
As a backup, in case something goes wrong with the tech setup or in case there’s no one watching the sessions, we always have a second person in the interviewing room taking their own notes. At the end, we upload all testing videos to a private Youtube channel so people can check back on them if necessary.
Challenges & next steps:
This was just the beginning of our research practice’s evolution here at Nubank. During this time, we’ve learned a ton about processes, methodologies, and best practices from our peers and the external community. But perhaps the most important finding was that there’s still a long way to go. Here are a few things we’re already looking into for the future:
Diversify methods and broaden populations: We’ve done a lot to accelerate and master our in-house user interviews, and they have since added a lot of value to our teams and features. It’s very clear to us now that, in order to keep up with the complexity of our user base, we’ll need to start expanding our vocabulary of research methods, especially in getting out of the office, the privileged neighborhood we are based on, and even start hitting the road to talk to folks and better understand their particular uses of our products.
Built a more cohesive, universal research practice across teams: While we’ve covered a lot of ground for User Research specifically, there’s still a long way to build a company-wide, entirely connected research practice. A lot of growing companies suffer from research silos. This is when multiple teams are doing different kinds of research (market research, A/B tests, user interviews, customer support feedbacks, app analytics, brand research, surveys, etc) that don’t get connected in the end, and there’s no one looking at the big picture of learnings and results. We’ve been reading a lot and have started exploring ways to connect those teams into a conversation that helps to build a shared understanding of what we know and don’t know about our users. If you can relate to the problem described above, and haven't read the article below, please do:
Make knowledge useful and usable: One of the biggest challenges of a growing research practice is the management of the also growing pool of outputs. Different teams are constantly producing large quantities of insights that are well documented and shared with the immediately interested stakeholders, but we’re aware that there are better ways of making this content available and usable for everyone else. Our end-goal should be that any person in the company can type a few keywords somewhere and learn everything that anyone else has researched about that topic. Furthermore, anyone should be able to add their own discoveries and learnings to this “insights database”. We’re big fans of Tomer Sharon’s work on the topic of Democratizing UX and atomic research units. 👇
Make data our friend: It’s very easy to get used to the tools and processes you have more mastery in. Designers will probably be more comfortable with learning from usability tests, product managers with app analytics, and data scientists with running queries on a database. The richer insights, though, will come from the aggregate analysis of this viewpoints. We as designers need to make an effort to be more comfortable with quantitative analysis and to make more data-driven people understand the value of being in a room with a unique user. It’s extremely easy to disregard someone else’s learnings or observations if they were made based on a methodology you don’t understand, trust, or relate to.
Be more accountable: If we designers/researchers want the practice and its results to be taken seriously, there’s a lot we can do to make our efforts more visible and credible for everyone else (as they say: what gets measured, gets managed and improved). Building trust from leaders and helping them take our (often subjective) point of view into consideration when making decisions means that we need to put some effort into translating our learnings to a language they can understand and navigate.
As I've said, we've been growing a lot and learning as we go. Finding the right format for user research in an agile startup environment is a very complex endeavor and one that should be approached step-by-step, as any other design problem. It's definitely worth the effort, though. Every new customer that we get in touch with humbles us and changes us in an unparalleled way, getting us closer to our mission of understanding their problems and execute solutions that truly empower their lives.
We've learned a lot from the community reading your blog and medium posts. If you have more interesting sources, please contribute! We'd also love to hear your experiences on building effective research practices in your organization.
