Why we have a research plan at Nubank
Or, how a well designed research plan will help you have a clear vision of what you should be studying
Nubank has been building products without a dedicated researcher for a long time. In March 2018 I was hired as the company's first full-time researcher. As of today, I am part of their Nubank Design team, helping both Product and Design teams better understand how our users behave, how they are using our features and where we envision our product to go.
Up until my arrival the designers were responsible for doing research, and they were having a huge overhead dealing with both research operation and design work. They were handling it in a great way, taking some first steps and evolving as fast as they could.
I saw places where I could help — such as best practices, measurement of impact or documentation, and decided to start with the later.
Documentation is a subject I really like, and one that is often overlooked by us (designers).
Just a quick disclaimer: I love to be lean and agile. But, paraphrasing the agile manifest, "less process/documentation and more people" doesn’t mean no documentation or poor documentation - and we all know that poor documentation is worse than no documentation at all.
Any kind of documentation should exist to make your thought process clear and create a shared understanding of the problem.
Why should I use a document?
Usually, everyone that is participating in a project has some personal opinions or thoughts on the subject being researched. A good research document is something that puts everyone on the same page and helps them discuss the research topic from the same perspective.
There are many ways to do that, but a well-designed document will help people debate, agree and easily remember everything that was discussed and their reasoning behind it.
Also, a good document will help the group deepen its understanding of the real issue behind the subject, moving the team from a broader, more open-ended question into a more specific and measurable situation.
Documentation can also work as a form of history. A good document is the best way to show the reasons behind the decisions we made, and why some choices were implemented.
How we do it
We like to start by distilling the problem into a list of the real and measurable questions we want to answer in the most granular way possible.
And to do that, we focus on 4 topics. (1) Goals, (2) Questions we need to answer, (3) Expected impact and (4) Hypothesis.
Following guidelines will make everyone in the room think harder about the problem. Perhaps, some people’s expectations were just too high, or they were simply looking at the question from a wrong angle or considering a different result altogether. Again, documenting puts everyone on the same page.
(1) Goals
The goal is, in fact, the open-ended question we began with — the broad problem that we all have at the beginning of the research. "Help us increase the number of messages exchanged inside the app", for instance. Well, that is super broad and may mean different things for each stakeholder, right?
(2) Questions we need to answer
These are not the actual questions we are going to ask users, these are broad questions that we want to answer during our research.
To transform this goal into something more tangible, we need to have a clear picture of the questions being asked. Asking your stakeholders what they need to achieve that goal will result in a group of more grounded questions like: "Are the users able to create new conversations? Is there any usability issue on the chat? Do users understand that they can send multiple messages at once?" and so on.
This questions will help you shape the methods you'll end up using during your research. And will also help you know if you are on the right path to answer your goal.
(3) Expected impact
With that in mind, we can shape the expected impact that our research will have on the users. This will also help us guide the direction of the research and some of the methodologies we may choose. "Remove any usability flaws that we find. Increase the number of new conversations started. Increase the number of chats created. Better understand why users create groups", are some of the examples.
This is how you will measure the outcomes of your research, and this will play an important part in shaping how you will collect your data. So, take your time and be sure that you and the stakeholders know exactly what you are expecting, otherwise you may end up not reaching someones expectations.
After going through these 3 steps, and with a clear picture of what you are trying to achieve, you can finally start working on the final step: your hypothesis.
(4) Hypothesis
According to Google, a hypothesis is:
"a supposition or proposed explanation made on the basis of limited evidence as a starting point for further investigation."
A hypothesis can be written in multiple forms, but most of them are structured in an if/then situation. A well-written hypothesis will help you know what to test and give you an idea on how to test it. "Asking for the user to put an personalized avatar at the beginning of the registration process does not affect the number of messages sent" is one example of a hypothesis.
Having everyone agree on the same hypothesis may be tough, but again, this will make everyone think even harder about the problem that is being studied.
Final considerations
The more we grow, and as more people have an interest in research, the more relevant this type of document becomes. So, putting some effort in it will always bring good results.
We usually start this document during a kickoff meeting, but the meeting works more as a way to start the research process. We tend to finish the document in a more collaborative fashion, over a shared Google Docs file.
Sometimes these document may be longer than usual to be completed, but, the amount of time and effort that you put into each of these steps will change according to the size of the research.
Documentation works more like a way to make people think about what they want to uncover, what are their real doubts and what is the impact that they want to cause. But remember: documentation is a tool — not something carved in stone and cannot be changed.
Feel free to check the template that we use.
What about you? Do you have a research plan document? What for? What does it look like?
