Building Together at Uber
When Research Went Viral
Building From The Ground Up
In late 2017, Uber was nearly a year into a complete redesign of its Driver app and was about to launch its first ever global beta test, with 700+ Uber driver-partners across 7 cities. The Driver UX Research team designed a research program that departed significantly from previous approaches. Its goal: to better gather and use beta drivers’ feedback by including a mix of customizable ethnographic methods and an unusual way of delivering the research learnings back to the team.
The team will present this case study in Glasgow at the 2019 Computer-Human Interaction (CHI) conference, held May 4–9, 2019, describing the details of the research and what made it go viral inside the company.
Researcher Roundtable
To find out what went into beta testing Uber’s Driver app, I sat down with some of the key people involved in this research success story.
UX Research Team Interview
Your paper describes a research program you carried out shortly before the global rollout of a redesigned Driver app. What is the problem that your research was focused on solving?
Bjorn: Well, a total Driver app redesign is a big deal. This is an app that a huge number of driver-partners all around the world use to earn money — it’s important to us that it works for them, and the best way to make sure it does is to get actual feedback on the new app and understand how well it works out in the real world. That was the goal of the beta program: to get that valuable feedback from driver-partners all over the world and turn it into a set of actions the product team could take to improve the app.
“Given that a Driver app redesign was a huge deal and would impact over 3 million driver-partners, we wanted to ensure that we accounted for cultural and geographical differences, making the app global yet locally relevant.”
Molly: Once the redesign started, we knew we’d need live user feedback — from the beginning of the process through to the eventual launch. Ensuring that driver-partners would be able to continue earning on the platform and find the new system intuitive and easy was critical to the success of the project.
Minal: One thing we were aware of was the fact that cultural differences among driver-partners across the world might impact how they perceived and interacted with our products. Given that a Driver app redesign was a huge deal and would impact over 3 million driver-partners, we wanted to ensure that we accounted for cultural and geographical differences, making the app global yet locally relevant.
Edu: Our challenge was also to create the next generation of Uber experiences by bringing our partners and internal teams — over 1,000+ people) — together to build something that worked everywhere, without causing a major disruption. We flew dozens of Uber employees from HQ to different parts of the world and we involved senior leaders from the local markets in order to align on the right experience and support.
Sas: At a very high level, the challenge we were given was: how do we roll out an essential experience like the new Driver app globally? How do we build a product from San Francisco that will work in Jakarta from day one? Some prior launches at Uber had been slower in reaching 100% roll-out status globally; we wanted to break that precedent. The beta we describe in this paper was our way of replicating how close we were to that initial research ask in a more contained way.
What research method(s) did you use in this effort? Why was this the best approach?
Minal: We wanted to understand how the driver experience evolved in the first few weeks of using the new app. Since global perspective was key, we embedded a researcher in each of the seven beta cities for a three-week period starting the day of the beta release. Every day, each of these researchers used a variety of methods to collect feedback from drivers. This brought us a continual stream of data from across the seven cities as drivers began to use the new app.
Bjorn: Because the new app was a top-to-bottom redesign, we wanted to be as open as possible to any kind of feedback beta drivers might have. We used a set of pretty open, non-prescriptive research methods we thought would together collect a diverse set of feedback. We visited beta drivers’ homes to get their thoughts in a familiar, comfortable context. We rode along with beta drivers to understand how well the new app was working for them in a real context. We organized regular lunches with groups of drivers to get feedback from multiple, diverse perspectives. Each of us even kept up one-on-one text conversations with beta drivers over the course of the beta to get every last bit of feedback they had.
Edu: This program embraced flexibility on all fronts. Instead of executing on a research interview script in each market, local researchers had the ability to adapt the methods and number of sessions to local realities to learn what mattered the most to driver-partners. This high degree of agency increased our motivation to go out in the field and decide how long we need to spend with each driver-partner and which snaps were more interesting to capture in a video with professional videographers.
“We really got to know the driver-partners and their families. It was a lot of fun.”
Molly: As Bjorn and Edu described, we needed to be precise about some specific methods and data gathering, while also understanding that the contexts for the launch would vary by location. We used a mix of standard research methods to gather data for the product development, as well as other methods to gather the story of the launch to share with the teams in SF and others around the world.
Simultaneously collecting feedback from hundreds of drivers from all around the world is no easy task. Why is it worth it to do research like this?
Edu: Let me give you a bit of context to put things in perspective. Traditionally, Silicon Valley-based companies design and launch new products in the US first, and then expand to international markets. In this project, Uber took the risk of reversing this order. We developed the product based on feedback from seven global markets, then launched the beta in those markets to live-test it with actual drivers. This research approach was the best way to ensure that the new Driver app would represent the diversity of the drivers and cities it would need to serve. We aimed to capture the cultural nuances of how drivers would learn and interact with the new product in different socio-cultural contexts (languages, currencies, payment methods, etc.)
Molly: Transportation is very culturally defined, so we were as locally targeted as possible in our data gathering. If you have ever been in a car in Los Angeles, and/or London and/or Cairo, you know that the experiences of traffic, navigation, and local norms are wildly different. Ensuring that our research covered these basics was incredibly important to the ultimate success of the project.
Minal: Along with bringing out critical regional differences, this simultaneity helped us learn from each other and build on each other’s learnings as we progressed through the three weeks.
“Even one-and-a-half years later, I hear how important this was to, for example, some of our engineers, because it’s not something they do very often.”
Sas: We did not just do this research ourselves. We enabled cross-functional teams to go out to different cities around the world and spend time with people who’d be using what they built. By bringing them face-to-face with real people, we achieved an identification with driver-partners that is very hard to replicate in a lab setup. Even one-and-a-half years later, I hear how important this was to, for example, some of our engineers, because it’s not something they do very often.
The paper mentions that you wanted to close the “time-space-cognitive distance” between beta driver-partners and the product team. What does this mean, and why was it so important?
“We believed that would create more empathy, provide more agency and hence make the team more invested in the learnings from research. And voila! That’s what happened!”
Minal: Uber is available in hundreds of cities. We are very cognizant of the physical, temporal, and cognitive distances that exist between many of our users and the majority of the product development team, based in HQ. We did not want this research to be like a traditional research project: researchers conducting studies in a remote market with a small team, then communicating findings back to the larger team in a couple of weeks. We wanted the team to feel closer to their users in all possible aspects. We wanted them to be able to talk to and hear from the drivers directly — after all, that’s who they were building for! We believed that would create more empathy, provide more agency and hence make the team more invested in the learnings from research. And voila! That’s what happened!
Bjorn: Exactly. There’s a huge difference between stakeholders reading your research findings in a report at their desks and being in the field with you seeing the data first-hand. When they’re out there with you talking to real users and hearing feedback directly, they internalize and act on the findings in a different way. We wanted to create that effect, despite not being able to bring hundreds of product team members from all over the globe into each research activity. That’s why we came up with the idea of using a Google+ community as a main research deliverable: a central place where we posted our findings daily from each activity, from each of the different beta cities. Stakeholders would be able to see a Bangalore or Jakarta beta driver-partner feedback not days later, but minutes or hours after we did a ride-along. Using Google+ as this sort of real-time clearing house really caught on with our teams.
In the paper, you write that Google+ community “went viral” inside the company. What about it made the research such a hit with the product teams?
“We had participation from designers and engineers and content strategists focused on building specific screens, all the way up to organization directors and one or two company VPs.”
Bjorn: [Laughs] Yeah, it did go viral. Over the course of the beta research, we had over 600 employees join the Google+ community board. We had participation from designers and engineers and content strategists focused on building specific screens, all the way up to organization directors and one or two company VPs. After the first few days, it just took off. We had a ton of team members reading, commenting, adding collaborators into threads, troubleshooting issues and asking for more detail, or asking us to check back with the driver-partner to see if the code change they just shipped fixed their issue. It was kind of surreal — I think most UX researchers would love having this much attention on their research findings. It was really invigorating, personally. It made me want to work that much harder the next day to get more quality feedback from driver-partners to bring back to such an engaged set of stakeholders.
Edu: We focused on posting very visual, human and small pieces of feedback soon after we got them. Stakeholders loved the bite-sized, intimate, immediate feedback, and their ability to act on it. It was energizing to see that everyone was checking this Google+ site daily to engage with the findings and problem-solve them within their own domain. A fun fact is that since we’ve shared this social media deliverable with researchers at other companies, some have adopted a similar process to make research more accessible and digestible. We need to keep challenging our research practice! These days, I am experimenting a lot with remote research methods to replicate this on a smaller scale
Minal: “The real-time-ness” of the learnings coming through the G+ posts, made stakeholders feel closer to the drivers across the world. They felt like they were with us on the field and could talk to/hear from the driver-partners themselves. THAT, I feel, made this research viral!
What did this research teach you? What can other researchers (and non-researchers) at Uber and elsewhere take away from this?
Edu: I learned a bunch. I learned how trusting other researchers to do what is most applicable in their specific context results in increased motivation and quality of findings. I have replicated smaller versions of this in my most recent projects and it works really well. What researcher doesn’t want to have a say in how they do their research? I also experienced a success story of bringing stakeholders along for the research and communicating the value of research across the Org. Finally, I discovered that we could create touchpoints to learn from each other — regardless of where in the world we are — and iterate our plans quickly, even during a three-week-long research sprint.
Molly: We also solidified our belief that we need to be conducting research in markets with target users as early as possible. Additionally, we found that for expert users like our driver-partners, they can be fantastic and passionate beta partners. Finally, we saw that Google+ and similar platforms can be great places to create real-time dialog around research findings.
“Social media probably wasn’t designed with UX research as a key use in mind, but it turned out to be uniquely great at creating engagement with our research — something researchers always have to think about.”
Bjorn: Molly’s right — one of my main takeaways was to try out non-traditional research deliverables more! I was shocked at how much the Google+ took off once we started posting findings to it. In retrospect, of course it makes sense: social media platforms are explicitly designed to create high, sustained engagement with regularly updated posted material. They’re public spaces that provide many distinct ways to engage with posted material. They encourage adding others into conversations, which becomes self-reinforcing. Even basic social media features like emails and mobile notifications when new activity is posted can tie stakeholder attention to your findings, bringing them back again and again. Social media probably wasn’t designed with UX research as a key use in mind, but it turned out to be uniquely great at creating engagement with our research — something researchers always have to think about. This project was a great reminder to venture outside the typical research report more, and to try new things.
Sas: My biggest take away from this research was how we needed to move away from typical research share-outs. Who has the time to put together a deck when people are going live on a brand new app for the very first time? As a researcher, I was as excited about the opportunity to innovate on our form of delivery as I was about the feedback. I really wish I could recreate the energy and anticipation of the beta in all the work we do, every week. One big takeaway: no matter how brilliant your research may be, it’s not very useful if it remains unseen. it is worth innovating to bring your research alive in a world that is moving so fast and where people have so little attention. We found a great way to share our findings, and in doing so we made sure those findings had as great an impact as possible.
Thank you Bjorn Hubert-Wallander, Eduardo Gomez Ruiz, Minal Jain, Molly Stevens, Saswati Saha Mitra, and Laura Garcia Barrio for taking the time to share this interesting research with us. For the readers who are interested in learning more please read the paper! We are excited to hear your feedback. If you have any questions please comment below.
If you are interested in joining our team, please check out the current job listings.
