Meet the Airbnb Research org: A team leading the rise of research
A Q&A with Airbnb’s Mandy Owen on championing and scaling research both inside and outside the building
Research has been baked into Airbnb’s design process from the company’s inception.
From the early days of 2009, when Founders Brian Chesky and Joe Gebbia flew to New York to meet hosts face-to-face to discover pain points, to today, Airbnb has proven to be a company propelled by insights. And as the organization has grown, so too has its research practice — from one that was at first foundational, to becoming multi-disciplinary and global.
Today, the Airbnb research organization is one of the largest and fastest growing in tech — one just shy of 100 practitioners. This talented team of passionate people, acts as a bright light in the international, budding research community; the north star for research done right.
On June 6th, Airbnb’s Nanako Era will be speaking in the Research Foundations Track and Shruti Kataria will hit the Mainstage on June 7th. As presenting sponsors of the 2019 UX Research Conference, we wanted to learn more about Airbnb’s internal research processes and how they are working hard to elevate other research organizations around the world.
I had a chat with Airbnb’s Product Specialist Team Manager (Research), Mandy Owen, who gave me the scoop on Research ops at Airbnb.
Tell me about your role at Airbnb.
I manage the product specialist org at Airbnb, a function made of two distinct teams:
- One program manages an outsourced workforce of over 40 people who sit in all of our community support contact centres across the globe and focus not only on data quality of insights, but other research-related projects that benefit multiple teams within Airbnb.
- The other is an embedded team of product specialists who work side-by-side with folks like experience researchers, product managers, engineers, designers, content strategists and data scientists. They help their teams make smart decisions on how to fold quality focused feedback and platform quality issues like technical bugs, into product development.
How is the research organization at Airbnb structured?
The Airbnb Research org is made up of 5 distinct functions:
- Experience Research
- Product Specialists
- Survey Science
- Research Operations
- Marketing Research
The product specialists, experience researchers, and some survey scientists and research ops folks are all embedded within product teams.
The product specialist program, other survey scientists and the research ops program that we call the “International Community Panel” are all centralized functions, that allow us to do things like conduct international research, quickly, without having to fly around the globe.
Because the majority of our team is embedded, most of that work is focused in San Francisco — but there are product teams that are outside of San Francisco, too. We have a presence, for example, in Portland, Seattle, Beijing, and Los Angeles. So generally speaking, where there is a product team, we want researchers to be there, too. And then with some of our other functions (specifically the program management team that I manage) — we actually have folks in Portland, Dublin and Singapore.
How do you think the company culture is affected by the research org?
The foundation of research at Airbnb has sort of been baked into our culture for a long time.
In our early days, our founders received advice from Paul Graham ( known for co-founding Y Combinator and Hacker News), who told them to go to our users — and so they actually literally did that. They were visiting our hosts in person basically, week after week, until it wasn’t scalable anymore. And so I think a lot of that kind of philosophy was baked in early on.
More recently with research, the way this philosophy shows up is in examples like this: my team has played a role in creating a culture of quality across many teams and I would say more broadly, research drives understanding and empathy with evidence on this long, large scale.
So it looks like influence on individual projects, but it also looks like cross business share-outs in things like “ShAir Fairs” (we like “air” puns!) and then a recent expo that we did called “The World’s Fair” to showcase international research. What these do is fold together insights from all 6 of those functions I mentioned and is really an opportunity for us to triangulate and show that work as one research org as opposed to individual research projects.
How does mentorship among researchers work in the research org?
Mentorship comes in all forms in our organization from formal to less formal.
On the formal end, we hold cross functional skill shares and we have critique sessions (we call them “Critz”). Critz basically provide an open, safe forum for researchers to collaboratively kick the tires of each other’s work and it’s also a space to share best practices on strategies that have been particularly useful.
There is also a broader mentorship programs across our entire design organization that encourages mentors and mentees to connect — not only to go deeper on a particular work problem, but to also talk more broadly about growth and development within the function or beyond the function.
We also create space for researchers to share and showcase their work, and we focus on connecting researchers who might not otherwise get to work with each other. One thing that is a cultural touchpoint for us is that we actually ring a cowbell whenever potential connections are made. It’s really a moment to highlight “oh — you’re doing work over here, this person is also doing work in a similar space, you should probably think about how to collaborate with each other.”
I think a lot of mentorship also happens organically, where you start to kind of look around and you see folks who are doing things that can really help your own growth and development.
I mostly participate in these programs as a mentor which has been really, really rewarding to work with folks across research, across design, across teams even outside of our org.
How do you measure the impact of your research work?
To be completely honest, directly measuring impact is often a challenge for us because measurement is often expected to be mapped back to a specific design or strategy — we don’t write code or ship pixels, which means we have to think about measuring impact differently than an engineer or a designer might. Here’s how we think about it.
In a lot of these cases, our influence is not always a direct one-to-one relationship with an outcome, which makes tracking that impact a little tricky. So one measure of success is the quality of relationships that we have.
Another measure of that success is how indispensable we have become, both within a team and as a function, to the point that a product team considers themselves incomplete without adequate research representation. This means we’re doing our job. And to that end, we see growth as a sign of that impact too. Every new researcher on a team is someone who could have been an engineer, they could have been a data scientist, a designer, PM etc. The fact that a team chose to invest in say, a product specialist, for example, tells me that my team is having an impact.
What are you most proud of when it comes to your research work at Airbnb?
Our org’s work has, and continues to be, the foundation of front line obsessed product development at Airbnb. A company like ours must continuously balance growth objectives with quality focused initiatives so we don’t leave our community behind. Research has been the backbone of this kind of balance from our early days and continues to be the beacon of that. So through several initiatives with lots of cross functional stakeholders, my team has been able to drive cross company visibility into platform quality and we continue to be the engine for understanding the kind of pain points in our product that drive our community to reach out and contact us. I’m really proud that my team is able to provide that partnership and insight within every team we sit on.
Do you see a future with UXR’s in leadership roles? If so, how do we grow research practitioners into organizational leaders?
Yes, absolutely. I think researchers understand how to plug in at every stage of development, meaning we often have a horizontal view that gives us a strategic edge. So not just with respect to the depth of our knowledge in a particular area or vertical, but because our discipline focuses on cross research collaboration. So it means we can continue to align and lift each other’s work and I think that positions the discipline really, really well to take on roles as business or organizational leaders.
How do you feel Airbnb has contributed to the growth of UX Research outside of the company itself?
We are all about connecting with other researchers and research orgs outside of Airbnb so there is a lot packed in this answer! I also think we want to do it so much that we also recognize that we haven’t done as much as we want.
Our field and this community of practice is basically at an inflection point — similar to what design experienced about 10 years ago in tech. We want to help facilitate that growth in a way that benefits researchers of all types in terms of positioning them for influence within their orgs and their teams.
So one example that comes to mind, and one that I am really excited about, is that we have a burgeoning research operations team that’s become the backbone of a lot of what we do. At Airbnb, we realize that to effectively grow our research discipline, your research ops team has to scale with you, to effectively keep pace with the needs of the team. And we want to make sure that rather than being reactive, we are proactive, which allows researchers to focus more on what they do. So what has been really cool in the past year or so, is that the research ops team here has hosted other research ops teams and companies who are considering creating an operational arm within their research org. We compare notes on methodologies and also on the org design itself.
We have also had the pleasure of working with Kate Towsey in terms of building an international research ops community. We are so pumped and very excited to see her research ops workshop at Strive this year.
One sort of reflection I’ve had with some tech companies is that research ops can be a bit of an afterthought. It’s a little bit like your research team grows and scales and then you realize “uh oh, we have a lot of work that we have to do” in terms of participant recruiting or these programs that you’re standing up.
I’m proud that Airbnb kind of saw that before it was too late. We are trying to basically focus on operations as a really important discipline within research so that we can grow it alongside the rest of our disciplines in a way that is strategic and not just reactive to excessive growth.
Will you be attending the conference? If so, what are you most excited for for the Conference in June?
I would be totally lying if I said I wasn’t excited about talks from fellow Airbnb researchers Shruti and Nanako. Any excuse I have to learn from my teammates and to see their craft is always an exciting thing. We have a really impressive, large team, so I don’t always get exposure to their work. So selfishly, I’m very excited about that.
Personally, I have become increasingly interested in the intersection of the public and private sectors in the US and specifically the large disparities and the capabilities of each. I’m really curious about how tech can be leveraged to help solve large scale social issues, so I’m very interested in Mithula Naik’s talk “How to Serve a Nation.” I’m really interested in hearing more about Canada’s approach.
Join Airbnb at Strive: The 2019 UX Research Conference
Tickets are still available for the Main Stage on June 7th and the Research Foundations & Advancing Your Practice tracks on June 6th! Purchase tickets here.
📅 June 6–7
📍 Roy Thomson Hall, 60 Simcoe St, Toronto, ON, M5J 2H5
