Agile Research Methods for Companies Big & Small
Old School Research
My first job as a product designer was at one of the largest software companies in the world, Microsoft. The company was in the process of elevating design within its corporate culture to compete with the likes of Google and Apple. They did so by transitioning from teams of solo designers working on individual products to a more holistic, design-first approach to their software products overall.
I was lucky to find myself as part of an early ideation team slated with sketching out a vision for what a new “cloud-based” Microsoft would look like. While Microsoft at the time was a series of individual tools like “Hotmail” and “Messenger” and “MSN”, we envisioned a people-centered brand where the software felt like a single destination to help you better connect with the world and the people dearest to you. For the company to get behind this, the needs, aspirations and painpoints of real people had to hold equal weight to business goals and the technical restraints of the time.
Fresh out of graduate school, I was paired with a well-traveled ethnographer and was sent to China on assignment. We set out to answer a slew of questions around cultural differences between China and the US: How are social relationships developed and maintained? How do notions of family differ? What are the cultural values of success and happiness and which local consumer brands reflect these values? We shadowed real people, with the help of a local recruiter, and we had a translator that took us through tenements, suburbs and college dorms of Shanghai and Beijing.
We got a glimpse of early social media at a “karaoke hotel” with college kids, had a traditional breakfast at the home of an ex-military commander, and hit the black market with a guy passionate about turtles, who had a collection of pets in his home that rivaled some of my favorite zoos back home. We learned how tight knit local micro-communities were formed in China’s biggest cities, the subtleties of family dynamics, and the importance of the concept of “Face” (preserving one’s sense of dignity in social contexts) in Chinese culture.
These insights had deep implications in how we implemented UI decisions for early social software products for localized markets. We also learned that aesthetic and design sensibility was quite different: for example, software applications with generous white space and clean design, an aspirational trait in the work of most designers I know, felt “empty” and valueless in certain applications to folks we spoke to (a couple of China’s Top 10 websites which demonstrate a high tolerance for clutter: http://www.sina.com.cn/, http://us.weibo.com/gb).
This was big budget research — multimillion dollar budgets spanning dozens of countries. I experienced just a slice of a year-long study that took place across continents, with the goal of creating products that would be loved by millions upon millions of people.
Time as the new restraint
While many people think of research as a massive, months-long endeavor for big-budget corporations, it doesn’t have to be. There are many examples of successful companies, from media giants like the New York Times to once-scrappy startups like Instagram, who owe great measures of success to using research to establish processes that allow them to stay connected to their customers regularly, as opposed to once or twice a year.
With perspectives like Eric Ries’ lean product methodologies (finding the shortest test and iterate loop possible to iterate on your product) having impact on the way products get built, time becomes the biggest enemy to traditional research methods. The old school research style of spending months on a single research endeavor every other quarter no longer applies.
“Today’s most successful companies, big and small, have built practices to stay connected with their customers regularly, not once or twice year.”
How to Pitch Research at a Startup
Fast-forward a decade from my big budget Microsoft days, and I find myself working with a startup in the food space called Kitchensurfing, a company that makes a personal chef an easy and affordable solution to everyday meals. Since the launch of our casual, everyday dinner program, Kitchensurfing has seen steady growth through organic word of mouth, not surprisingly, through the tech and food communities. But some big questions needed answers: Are the current customers the right customers for “hockey stick growth”? Or, put another way, for whom is this product a need more than an occasional want? Analytics show us how people are clicking and buying, but who are the ones that aren’t converting and what are they looking for that we’re not providing?
Though the company recently came out of a fundraising round and had resources to invest in growth, I needed to “sell” an investment in research to negate three popular misconceptions that could seem scary to any company, large or small:
- Research seems like it will slow things down
- Research seems expensive
- Research can become dated in a matter of weeks
I was able to build and evolve a research practice at Kitchensurfing by staying true to these principles:
1) Build to test: Rather than letting our larger research initiatives require that we pause on building product, I made sure that we were always pushing forward on product work in such a way that it supported our research endeavors and vice versa. Instead of dedicating special resources to creating assets for testing, I timed both the design and research programs so that we both building and learning. This way, we were both getting insight and putting valuable time into meaningful iterations of the product itself.
2) Involve your colleagues: In my first few months at Kitchensurfing, I noticed that we had an entire customer Service And Logistics Team (affectionately known as SALT) that held vast and valuable knowledge about our customers that were being underutilized in our product process. I built an alliance with SALT and product (known as PEPPER, get it?), involving and mentoring individuals (one in particular who happened to have a research background!) in UX research practices to extend our lean product team and also benefit more directly from their day-to-day interactions with customers.
3) Involve outside partners: While we did a lot of the heavy lifting on the strategy and synthesis in-house, some aspects of research can be extremely time consuming and there are folks that can affordably take some of that burden. Having run an agency myself for 4 years, I know that hired help can seem expensive. I also know that good agencies more often than not want to be helpful anyway they can, and boutique shops are more flexible than bigger ones in working with clients on what they need and nothing more. I found some great external partners that were willing to negotiate contracts that focused our research dollars on the areas that take the most time away from the team (ex. recruiting very specific samples in challenging zip codes, cleaning up large data sets, survey compliance).
4) Always be testing: The idea that you can do a single sprint of research, pat yourself on the back, and call it a day is simply not an option for a new product or service that’s trying to find its place in the world. Design is an iterative process, and the quicker we can learn, iterate and improve, the faster we’ll find the right customers and understand exactly what they do (and don’t) want.
4 Methods of Research You Can Do in a Snap
When I started working at Kitchensurfing, we had a lot of data about what our users were clicking on and buying, but we didn’t have as clear an idea about who they were or what was driving their decision-making.
To remedy this, a core team of about three of us implemented multiple iterations of each of the following research methods, while doing plenty of our other time-intensive work like actually designing, prototyping, building and supporting customers. Rather than one-off studies, we developed a cadence of launching these studies on the regular, building on our knowledge base and shifting focus with our business and product priorities when necessary.
Each of these methods can be completed in a couple of weeks, and some can be completed in a few days, start to finish.
1. Diary Studies
Diary studies are a great way to get ethnographic insight around customer behaviors. You can test how customers use your product, or you can simply get insight about what they do in their day-to-day to understand where your product fits in. Our first diary study was the latter, and it fueled both product prioritization and brand positioning around several marketing initiatives that were in development.
Since we were more interested in learning about potential customers than our current ones, we worked with an external partner to help recruit a very specific slice of Manhattan and screened for participants. We used a tool called Dscout, which is sort of an Instagram on steroids for design research, to collect over 250+ stories, photos and videos from individuals about what they eat, who they eat with, and what sorts of occasions present the biggest pinpoints and opportunities that our product can solve. Dscout even has built-in recruiting nationally in case you want to set up a simple screener and start talking to customers ASAP.
2. Concept Validation & Usability Studies
The goal of a usability study is to see if your product is intuitive in allowing people to use it for what it’s intended to do. A concept validation study, on the other hand, is one in which you introduce concepts to potential customers to see if you’re actually making the right product, if the overall experience resonates and you’ve defined the right set of features. Instead of focusing on UX, the insights we hope to glean here are: is the product providing real value? What’s missing? What’s extra and unnecessary, so we can focus our resources on the right problem?
For both usability and concept validation studies, there are many different tools and approaches but these are a few that especially powerful for lean research practice:
PROTOTYPING
User experience designers have long understood that you don’t have to invest in a fully baked, shipped product to get feedback from customers. What’s really exciting is that we’re in midst of a golden age of high quality tools for designers (and non-designers) to create quick and dirty or high polish product concepts to validate ideas and ensure a product is usable. We’ve used a spectrum from simple walkthroughs (using InVision) to more elaborate and interactive prototypes (using Pixate and Tumult Hype) for prototypes where we need a more immersive and engaging product concept to tell our story. The choice of fidelity is always based on the type of feedback we’re looking for, and sometimes, lo-fi is actually the way to go to focus on the offering and not the sexiness of a slick UI or animation.
FACE-TO-FACE STUDIES
If you don’t have an expensive one-way mirror research facility in your office, not to worry. There are lots of tools like Silverback that can help you record on-screen or on-device interactions as well as the participant’s commentary on your product, so you can share the studies with your larger team. The tool records the customers faces as well as the screens they are looking at. This information is invaluable — as the reactions of the customers is often as interesting as what they’re clicking on.
DIGITAL INTERCEPTS
Another great tool is Ethnio, which allows you to intercept customers while they are visiting your website. You simply set up a screener, generate a small snippet of code, and drop it into your site. When visitors come to your site, they fill out the screener and you get a notification — if you want to talk to them in the moment, you can share screens with them and jump on the phone immediately. Alternatively, they can use the software to schedule a more convenient time to speak with you. At Kitchensurfing, I set up what I called “Ethnio office hours”, in which we booked a couple of small conference rooms for a few hours each week and split up moderation and note-taking duties among the design team and the company at large. We had different insights we were looking to glean for each product sprint.
3. Focus Groups
This format of study is older than Mad Men, and involves getting a group together — sometimes diverse, sometimes highly focused in demographic — to provide feedback or insight in a group setting. Our quickest focus group was planned, designed, implemented and synthesized in less than a week.
To the participants, focus groups should feel very conversational, but in reality a lot goes into planning a structure that fosters a productive conversation. While focus groups can often be as simple as Q&A, I am a fan of mixing conversation with hands-on activities. For example, incorporating paper prototypes and printouts of product and brand work into the conversation gives participants a chance to write directly on the artifacts and discussing them as a group. Another common format that works well in a wide range of topics is “Day in the life” walk-throughs. In this activity, the group is given a calendar or a timeframe in their lives or their jobs, and they discuss their daily activities, needs and painpoints out loud as a group.
For example, we asked a segment of customers to describe what a week of dinners in their household looks like. This gave us a sense of how, say, Monday night meals are emotionally and functionally very different than Friday night dinners. To learn how we could enhance our in-home customer experience, we brought in a group of chefs to discuss what areas they find particularly challenging in the process of preparing meals, and captured their creative ideas on how we could create a better experience for customers and also help chefs excel at their work.
As an output, especially in an agile environment, I encourage our team to deliver both “Insights” and “Actions”. Insights form or refine our hypotheses and tell often quite emotional stories that inspire empathy and understanding of our customer’s needs, for both our internal team and external stakeholders such as investors. “Actions” are more tactical next steps — these provide the glue between what we learned and what we need to do next, and often involve sharing insights with multiple teams to discuss possible outcomes.
4. Surveys
Another (very) old school method is the survey (the first census was supposedly implemented in 3800 BC by the Babylonians). While there are plenty of methods for designing and implementing a survey, what’s new is the variety of methods for reaching out to folks, from phone calls to emails to SMS to in-product pop-ups. What’s also new is the variety of contexts the individuals can actually take surveys, most notably different being on a mobile device.
I’ve become a huge fan and advocate for Typeform, a product that has the philosophy that “one question at a time”, especially on a tiny screen, is psychologically easier process and actually quicker than dense Google Docs. We’ve used both at Kitchensurfing and the Typeform surveys have always had a higher rate of completion in mobile, and even more surprisingly, consistently take less time than the same Typeform on the desktop.
How you implement research will vary based on your organization — your goals, the stage of your company and product, your available time and resources, what you already know and of course what you’re trying to learn. There’s a spectrum of methods for gaining customer insight that can suit every timeline, and finding the right cadence is an iterative process in itself.
Beyond just making for a strong product, having a regular line directly to the customer and sharing those insights with the company at large can inspire a tremendous sense of empathy and purpose in companies. When things inevitably move at a lightning pace at companies big or small, it’s easy to lose sight of the “why” behind our every day work. Sometimes, all it takes is one powerful story about how your product changes the life of a customer to see what’s truly important about the work you’re doing.
I’ll be adding links to deeper dive articles on each of these methods to give a little more color on how we’ve implemented some of these methods in a lean environment and gotten great results, so stay tuned.
