Lessons from a Summer of UX Research
If you asked me in high school what it meant to be a researcher, I‘d say it had something to do with lab coats, microscopes, and five-syllable words. In college, I‘d expand the definition to include craftiness and making do with limited resources. This comes from overhearing graduate students in the lab talk about their all-Soylent diet as they sweat about research grants (“You can survive indefinitely off it, I swear”).
But these are superficial appearances. Postures, even. What really happens behind the scenes in the mind and life of a researcher in the tech industry?
I had the chance to find out over the past six months with my mentors, Bret and Aona. Here are some perspectives and lessons I have learned:
1. Look out for false dichotomies in research, and tackle unspoken assumptions head on.
Here’s a not-so-hypothetical question. When you’re greeting research participants, which approach is more effective for gleaning actionable insights for the product team: walking in with a furrowed brow and distant lab coat persona, or walking in with a warm smile and outstretched hand?
The unspoken assumption here is that you must choose between two options: be warm & inviting (giving the participant a positive experience at the expense of research rigor), or cold and serious (gaining research rigor at the expense of participant comfort).
But that is a false choice. In the spirit of this 15-second Old El Paso taco shell commercial — why don’t we have both?
While no research method or approach can cover all the bases, making sure the participant has a good experience during the session doesn’t have to come at the expense of good research. As a researcher, you can be warm, inviting, and conduct professional research without eliciting biased responses from the participant. The happy medium is when the participant feels safe and relaxed, so much so that they feel free to share their thought process (even if it’s messy or unpolished) while they’re engaging with the product. The conversation feels natural, even while the questions you ask as the researcher are structured and systematic.
That said, I’d tell my younger self to quit the giggling and raised eyebrows when the participant does something surprising. As a researcher, you want participants to feel relaxed and free to share their thoughts, not biased and focused on your own reactions.
2. Prepare, prepare, and then prepare some more. Researchers make it their job to know the history and limits of their product.
The same way adolescence compelled me to test the boundaries of authority figures and friends alike, you as the researcher must also test the limits of the prototypes and products you are pushing in front of customers.
Knowing the prototype so well you know where it breaks can give you peace of mind during research moderation sessions. Expect products that have been shipped to production to break, too — consider this the price of admission when your company is aggressively growing and taking risks with development, or when the system you’re working with is distributed.
Preparing includes knowing the history of the product, warts and all. What problem was it trying to solve? What were the technical decisions made in the past that haunt the product team today? Documentation can provide some clues, but the meat of the story is held within the memories and experiences of the people that work alongside you. Your colleagues can provide context (e.g., past conversations, juicy gossip) that comments in code and old research reports may not.
It helps to be creative in crafting your history lessons. Just joined the company and don’t know the history of the product? See if any of the staff engineers are up for story time over beers at the local pub, or downstairs around the refrigerator (the folks at Sumo Logic usually are, and are quite happy to chat over IPAs and giant German pretzels). Don’t drink? Go and play some board games with the engineers in the lunch room. Don’t play board games? Stop by their desk and reach out to them anyways. And then figure out some other interests you can share if you’re feeling intimidated in a new environment.
You are responsible for your own history lessons. You may even grow to enjoy playing detective (that is why you joined this field after all, right?), and you’ll learn more than you originally bargained for.
2b. Don’t expect everything — or even anything — to go according to plan. But plan anyway.
Consider it a rule of the universe that not everything is going to work according to (your) plan — especially when it comes to studying human behavior and emotions. Here’s a common scenario: less than three seconds after you ask a participant to speak about their first impressions and not to touch anything, they go ahead and touch it. Maybe they’re hard of hearing. Maybe they’re just too excited to play with the prototype.
A research session is almost like a dance, with some participants more attentive and graceful than others — and that’s okay. Perhaps you enjoy that uncertainty, and believe it keeps you honest and on your tiptoes. (That, or you’ll have to learn to love it, because it will keep happening). As a rule of thumb, expect double the unanticipated effects when you’re running a remote usability session, and double it again when you give the participant screen control.
Building redundancy into the workflow can be another form of preparation. This goes from multiple video recordings and data backups, to having a note taker join you in the research session so you can focus your attention on the participant and moderating. If you don’t have that luxury, even booking yourself fifteen minutes of wiggle room to decompress and debrief after each research session can make a huge difference in retaining valuable insights. Plus you’ll be recharged before the next research session.
3. Praise be to “cross-functional” teams.
Move over big data™, SaaS®, and end-to-end B2B solutions©. The buzzword I want to talk about today is the cross-functional organization.
Working in a “cross-functional organization” means you work with people outside of your core team, or outside the restriction of your job title. So as a researcher that means I get to interact with back-end & front-end engineering, sales, product managers, designers, customer success, and more. Each role and each person within these roles has a unique perspective to contribute. Instead of guessing what they would think or how they would behave, I can just walk up to them or schedule a meeting. It’s an open office after all.
Consider it your job to understand the motivations and goals of everybody involved at work. The sales team has to meet their quotas. Product managers have to make sure the product meets their criteria for success, or that they’re providing the quality of service their customers expect. Engineers have to make sure their next git commit doesn’t break the whole operation, and so on.
Being cross-functional extends beyond the conference rooms and public Slack channels (#random, #music, and #learn are my personal favorites). You might find yourself playing frisbee in the park with sales and operations teams or digging into board games with the staff engineers. A modern workplace with a playful atmosphere is not a daycare or summer camp like some articles may deride, but rather a public acknowledgment that you are encouraged to have interests outside of your job description. Yes, you are a professional. And you are a human being.
4. Start and end with curiosity — but don’t forget about utility.
Before Sumo Logic, I spent my time in an academic design lab. I loved it, and I loved having the space to “ask the bigger questions” and think five years into the future. But unlike academic research where you can remain relatively detached while scratching your chin, with industry research, the existence of the whole company is tied to the performance of the product (or more accurately: who’s willing to pay for it).
In other words, a prototype that works for only one or two use cases may suffice for an annual conference paper — and even delight the people who approved your research grant — but it’s not going to satisfy customers that pay millions of dollars for a subscription.
Of course, you could also look at this and say that there’s not really much of a difference between academia and industry. In both environments, you must keep an eye on funding and be creative with keeping research costs manageable (although that feeling is increased tenfold for academia). At the end of the day you’re trying to get buy-in from prospective customers who may or may not renew their annual subscriptions, the same way government funding agencies or private donors may give you another research grant before moving onto other projects they deem more exciting.
You can still ask the big questions in your field while in industry. You’ll just have more short-term checkpoints along the way, like four fiscal quarters instead of an annual conference or journal submission. Maybe the rapid-fire pacing is a blessing in disguise: prototypes won’t just remain prototypes forever.
5. Play the “clueless” card. Often.
Think of it as your task — no, your duty — to ask the simple questions. For Sumo Logic, that could mean asking the differences between a log and a metric (they’re not the same), or what to expect when clicking a menu button with three dots as opposed to three horizontal lines.
By choosing to conduct research, you are making at least two presuppositions:
- There is something out there in the world worth discovering that either hasn’t been discovered yet, or hasn’t been adopted in wide practice yet.
- You are humble enough to acknowledge that you don’t have all the answers, and that it’s unlikely that you ever will. But at least you’re going to give it your best shot and make as big of a dent in the realm of human knowledge as you can.
By the way, if you’re an intern or thinking about becoming an intern — whether at Sumo Logic or anywhere else — recognize that you are in an incredibly fortunate position for growth and learning. You’re expected (and even encouraged) to make rookie mistakes, and you’ll have nearly free license to bug your mentors and colleagues.
So I’m grateful for that life-long humility that effective research brings, and the knowledge that the questioning never stops, whether you’re a goofy intern or a seasoned veteran.
6. The people you work with in industry have an incredible wealth of outside experiences and backgrounds. Learn their story.
What brings people to research in the first place? I entered university thinking I would be a therapist (“I like psychology”). And before that, a journalist (“I like writing”). Maybe I stuck with research because I wanted the ability to observe people and be nosy for a living.
There’s a wide variety of skillsets in the office — just within the UX team, we have sculptors, welders, activists, and competitive video game players. Walk over to other departments, and you’ll encounter professional bodybuilders, bakers, scuba divers, and more.
If I could make only one recommendation in this entire article, it would be this: do everything you can to discover people’s stories! Research does not start and end with whatever usability session or prototype you push out in front of people.
People are not their products. Lose sight of that, and you’ll miss out on so many of the things that make life colorful and worth sticking around for.
Now go out and there and do some fantastic research!
If anything I wrote resonates with you, please reach out to me and the UX team! Our doors are always open, and I’m happy to share more about my time as a UX research intern at Sumo Logic. My email is wesleyschan [at] gmail.com, and you can reach out to the team through join-ux@sumologic.com. :-)
