Hint® Water Taste Test: How We Research @ SumoLogic
It all started with a question I asked during lunch:
“Can you guys really tell the difference between Coke and Pepsi?”
Whenever, wherever you ask this question on whatever occasion, chances are you will immediately get those obnoxious “how dare you” faces in return, followed by confident judgments on the level of sweetness, fizziness, after-taste (i.e. a hint of vanilla versus citrus), etc.
In reality, multiple blind taste tests have been done in the past 20 years in scientifically controlled environments, only to prove that most people can hardly discern any difference between Coke and Pepsi. In fact, Pepsi is only a little bit higher in sugar content whereas Coke is a bit higher in sodium.
Let’s take a step back. The more reasonable explanation is something tricked your brain: the commercials, the marketing, the corporate philanthropy, or a combination of all. Granted, there are super tasters who can tell even the manufacturing date of a can of Coke. But for the rest of us, we gained this knowledge about cola with the influence of branding efforts, and we overestimated our ability to predict the difference between the two that most likely couldn’t be predicted.
It’s fascinating how human brains process information differently depending on whether or not we have context. We’ve heard about the top-down and bottom-up cognition theories in every Psychology 101 class. I like the explanation UX Daily gives in their blog Top-down and Bottom-up Information Processing:
“Top-down processing is the development of inferences through the use of contextual information, past experience and prior knowledge related to a stimulus. However, the psychologist Eleanor J. Gibson argued that perception starts at the sensory input, the stimulus; it is not subject to hypotheses; rather, perception is a direct What-you-see-is-what-you-get phenomenon.” (The UX Daily, 2015)
In short, there are two main cognitive processing strategies. A top-down processing approach is a contextually driven process in which your cognition forms with the influence of existing knowledge (e.g. the branding of coke). On the contrary, bottom-up processing suggests that perception begins to form with basic senses (sight, smell or taste) directly from the object. It’s like information processing with a blindfold, without external interferences.
Another noteworthy proof of this theory, which happens to be the story behind this blog’s title, is contributed by the Sumo UX team. This adventurous group of people set out to combat the top-down and bottom-up theories with a methodical research approach — a hint® water blind taste test. The research question is simple:
How many flavors of hint® water can you distinguish without seeing packaging?
Before getting into the details, it’s necessary to disclose the reality of our daily consumption of it: the Sumo UX team absolutely LOVES hint® water! We drink these sugar-free, fruit infused bottles of water everyday. We all have our favorite/least favorite flavor(s) and we’ve raved about which flavor is the most real, how the pineapple tastes the most refreshing, and how the strawberry-kiwi tastes just like any other artificial strawberry-flavored stuff. So after the Pepsi vs. Coke discussion, it would only be fair for us to do a hint® water blind taste test as the most faithful believers. It was a very spontaneous decision and everybody was hyped about it.
On the game day, I started off with randomizing 7 flavors of hint® water and marking each flavor with a number. Then I removed all the distinguishing labeling around the bottles. Also to remind everybody what flavor options they were going to taste, I listed all 7 flavors(randomized) on the whiteboard.
The seven numbers I assigned previously represented the flavor being tested for that round, resulting in a total of 7 rounds. Here’s how we did it:
- Round 1, you’ll be offered approximately 4 oz. of hint® water from the bottle marked “1.”
- Drink it and write down your perceived flavor on a sheet of paper.
- Cleanse your palate in between tastings: drink some salt water? Apple juice? Eat a piece of cracker? Your choice.
- After that, you will start with the second bottle. One rule: each flavor can only be selected once - if the flavor in the current round tastes like something you chose for a previous round, you need to make your decision.
It was a tough game.
Clearly I was having the most fun during the test— observing all those confused faces and the sentiment of comments changing from “yes I’ve got this one” to “wait a second…I thought I tasted this flavor already.”
Final results?
Hint water without packaging crushed our dreams!
Who would’ve thought that the average success rate of the game was less than 40% despite that everybody walked into the test room with 200% confidence that they would nail it all?! To be fair, I was also drinking the water with them during the test and omg was it hard! Without the packaging, all the flavors tasted the same, but different, but same, from the very first round. We initially thought that it would be so easy to tell at least some flavors and in fact, Jason (product designer) drank the Pineapple one almost every single day prior to the study but still failed to get that one right. Even the Watermelon, the beloved sweetheart of all of ours, was guessed correctly by only one person.
The test later became a great lunch story to share at work. While thinking of it in the context of top-down and bottom-up processing, it all of a sudden made so much more sense: we thought all the flavors tasted so real because the colored packaging influenced our perception of taste! With all the packaging removed, our taste perception was isolated from the surrounding context, greatly inhibiting our ability to discern the artificial flavors from one another.
Think you know a problem, but do you really?
At Sumo, we thrive to incorporate both of these information processing models into our research-design life cycle. On one hand, a big chunk of the design iterations and usability research we do on a day-to-day basis is based on what we already know about a problem in our product. The top-down process helps streamlining the delivery of a feature and brings agility to product design, like a cookie-cutter framework. Quickly knowing user perception on a specific feature motivates us to constantly modify the product, and in return the improved products also keep driving users to dive deeper.
On the other hand, the Bottom-up approach helps us to research by isolating our users from external biases and assumptions, on an exploratory level, so that we can uncover problems we haven’t yet encountered and inspire new ideas of product design. For example, researchers on our team, including myself, are constantly brainstorming and implementing innovative ways to do generative research. We’ll sit with a DevOps Engineer and observe his/her troubleshooting process when he/she is on-call, without the context of using our product at all, just so we can closely empathize with our users and refresh our existing perception on their behavior in an untainted way.
Though the hint® water taste test was not the first time we realized that there’s so much more to learn about our customers, it definitely enlightened us about how many more ways we could learn. This is the most awesome part of doing research and design at Sumo— it’s a refreshing process everyday, while having fun.
However, despite all these serious conclusions, the blind taste test was mostly for fun. We admit.
And we were all so over-hydrated that day.
