The Froot Loops NPS Case Study (Part 3) Qual to the Rescue!

MIKE RYAN
MIKE RYAN
Jul 20, 2017 · 7 min read

< Part 2: Methodology and Quantitative Results

“I will not be ignored!”

Qualitative data will not be ignored

On June 8, 2017, I approached attendees of the UXPA 2017 conference in Toronto, Canada one at a time and asked if they would like to try Candian Birthday Cake Froot Loops (BCFLs). If they agreed I would have them eat a few and ask them the standard NPS question:

“How likely or unlikely would you be to recommend Birthday Cake Froot Loops to a friend or colleague? With 0 being not at all likely and 10 being extremely likely.”

The final NPS score based on 51 respondents was a -64. This is not a good score. That would be the extent of the findings if this were strictly a survey. But being a qualitative researcher, I was compelled to find out more. I wanted to know why they gave that score. If people did not want to try the BCFLs I wanted to find out why not. I observed and recorded facial expressions and behavior when they tasted the BCFLs. I also wanted to record their reactions to the NPS question itself. Ultimately the qualitative data delivered the most insights.

Photo and composition by Bob Couture

Reasons for refusal

Special diets

Of the 61 people I approached 10 refused to try the BCFLs. Half of these refusals were due to diet restrictions, mostly gluten intolerance. They all assumed gluten was present. I do not recall any checking the ingredients to confirm this.

Contempt for sugary cereals

The other half of the refusals were from people who hated sugary cereals due to their super sweet flavor and general unhealthiness. (They gave the best quotes.)

  • “I would rather die. Hearing ‘froot loop’ triggered the memory of the smell and texture.”
  • “Get those things the hell away from me.”
  • “Hell no!”

Dislike of BCFLS

Hate the taste, smell, and/or aftertaste

The most frequent comments by participants who gave the BCFLs an NPS score of 0–3 was that they did not like the taste, smell, and/or aftertaste. Many of these participants physically grimaced when they ate the cereal. Several complained that the flavor lingered and grabbed water to rinse it away.

  • “WHY? Tastes like chemicals. It doesn’t taste like food.”
  • “Oh man… Oh my god. That is vile.”
  • “That’s awful. Honestly these are obnoxious. It smells like a scented candle.”
  • “It tastes like I just threw up. F*ck.”
  • “Smell this.” (gives box to friend) “Does that smell bad?”
  • (a few minutes after eating) “I’ve still got the aftertaste… It’s still there.” (rinses out mouth)

Lack of health benefits

There were many comments around how unhealthy sugary cereals are. Many participants did not like the idea of anyone having birthday cake for breakfast.

  • “It’s sugar. I wouldn’t recommend. I won’t eat it.” (looks at ingredients) “Sugar.”
  • “I can’t imagine that there is any health benefit.”
  • “I wouldn’t recommend to children. It’s horrific.”

Bizarre concept

Birthday Cake Froot Loops cereal was created to celebrate Canada’s 150th anniversary. This whole concept was seen as strange. Birthday cake is not a fruit. This makes it an odd flavor for Froot Loops. Some commented that it didn’t taste like cake at all. Several participants snarked that the box said “prepared in Canada” instead of “made” or “cooked.” This added to the artificial “not food” quality.

  • “Isn’t it supposed to be fruit?”
  • “What does ‘Prepared in Canada’ mean?”
  • “I would prefer actual birthday cake. Not this.”
Say a prayer for anyone trying to decipher my notes. Including myself.

Positives

Stoners

The most frequent positive comment was that “stoners would like it.” Participants said that they would like to eat BCFLs when they were stoned and/or they would recommend them to friends who were stoners.

  • “Depends on the context of the cereal usage. Would eat this when I am stoned.”
  • “Disgusting. Would recommend to my stoner friends.”
  • “It’s a good stoner cereal. It tastes like it looks.”
  • “I don’t hate it. It’s like strawberry perfume… This would be good when you are stoned.”

Nostalgia, nihilism & nationalism

There were several nostalgic comments about loving sugary cereals as a child or wanting to eat them but not being allowed to. Some of the younger participants took nihilistic pleasure in how artificial and unhealthy the BCFLs are. Some of the Canadian participants liked BCFLs due to a sense of patriotism in celebrating Canada’s 150th birthday.

  • “I would make them into a froot loops necklace (but not eat the necklace).”
  • “Nobody likes fake cake flavor like I do.”
  • “It’s like cheap strawberry shortcake.” (Note: this was a positive for them.)
  • “Yes, because it’s Canada’s 150th anniversary.”

Doesn’t taste like soap

The most memorable comments were from a participant who was recovering from chemotherapy. The treatments were making most foods taste like soap to them. The audacious sweetness of BCFLs broke through to their delight.

“Smells like vanilla. I am recovering from chemo and everything tastes like soap. This tastes like sugar. I get the whole birthday cake thing.”

Recommendations

Based on the qualitative feedback we can offer a few recommendations to Kellogg.

  • Create a gluten-free option
  • Make it obvious to consumers when gluten is included
  • Think of ways to make the cereal healthier (or at least seem healthier) to better market to parents
  • Reduce the aftertaste
  • Appeal to childhood nostalgia

Dig deeper into the stoner demographic

The larger recommendation would be to do more research with stoners. I would recommend stoner interviews to explore the viability of marketing sugary cereal directly to them. A potential exercise would be to give them BCFLs in a new box concept of “anti-Wheaties” with Phish over a psychedelic background. This could also appeal to the nihilists.

I was not expecting that BCFLs would appeal to stoners. This is one of the biggest strengths of qualitative research: discovering something completely unexpected. This is much more difficult to do with surveys because you don’t know what you don’t know.

NPS question comments

In part 1 we identified a number of questions around NPS. Two of those questions were explored in the qualitative comments and observed behavior.

Is the 0 to 10 scale difficult for respondents to rate?

It took a little longer than a normal 5 or 7-point scale. I don’t think it was because of the scale though. I think it was because of the question.

Is the NPS question awkward and difficult to answer?

Yes. Most participants had difficulty answering the question because they were trying to define “friend or colleague.” There were a lot answers that said that they would or would not recommend BCFLs to a specific group. Many participants would recommend it to kids (but not their children). There is ambiguity in the NPS question as to whom we are talking about recommending to.

  • “10 if I was a kid.”
  • “I would recommend if it was for my 6 year old but not for my older daughters.”
  • “I wouldn’t recommend it to my wife.”

Some would try and qualify their answer by adding some extra details. This clouds the answers from its original intent.

  • “I would recommend if I knew they liked sugary cereal. It’s not for me.”
  • “Maybe I would recommend with milk.”

Then there is the questions of 1) how often an individual recommends things and 2) when it is appropriate. Many adults don’t recommend breakfast cereals.

  • “I don’t recommend cereals.”
  • “Why would I recommend cereal to a colleague?”

Conclusion

I am ambivalent about NPS. I would recommend it (whoa that is meta) if it is being used frequently in your organization. If it is a standard metric, it is good to use for comparative purposes. But I would always ask “why” for more insight.

I find that with most scale questions I learn more from the “why” than from the scale scores. In this example I gained one insight from NPS and a slew of them from the qualitative data.

If NPS is not being used at your org I would choose a set of 7-point scale questions which measure different aspects of customer/user satisfaction. Always gather qualitative data for deeper insights.


FYI BCFLs are not the only birthday cake flavored “food.”


Part 1: WTH is NPS?

Part 2: Methodology and Quantitative Results

Read more:

)

MIKE RYAN

Written by

MIKE RYAN

Principal User Researcher @ Liberty Mutual. Bentley MSHFID. Dad. Master of useless info: Film, Comics, Metal, Beer, Psychology & UX. I read too much.

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade