Celebrating Spotify Wrapped SZN the Resultid Way
How Spotify uses mixed method research to test new features
The wait is finally over… it’s officially SPOTIFY-WRAPPED SEASON! 🥳💃 If you’re like me, you’ve spent tens of thousands of hours cultivating the perfect playlists, discovering new music, and dodging “Rain Sounds To Fall Asleep To” playlists to have a Spotify-wrapped story that knows you better than anyone else does. For those of you wondering, this year I learned that my music personality is ENVU (“The Adventurer.”) I also learned that I listen to rebellious, joyful music in the morning, bittersweet, comforting music at night, and way too much Taylor Swift 24/7.
The Spotify algorithm never ceases to amaze me when it comes to personalization. For those of you keeping up with our blog series, it’s no secret that I’m a big Spotify fan, (despite using our qualitative analysis app to find the Top 5 Complaints People Have About Spotify… awkward. 😳) But since I’ve had text data on the brain (interning for our aforementioned app, 👈) I started wondering what kind of qualitative analysis Spotify does to dominate the music streaming industry. After a quick google search, it turns out that everyone’s favorite little green app has a similar philosophy to Resultid when it comes to combining quantitative and qualitative data to find the “what and why” behind your results.
“The What-Why Framework” and Mixed Method Research at Spotify
According to Colette Kolenda, a past Product Insights Manager at Spotify, their team uses a “What-Why Framework” that combines quantitative analysis to identify overarching themes (the “what”) and qualitative to understand the context behind them (the “why.”) By exercising this framework, Spotify utilizes user behavior methodology, such as A/B testing and statistical modeling, and combines them with user interviews and surveys to truly understand the user listening experience, (sounds familiar? 👀)
Qualitative data often gets treated like the ugly stepsister to quantitative data. 🧌 Let’s face it: qualitative data isn’t as pretty as quantitative data. People like numbers to solve their problems; they feel more definitive and less subjective. But when numbers stand alone they can be just as vague — and Spotify happens to agree with us. 🤷Their team ran an A/B test to divide listeners using their “Spotify Free” feature into three groups: “power skippers”, “medium skippers”, and those who never skipped a single ad. Then, through user interviews, they learned why each group used the feature as they did.
To their surprise, they found that their A/B test and interview results were somewhat contradictory. At face value, you would assume that power skippers loved the feature and know how to use it. However, their user interviews revealed that many of these so-called power skippers had some confusion about which ads they could skip. Without open-ended qualitative responses that came from user interviews, Spotify would’ve missed out on this crucial piece of information and had a less complete story about their feature.
You can (and should) check out their three steps for mixing methods effectively through simultaneous triangulation. Then when you’re done reading that, let’s start brainstorming ways to incorporate qualitative data analysis so you can tell more complete stories with your data! After all, if Spotify can top the charts over competitors by mixing and mashing quant and qual methods together, so can your business. Plus, we’ve got the perfect app to make your text-data analysis easy, breezy, and painless. 😌
