The race to build a podcast recommendation engine and why it matters

Source: Pexels (Creative Commons)

Ever since Spotify plopped down over $400 million to buy three podcast companies, there’s been plenty of speculation about how it would leverage its recommendation algorithms to improve podcast discovery. After all, its music playlists — curated by both humans and algorithms — have driven songs onto the Billboard Top 100 list and turned previously-unknown artists into overnight stars. Now imagine applying these recommendation features to podcasts, a still-nascent medium for which discovery is still a challenge.

And sure enough, this past week we’ve learned that Spotify is testing out new designs that would feature podcasts more prominently within its app. The Verge’s Ashley Carman reported that Spotify was displaying podcast recommendations next to music on a personalized playlist called Your Morning Drive. And then a few days later, Bloomberg’s Lucas Shaw reported that Spotify was testing a version of the app that gave podcasts more prominent placement within the app’s libraries, making it so it’s easier to navigate to them. “We want you to get there in two clicks versus seven,” said Chief Financial Officer Barry McCarthy

Spotify isn’t the only app that’s forging ahead into personalized podcast recommendations. Pandora has spent over a year developing what it calls its Podcast Genome Project, a recommendation algorithm that uses language processing to push podcasts at the “individual episode level.” Engadget reported that the Podcast Genome Project “looks at hundreds of data points (including your skips and thumbs-up or thumbs-down ratings) to suggest shows and episodes. The more you listen to podcasts, the better Pandora will get to know your tastes, and the recommendations should align more tightly with your interests over time.”

The race to build a better podcast recommendation engine isn’t about introducing incremental improvements to podcast discovery. It’s about the next great leap forward in podcasting, another Serial moment that will introduce the medium to millions of new users.

Why do we need another Serial moment? After all, podcast listening has seen tremendous growth over the last few years, and 62 million people in the U.S. listen to at least one podcast per week, according to Edison Research. But podcast listening still represents only a tiny ratio of audio consumption. According to that same report, podcast listening accounts for 4 percent of listening in cars, and only 22 percent of Americans listen to podcasts weekly. At the same time, 51 percent of Americans say they’ve listened to at least one podcast, meaning tens of millions dabbled in the medium without turning it into a habit.

To understand why this gap exists, think about the typical way a new listener experiments with podcasts. They might be a fan of a hit radio show like This American Life or Fresh Air or Wait Wait…Don’t Tell Me! Or maybe a friend recommends to them a great indie podcast. That person then opens up a podcast app for the first time, downloads the podcast, and enjoys the experience.

But the next episode of that podcast doesn’t come out until next week, and while the listening experience was a pleasant one, it wasn’t mind blowing. Are they going to remember to open up the app again next week? If they haven’t yet formed a daily habit around listening to podcasts, the chances they make the effort to check back for new episodes are fairly low.

Unlike online videos, images, and text, podcasts don’t really go viral. And unlike with longform mediums like books, films, and television shows, there isn’t a group of critics dedicated to reviewing podcasts. In fact, I could probably count the number of full-time podcast critics on one hand. Podcast discovery is still driven largely by old-fashioned word of mouth; you have to either know other fervent podcast listeners or follow the right people online.

That’s not to say there aren’t some influential recommendation engines already in place. The Apple Podcast app, which reportedly accounts for 60 percent of all podcast listening, features a Browse tab. Click on it, and you’ll be exposed to the most popular episodes across dozens of categories, along with “New & Noteworthy” shows and “Staff Picks.” Evidence suggests that getting featured in the Browse app, depending on the placement, can drive a significant spike in downloads.

But these recommendations aren’t personalized, and they won’t do you much good if you’re not already regularly opening the Apple Podcast app. One major advantage for apps like Spotify and Pandora is that hundreds of millions of users already open them every day to listen to music; if music streaming apps can tastefully and unobtrusively introduce podcast episodes to these users, then their ability to drive podcast listening could be huge.

Building a great podcast recommendation engine won’t be easy, however. With music, you can typically listen to a song for less than 15 seconds before deciding whether you like it or want to skip to the next one. This allows machine learning algorithms to learn fairly quickly about your music listening habits. Deciding whether you like a podcast, on the other hand, can require several minutes of listening, and it wouldn’t take very many bad recommendations before you’d completely give up trusting the algorithm’s ability to predict your tastes.

This doesn’t mean that building a recommendation algorithm for longform content is impossible. In fact, Netflix has spent pretty much the entirety of its existence doing just that. Even before it became a streaming behemoth, it launched the Netflix Prize, a $1 million reward for whoever could improve its DVD-recommendation algorithm by 10 percent. According to Wired, “more than 80 percent of the TV shows people watch on Netflix are discovered through the platform’s recommendation system.” It even has dozens of employees who spend their work days watching its content so they can tag the shows and films into its thousands of “micro-genres.” Its algorithm not only tries to predict what shows you want to watch; it also will serve you up customized thumbnails that you’re more likely to click on.

Why go through all this trouble? Because Netflix knows the biggest threat to its continued dominance is a user running out of stuff to watch. “According to its own research, the company has a 90-second window to help subscribers find a TV show or movie before they give up and go somewhere else,” wrote BuzzFeed’s Nicole Nguyen.

Imagine if the average user only opened up Netflix when they knew exactly which show they wanted to watch. My guess is that the company would have far fewer than its current 148 million subscribers. And yet this describes the current state of podcast consumption; you only open up your podcast app to listen to shows you’re already subscribed to or to specifically search for a podcast by name. The future Spotify and Pandora are betting on is one in which podcast discovery is a more passive experience, where you stumble upon new podcast episodes the same way you encounter new songs. Only then will the medium truly be considered mainstream.

Simon Owens is a tech and media journalist living in Washington, DC. Follow him on Twitter, Facebook, or LinkedIn. Email him at simonowens@gmail.com. For a full bio, go here.