Reporting subscriber counts when crawling podcasts

Steve Agalloco
Breaker
Published in
2 min readSep 12, 2018

Over the past few months, we’ve been focusing a lot on helping podcasters grow their audience on Breaker.

We recently launched Breaker Upstream which allows podcasters to publish premium content, creating a new revenue channel to monetize their work. We also teamed up with Anchor to enable Listener Support on Breaker.

Today we’re rolling out an update to our feed crawler that will provide some of the subscriber data within Breaker back to podcasters.

Crawlers in the wild (PS: ours is much faster)

A few weeks ago, our friends at The Changelog asked podcast apps to report their subscriber count in the User-Agent when feeds are requested. Many podcasting apps have already jumped on board and we’re happy to be adding Breaker to the list.

Now, when we crawl a podcast feed, we’ll do so with a Breaker-specific User-Agent:

Breaker/v264 (subscribers=1690; feed-id=185589; url=https://www.breaker.audio/fresh-air)

If you’ve read The Changelog’s letter, you’ll notice that we’ve implemented this with a different format. We’ve formatted the count as a key-value pair: subscribers=536. We think this is an improvement because it’s more consistent and more importantly, we know that in most cases the User-Agent header is read by a machine, not by a person. Delimited values are much easier for machines to process and interpret. This structure also provides a logical way to add new values over time. We hope other podcasting apps out there agree that this is a better way forward.

This change prompted us to think about our crawling behavior and RSS in general and we’re working through some ideas that will bring even more improvements to podcasters and to Breaker users. We want to make it easier for podcasters to identify everything about their podcast, not just the content. This holds especially true for areas we feel are underserved at the moment such as tagging guests, identifying rebroadcasts, and linking to sponsors and advertisers. We’re excited to bring some of these ideas to light in the near future!

--

--

Steve Agalloco
Breaker

I’m a million different people from one day to the next. Currently: Engineering at @breaker 🌊 Previously: co-founder at @tradier