Spotify’s Machine Learning Algorithms and Your Daily Mix.
Written by Stephen Denly
Today around the world, 180 million Spotify users will find 6 new refreshed playlists waiting for them as part of their Daily mix. If you are like me you will rely on these playlists to get you through the daily commute whether you are in the car or on the train!
Recommendation is a particularly tricky area for data scientists and firms. Get it wrong and the user will not be coming back!
Aside from consumers, this algorithmic approach to recommendation affords musicians and labels a new opportunity to connect with potential fans. Spotify’s daily mix offers an example of how machine learning (AI) can provide a unique experience to consumers and help artists release music to a global audience via algorithmic generated playlists. To illustrate the impact these playlists have, I will share my personal experience with the music I publish in my spare time under Point Oblivion.
In addition to the below I’d also highly recommend going and reading Sophia Ciocca’s piece on the Discovery weekly feature as it addresses some of the history when it comes to content recommendation engines for music.
The AI Technology behind the scenes
Spotify use clustering techniques to identify distinct sub-groupings within user listening patterns, from which recommendations are built. These clusterings come to represent your 6 most listened to genres.
A range of techniques across Natural Language processing, Deep learning and collaborative filtering are applied to curate what music appears in your Daily mix.
Architecture of data flow Discover weekly
From Galvanize’s blog — Spotify wants to be able to recommend new music to get around the cold start problem, it uses convolutional neural networks to analyse the songs themselves. This can be seen in practise by Sander Dieleman who wrote about his internship at Spotify in 2014 and shared some of his methodology and approaches during his time.
Computation for Deep learning & Machine learning
In Bennane’s article the network he implemented in Theano, and trained using minibatch gradient descent with Nesterov momentum on a NVIDIA GeForce GTX 780Ti GPU. Data loading and augmentation happened in a separate process, so while the GPU is training on a chunk of data, the next one can be loaded in parallel. About 750000 gradient updates are performed in total and it took between 18 and 36 hours to train.
Deep learning has come on a long way in the last 5 years with the advances of open source machine learning frameworks, availability of compute Power available in the cloud and in servers. One example of this advance would be the IBM AC922 which connects up to 6 NVIDIA GPU’s with NVLINK to improve data transfer and massively reduce the time it takes for organisations to iterate their AI models. (V100 as of the time writing this article). — *Disclosure I work for IBM*
Given Bennane’s example I imagine this could be completed in a fraction of the initial time using an enterprise deep learning solutions with NVIDIA’s latest GPU technology.
The Impact of AI algorithms for the Music Industry (musicians & Labels)
Personally, prior to Spotify I would lean on personal recommendations from friends for new artists and use YouTube, live shows and follow record labels to stay current with my music area I’d follow. Fast forward to today I find new music without the need to trawl the internet searching for that new artist — they often end up ‘in the mix’ on the Spotify platform.
Having Spotify’s algorithmic playlists do some of promotion work for you as an artist can’t be underestimated having an algorithm that focuses on music in some ways allows musicians to focus on quality rather than mass marketing. Some would argue this is still up for debate.
With this in mind I thought I would share my experience with Spotify’s metrics for algorithmic playlists for my music Point Oblivion. What do the algorithms do for a Musician?
Outside of my work at IBM I have been playing and writing music for a number of years and finally decided to release some material because why not! This gave rise to my personal project Point Oblivion a mix of metal & pop genre’s.
The Metrics from Spotify on Daily Mix and insights
The image below shows how artists/labels are notified of the metrics that drive new listeners and streams from these daily mixes.
What makes Spotify a dream to work with from the business side is the granularity for different insights over a variety of time windows (24 hours, 7 days, 28 days & since 2015). An extract below from my personal dashboard shows that 9% of my streams in the last month came from these algorithmic playlists, helping reach a global audience without the need to tour. These will ultimately (I hope) feed into the listeners personal playlists and the playlists of non Spotify curators to extend my reach to potential listeners.
Beyond listening insights, Spotify shares similar artists your audience also listens to. This is invaluable for (professional artists) to plan tours and understanding the cluster of music that an average listeners likes / subgroup.
As I eluded to in the stream insight section, the knock on impact of the Spotify algorithmic playlists Daily Mix and Discovery weekly is the content from artists is often then found curated in a user playlists that has a following for that genre or preference cluster.
See below an extract from my Spotify dashboard listing these user playlists.
The theme throughout has been reaching a global audience through machine learning algorithms which aim to know your tastes at a personal level. Being based in the United Kingdom I have never played a live show, nor do I spend much time promoting my music online. With Spotify’s algorithms this has helped reach a wider audience that would enjoy my music.
Conclusion
To summarise everyone has their unique tastes when it comes to music and Spotify’s Unique machine learning algorithms for daily mix collects 6 clusters of my most preferred genres and keeps me coming back for more, I haven’t seen a platform yet that rivals their recommendation engine. With further advances in AI, deep learning and the availability of NVIDIA GPU compute power these type of recommendation models will surely improve.
I would love to hear about your experience with Spotify and how machine learning could be further applied. All opinions and views are my own.
