The Story of Reginald: the tragic music recommendation hero, and the implications of new technologies
We’ve been waiting to answer the question everyone may or may not have had on their mind: who is Reginald, the Musical Mastermind?
The Origin Story
When setting out to create a music recommendation algorithm, it becomes increasingly important to consider the historical context around the state of music in general. Over the last twenty years, our society has watched the power of whose music is played, and when, has shifted from radio stations to large platforms like Spotify, Apple Music and YouTube.
Our group Neuralvana, as a group of self-professed music lovers, set out to create some new recommendation algorithm, as a response to our music tastes, and desire to understand how some algorithm like this is actually made.
The Recommendation Space
The primary force in the recommendation space is Spotify. While they have released a report about transparency in their products, it still remains mostly quiet about the origins of their algorithm’s development (and thus the biases it may have in its training data), and the actual effects that it has on the music space in general.
Putting aside the fact that Spotify does have interest in keeping the way they recommend music a more closely-guarded secret (as do its competitors), it still does raise ethical concerns about the way in which artists are discovered, and the effects that it can have on the space.
University of Oslo researcher Arnt Maasø, for instance, discusses many concerns about the nature of platforms like Spotify, and how they may or may not recommend music to users. He expresses, “these reinforcing feedback loops may have cumulative effects over time that in turn may impact the diversity of music culture writ large.” It’s no understatement to say that the way music is recommended can have wide-reaching effects on the way that people consume music. Therefore, it’s vital that, as we develop a new recommendation algorithm, we thoughtfully consider, as well, the ethical implications of amplifying some music while not others.
Musical diversity
Arguably the most notable implication of utilizing recommendation agents to suggest music from an industry standpoint is a risk of drastically decreasing the amount of musical diversity among users. Such a side effect effectively prevents both the user from discovering a wide variety of new music, and lesser known artists from having their works supported and streamed. Spotify themselves performed a study on the impact that their recommendation system had on the listening diversity of Premium users, computing organic stream similarity and programmed stream similarity, and noted a notable increase in diversity among organic streams not influenced by the recommendation system (The company even acknowledges that they explicitly account for musical diversity in their recommendation algorithms). While Spotify’s recommendation system has indeed achieved widespread approval and satisfaction, it is still prone to objectively reducing the listening diversity of its users, as are nearly all music recommendation systems to date.
In addition to hindering user listening diversity, another notable issue with any personalized, user based recommendation system of any kind (be it music, advertisements, videos, etc.) is that of personal data collection. For systems as straightforward as Reginald, (which solely takes into account music listening history and song metadata), this is obviously not an issue in the slightest. That is the exact data that users want to be parsed to allow the system to learn their preferences and bestow new music to them, and the user should have no issue with that data being collected! However, for major corporations such as Spotify, data collection becomes far more serious and invasive than just your listening history. In 2015, Spotify fell under fire for gaining access to and collecting (with user permission) “your photos, media files, GPS location, sensor data (like how fast you’re walking), and your contacts “ (Marr, 2015). CEO Daniel Ek went on to clarify the purpose of this data collection is to provide an even more accurate and customized Spotify experience, for example by utilizing GPS and sensor data in tandem with the app’s “running feature” in order to better account for your current workout. While the company provided users with an opt-in approach, requiring explicit permission to utilize this data, it should still raise various red flags that the company would have access to this data and the ability to market it off to advertisers, and raises the question of “how much input and data is really required for a music-recommendation algorithm?”.
Our algorithm
Here’s the basic stack for Reginald.
The database we used is called Music4All, a 2020 database that contains 100,000 songs and the listening history for 15,000 anonymized users. This was perfect for our project for a number of reasons: the primary one is that it was recent, and in a reasonably parsable CSV format.
We planned on using two main sources of data to contribute to the neural network that we eventually built: the number of times that each song appears in the listening history database, and the attributes of each song’s metadata. This would allow us to find “similar” songs, while also leveraging the amount of times that users have listened to them in such a database.
We used Keras, a Python library of TensorFlow, to implement a neural network for Reginald. This allowed us to train based upon our data without having to deal with too many of the technical details that can sometimes be associated with machine learning.
To implement our project with Keras, we created a neural network with 1 hidden layer with 20 nodes. Using sigmoid as our activation function and the mean squared error as our loss function we achieved an accuracy of 44.41%. The neural network was trained using 9 inputs including song popularity, release year, song duration, energy level, key, danceability, mode, valence, and tempo. The expected output was the amount of times the user listened to the song. Using this neural network we could predict the number of times the user will listen to certain songs.
Conclusion: expectations vs. reality
While we didn’t complete the goal we originally set out to do (recommend users new music), we were able to implement the prediction. And, as stated before, with the ethical implications of recommendation algorithms myriad and frequent, it was still a meaningful experience to be able to contend with the ethical implications of technology, while implementing one ourselves.
References
Anderson, Ashton, et al. “Algorithmic Effects on the Diversity of Consumption on Spotify.” Spotify Research, 3 Dec. 2020, research.atspotify.com/algorithmic-effects-on-the-diversity-of-consumption-on-spotify/.
He, X., Liao, L., Zhang, H., Nie, L., Hu, X., & Chua, T. (2017). Neural collaborative filtering. Proceedings of the 26th International Conference on World Wide Web. https://doi.org/10.1145/3038912.3052569
Maasø, A., & Hagen, A. N. (2019). Metrics and decision-making in music streaming. Popular Communication, 18(1), 18–31. https://doi.org/10.1080/15405702.2019.1701675
Marr, Bernard, and Follow. “Spotify’s Big Data Scandal: Outcry Against Intruding ‘Privacy’ Policy.” Spotify’s Big Data Scandal: Outcry Against Intruding “Privacy” Policy, 26 Aug. 2015, www.linkedin.com/pulse/spotifys-big-data-scandal-outcry-against-intruding-privacy-marr.
Mitchell, M., Wu, S., Zaldivar, A., Barnes, P., Vasserman, L., Hutchinson, B., Spitzer, E., Raji, I. D., & Gebru, T. (2019, January 29). Model Cards for Model Reporting. Proceedings of the Conference on Fairness, Accountability, and Transparency. FAT* ’19: Conference on Fairness, Accountability, and Transparency. https://doi.org/10.1145/3287560.3287596
Pedersen, R. R. (2020). Datafication and the push for ubiquitous listening in music streaming. MedieKultur: Journal of media and communication research, 36(69), 071–089. https://doi.org/10.7146/mediekultur.v36i69.121216
Pegoraro Santana, I. A., Pinhelli, F., Donini, J., Catharin, L., Mangolin, R. B., Da Costa, Y. M., Delisandra Feltrim, V., & Domingues, M. A. (2020). Music4All: A new music database and its applications. 2020 International Conference on Systems, Signals and Image Processing (IWSSIP), 1–6. https://doi.org/10.1109/iwssip48289.2020.9145170
Van den Oord, A., Dieleman, S., & Schrauwen, B. (2013). Deep content-based music recommendation. Advances in Neural Information Processing Systems, 26. https://biblio.ugent.be/publication/4324554/file/4324567.pdf
Kevin Doyle, Ian Herdt, Matt Levine, and Nick DeMarchis contributed to this article.
