Show Me Music
Can we browse music visually?
In this day and age of constant scrolling through apps to find something worthy of a glance, why should browsing music be any different? No, I know about the creative album art and million-dollar music videos that big music labels dish out to attract the audience. But seriously, if you are like me, wouldn’t you wish that audio recommendations on streaming apps were as good as Instagram’s algorithm for discovering new content? Well, Instagram has an advantage when it comes to browsing posts. It only takes a second to stare at even ten pictures at once. Audio is different. You can’t possibly decide if you like a song in a second. No one simply scrolls through a new album and says this song looks interesting. What if I say why not? For what it’s worth, that is the idea that got me started on this project.
Approach
What should we visualize? The first step in the process was to figure out the features of a track that can be visualized. Contemporary music, like my taste, is dynamic and constantly reinventing itself. Music being composed today often either creates a new category for itself or exists as a tasteful amalgamation of the existing ones. So, conventional labels like genre, sub-genre, region, language, and artist don’t seem helpful. I evaluated the time and frequency domains of audio signals with the help of Waveforms and spectrograms. However, these methods are often specific to only one property of the audio signal and involve complex transformations. While going through Spotify’s extensive API for developers, I found the following attributes that I have used in my analysis.
- Acousticness: Confidence measure representing whether the track is acoustic. 1.0 indicates high confidence the track is acoustic.
- Danceability: Combination of musical elements including tempo, rhythm stability, beat strength, and overall regularity. A value of 0.0 is the least danceable and 1.0 is the most danceable.
- Energy: Perceptual measure from 0.0 to 1.0 that measures intensity and activity.
- Loudness: The overall loudness of a track in decibels (dB).
- Tempo: the estimated tempo of a track in beats per minute (BPM).
- Valence: A measure from 0.0 to 1.0 describing the musical positiveness conveyed by a track.
These features are available for any song available for streaming and are not specific to a certain category of music. Further, together these attributes cover all key aspects of music composition.
Concept
The next step is identifying a visual idiom that encompasses our features in a way that is easy to understand and draws inferences. The visualization should primarily serve two purposes at a glance. Represent all features in a simple and perceivable way. Enable cognition for effective comparison between the visualizations to aid in suitable choices or recommendations.
While the radial plot is visually appealing, it is also difficult to interpret. On the other hand, the bar graph is easy to read and compare and even reminds of an equalizer commonly used with music.
Implementation
To keep up with personalized music trends like Spotify Wrapper Up and top charts, I wanted to analyze how a trending song would compare against my top songs in 2022. To keep my analysis open and accessible, I have developed my analysis in a Python notebook that is reusable and can be improved. During the evaluation of my concept, with the help of feedback, I identified potential bias in my approach. A playlist selected for analysis could have different types of music. Consider a slow and moving vocal track that I am interested in comparing with the entire playlist with many fast numbers and a few other songs. The visuals would fail to point out potential similarities due to this bias.
Solution and Results
To provide an accurate comparison between the track and playlist while gaining insights from the selected playlist, I used an unsupervised machine learning algorithm called K Means Clustering. The algorithm classifies similar songs from the playlist into groups based on the features. Then, we would use the features of this group to compare with a given track for our recommendation analysis. Doing so eliminates the possibility of generating misleading visualizations for analysis.
To learn more about the identified groups I have visualized them:
Finally, the comparison of the track with the playlist:
Ah! no wonder I have been obsessed with this song that I came across recently at the gym. So positive and energetic.
Interpretation
From the above plots, I learned that in 2022, I have leaned towards loud music ( A feature consistent among all groups). Does this indicate a general shift in music towards a louder spectrum? Another trend in my taste is danceability. For someone like me who is shy and doesn’t dance, isn’t that funny? Finally, Track Group 4, the minority seems to be about a category of slow vocal works. This is a direction for me to explore when I am looking for new music to try.
Conclusion
Based on my experience conducting the above analysis, I’m confident about the potential of adapting a visual representation for audio that aids discovery and insights. Although smart recommendations using AI are getting better with time, they are a black box to the end user leading to a lack of confidence and unreliability. My approach benefits from transparency by leveraging human cognitive abilities in the adjudication process and helping gain insights while improving recommendations. In the future, I would like to explore other features to increase the accuracy of the clustering model and visualization methods. There is also room for improvement with an increase in user adaption and evaluation based on user feedback. With improvements in graphical compute on mobile devices to accommodate AR/VR, streaming apps can implement personalized analytics seamlessly in real-time using on-device machine learning. This would greatly improve the listening experience for users in the future.
Finally, I would like to thank Prof. Dr. Niklas Elmqvist and my classmates in INST 760: Data Visualization at the University of Maryland for their feedback and support during the development of my project.
Demo
The above link can be used to view the notebook and run the analysis yourself.
