Deconstructing the Billboard Top 5
Using visualization to understand the billboard charts.
By Eyimofe Barrow and Meghan Lendhe
Music is important culturally, and it is always changing. Or is it? Research has been conducted to try and establish how exactly popular music has changed. One study at Lawrence Technology University found that music lyrics had become sadder and angrier over time¹². Although in the past, it has been difficult to make objective statements about music and its qualities, Spotify’s song features present a new way to quantify the qualities of music often thought of as subjective.
Spotify has 155 million paying subscribers, twice as many as competing services such as Apple Music. Part of Spotify’s success is due to it’s popular music recommendation system, that allows users to find new music they would enjoy based on the songs they currently like and listen to. In order to recommend music successfully, Spotify must have an accurate system for understanding the qualities of the music available.
Other studies have been conducted on popular Spotify API data, we look to improve on these by looking at more songs over a larger period of time³. We will also be the first to create an interactive visualization of this data, instead of the static images that are currently available. With our visualization, we will be able to provide users with richer storytelling and the possibility for more interpretations. It’d also be interesting to see how music changed over time and how it shaped our culture. This visualization could prove useful for historical musicologists, musicians and folks who want to learn more about music.
The Spotify API features we decided to focus on were:
- Danceability: How easy a song is to dance to. Calculated based on tempo ad beat regularity
- Acousticness: The presence of instruments not enhanced electronically.
- Speechiness: The presence of spoken words
- Tempo: The pace of the song in Beats per Minute (BPM)
- Key: The key signature of the song
- Loudness: How loud the song is in decibels (dB)
- Mode: Whether the song is in a major or minor key
- Energy: The song’s perceived level of intensity. Partially based on speed and loudness⁴
Design Process
We decided early in our design process that we wanted to represent the different features using radar charts. We thought that they would be the best way to visually present and contrast the different feature values of each year. The radar charts also serve as a reference to the circle of fifths, a widely known visual tool for organizing the 12 different key signatures in music theory.
We realized that the key signature feature was not quantitative like other features but was instead categorical, and would thus be difficult to represent on the same chart. We then decided to create a second radar chart with the different key signatures as the points on the axis, an even more overt reference to the circle of fifths.
Making the dashboard interactive was a top priority for us from the start. We wanted users to be able to browse through the data, changing the visualizations wherever necessary to find the trends, or stories they were interested in. A slider came up as the best way to move smoothly from year to year. Then we decided to include dropdown menus to the side of the dashboard to allow for filtering and combining the charts.
Beta release
We used this Kaggle repository’s dataset for our visualization. It contains data extracted using Spotify’s API on songs from 1960 to 201⁹⁵. The author made this dataset to build a hit predictor model using machine learning, so it had both hit and flop songs. We created our own list of top 5 songs of the Billboard Hot 100 and used Microsoft Excel’s inner join function to get Spotify API data for matching songs. This resulted in a smaller dataset than if we had used Spotify API ourselves to get data on all songs in the list. We used 9 of the 14 features available in the dataset as some features were irrelevant while some had too many missing values. We had to normalize the data as one of the features, song duration, was significantly larger than others and it.
In the sketching process, we decided to implement two radar charts in Tableau to visualize our data. They would be manipulated using filters for variables like the year of release, peak rank in the Billboard Hot 100, and song genre.
We used a Tableau extension called ShowMeMore. It saved us a lot of time as it had a radar chart implementation and we had to feed the data to it. The data needed some restructuring to be feeded into the extension
The beta version of the dashboard had some of the proposed features and a new one. Also, it was based on a subset of the final dataset. The radar charts were accompanied by a table showing the years and its data visualized using a bar chart. The bar chart was added automatically to the dashboard and we thought it could be useful in case it becomes difficult to interpret the radar chart. Users could view data for a year by selecting it from the table. We couldn’t get genre data for the songs as Spotify’s API has genre lists for an artist, but not for their individual songs. Gathering genre data manually for over 2000 songs would have been very time consuming.
User Evaluation
For our user evaluation we presented the beta version of our dashboards to 3 users. Each one was provided with a list of tasks to complete while we watched and took notes. The tasks were centered around finding information about certain years of music and comparing the features between years.
Based on the evaluations we were able to identify several issues with our dashboards. While users were able to understand the radar charts and gather information from them, their attention was often drawn towards the bar charts to the side of the dashboard which we had intended as a supplement to the radar charts. Several users also struggled with the filters and sliders on the dashboard. At this stage the different selection tools were dependent on each other without any hierarchy, thus whenever a user decided to use one filter, they would have to reset any other filters currently in use. The test users could not figure this out on their own and had to receive help to finish some of the tasks.
Demo: https://youtu.be/zg1BjkAfSNI
Data Insights
To analyze the dataset, we uploaded it into the R statistical software and ran a simple linear regression for each feature against the years. By doing so, we hoped to see which features had significantly and linearly increased or decreased over the years.
Our analysis confirmed that Danceability, Speechiness, Loudness and Tempo significantly increased from 1960 to 2019. Whereas, Acousticness was the only feature that showed a significant decrease.
These results are not surpsing given the changes we have seen in popular music over the years. Hip-hop and Rap music contained a considerable amount of spoken word and have become incredibly popular, thus contributing to the increase in Speechiness. Increases in Danceability and Tempo, along with decreases in Acosuticness can likely be attributed to the rise of electronic dance music since the late 1980s⁶. The increase in Loudness is a widely known phenomenon in music that has been dubbed “The Loudness War” by musicians and music critics.⁷
Takeaways
This was a very informative learning experience for both of us. Coming into this course, neither of us had any experience with Tableau, and this project gave us so much hands-on experience with it. We learned that while using app extensions like we did can make the process much simpler, especially for Tableau newbies, it can also limit creativity and flexibility when it comes to expressing our design ideas. Not being able to share our dashboards on Tableau public is another unintended inconvenience that came from using an extension. In the future, with more experience in Tableau or even a coding based software like D3, we believe that we could be able to create even more usable and effective versions of our dashboards.
During this project, we also saw the immense value of user feedback in the design process. It would have been much harder for us to see what a usability problem our original filters were without watching and hearing from users themselves. In hindsight, even more user feedback would have been helpful at various stages in the design process, from the early sketching all the way up to the final release.
Hopefully, one day, this visualization can develop into a significant contribution to both data visualization and music history.
References
- Elena Georgieva and Blair Kaneshiro
- Popular music lyrics become angrier and sadder over time
- Spotify Song Popularity & Genre Exploration
- Web API Reference | Spotify for Developers
- The Spotify Hit Predictor Dataset (1960–2019)
- Pop music is louder, less acoustic and more energetic than in the 1950s
- Pop music too loud and all sounds the same
