Text mining last 60 years of American Pop Music
In the following post, we will be exploring the lyrics of the Top 100 Billboard hits of the last 60 years in an effort to unearth what our music really talks about.
Art & literature depict everyday life & our everyday life mimics art & literature.
Within this vicious cycle is stuck a 43-billion-dollar industry of American pop music. From Elton John to Taylor Swift, all pop artists have been responsible for defining what we, as a generation, aspire for, act like & admire. Understanding the role of music in our lives will require an arduous study. This analysis is a modest attempt to understand how American pop songs have shaped a generation and some more.
The dataset was created by Kaylin Pavlik. The same can be retrieved here.
We are in love with ‘love’
The only reason a 17-year-old wizard could defeat one of the world’s darkest sorcerers in the epic saga of Harry Potter was love. Love, as we know, has been collectively thought of by philosophers, thinkers, artists, writers, poets, and musicians. It’s not a surprise that it happens to be the most common word in all the Billboard songs from 1958 to 2015.
Love has driven men to war and it continues to drive men and women, alike, to create pop songs. In its article, Why Exactly Are We So Obsessed With Songs About Heartbreak?, Vice explains how we find solace in songs about heartbreak. Music, that addresses the loss of love, helps the isolated and often shameful healing of a broken heart. While we continue to have our hearts broken, pop artists will continue to perform contemporary ballads of love for us. The Billboard of most popular hits will be the testimony of our admiration & dependency on these love songs.
Truncating words is a natural way of being for Millenials
Black Eyed Peas, a pop band, came out with the song, “Imma Be” in 2010. It continues to appear in every line of the song. The word is a truncated version of the phrase, I am going to. With texting culture (witnessed after the 90s) defining the way we speak, abbreviations, which are used to help send smaller & quicker texts, find their place in our songs too. Just like Imma, hella for a hell of a lot, kno for know, ballin for balling, lolli for a lollipop, lemme for let me are a few examples of words which are more prominent in songs after the 90s as compared to the songs of the 90s.
Increasing profanity & sexual content
There’s a presence of provocative words like bitch, sexy, bum, fuck, shit, smack in more recent songs (after the ’90s) as compared to older (of the ‘90s).
The listeners’ tolerance toward profanity & sexual content is increasing. “Approximately 92% of the 174 songs that made it into the [Billboard] Top 10 in 2009 contained reproductive messages,” asserted SUNY Albany psychology professor Dawn R. Hobbs in Evolutionary Psychology. While she carefully sticks to euphemism by terming it ‘reproductive messages’, contemporary musicians aren’t shying away from saying things as they are.
Gender Roles in pop music
The American entertainment industry is plagued with claims of sexual abuse, gender pay gap, lack of representation, cultural appropriation, and more. Women make up to 21% of artists, 12% of songwriters & 2% of producers. (Source: The Music Industry Still Has A Long Way To Go For Gender Equality, Forbes). While the precise contribution of male vs female artists isn’t a part of this analysis, what follows is an interesting insight into gender roles reinforced by pop songs.
To understand this, bigrams that begin with ‘he’ or ‘she’ will need to be analyzed. Stop words are a part of this since the retention of ‘he’ & ‘she’ in the tibble is imperative.
The figure on the left denotes the most used bigrams in American pop songs which begin with a gendered pronoun. Take account of the prominence of ‘she’ over ‘he’. This makes sense since the majority of artists dominating the industry are straight men who must mention a ‘she’ in their songs about love.
On calculating the log odds ratio, we can discover the words that exhibit the biggest difference in relative occurrence after ‘he’ & ‘she’. On arranging it in ascending order, we will get the words that are just as likely to occur after ‘he’ as they are after ‘she’.
With enough liberty, it can be said that these verbs are gender-neutral i.e. neither of these words defines either of the genders. Understanding gendered verbs (words with the most relative difference) in American pop songs over the course (songs from the 90s & after) will help understand the changing depiction of men & women in songs.
Women have cooed, moved & lied while men have drunk, played & sung in American pop songs of the ’90s & before.
While women continue to move, rock & like, men have taken a more assertive stance, showcased by negative words like wouldn’t & wasn’t in the songs featured after the ’90s.
While these words cannot be precisely marked to their creators’ genders, it is important to be mindful of the fact that most song creators, writers & producers happen to be men. Will depiction of women change, improve or deteriorate with an increase in women artists can be answered only when the utopian wish of equal representation is fulfilled.
Words that defined the last few years of music
Finding the highest tf-idf words in the last 5 years will unearth tokens that best define that year of music.
The tf-idf graph has acted as a throwback to cult favorite songs like Titanium(2012), Gangnam Style(2013), Anaconda(2014), Wiggle(2014) & Uptown Funk (2015). While only text from the lyrics was analyzed, a lot of artist names surfaced too, like Enrique (2015), Iglesias (2015), Nicky (2015), Iggy (2014), Justin (2013). Confusing at first, this was explained by the fairly common trend of shouting out the artist’s name in the song.
Art reflects the positives & negatives of society. What we read, listen & see, we mimic. The inverse is often true too. We create what we live every day. We wouldn’t be entirely wrong in demanding a handful of artists to break this vicious circle by creating more thoughtful & conscious music.
