Using the httr package in R to get audio features from Spotify API

Making POST and GET HTTP requests while keeping the credentials out of the R script.

Image by author, made with Canva.

Introduction

As I was looking for ways of interacting with the Spotify Web API in R, I found that some APIs have wrappers . Basically, a wrapper is a package with functions that facilitates the use of the API, by simplifying the HTTP requests and even wrangling the retrieved data.

For the Spotify Web API there’s a wrapper called Spotifyr, which is made for pulling back audio features and other catalog information. For example you can get your most played tracks or the albums for an artist.

However, as I’m just beggining to work with API’s in R, I think it may be a better idea to make direct HTTP requests instead of using the wrapper.

In this article we’ll go thorugh the next specific points

• Using the package httr to make HTTP requests.
• Getting Spotify API credentials and authenticating with POST/GET request.
• Avoid writing your credentials in your R script.
• Using GET requests to retrieve audio features for an album.

I’d like to make clear that this article is more focused on illustration purposes. This information it’s really useful in cases when there’s no wrapper, or when it can’t retrieve the data you need from the API.

A quick overview of the Spotify API

API stands for Aplication Programming Interface, in simple words its function is to make possible that two programs comunicate with each other and interchange data. A protocol is needed in order to allow that interaction.

In particular the Spotify API is a web API, so it uses the Hypertext Transfer Protocol (HTTP). Here, the R script that makes the request will be the client and the Spotify API the server giving the response.

Image by author, inspired by the image in Spotify API docs

“An API is the tool that makes a website’s data digestible for a computer” — Brian Cooksey

By exploring the Spotify’s API documentation it can be seen that it’s really complex. The API allows you to make several apps and programs for different purposes, for example you can build a smart watch app that displays a user’s most played artists.

However, in this case we’re only interested in retrieving audio features.

Spotify IDs

Every object in Spotify has a unique ID, for example a song, album, or playlist.

To get a Spotify ID, you just have to right-click > Share >Copy Link

All the Links (e.g. album, song, playlist, band’s name) have a similar structure, you can find the object’s Spotify ID in the character string between the last slash and the question mark, as highlighted in the next code snippet

https://open.spotify.com/track/5sYFVQ4Ov2zwiV6vFuELZB?si=1ab6f7f6f9114da8

Developer Account

First of all you need a Spotify account, both free or premium versions will work. Then you need to apply for a developer account:

1. It’s neccesary to go to the dashboard of the Spotify API website and log in.
2. Youl’ll be asked to read and accept the Spotify Developer Terms of Service.

And that should give you the developer account.

Developer App

After you got your developer account ready, you should see this dashboard when you log in

The Create An App option will generate a Client ID as well as a Secret Key, which we’ll use to obtain the Acess Token required to make the HTTP requests. More on that in the following subsection.

Authorization Flows

Even though there are 4 authorization flows that the Spotify API supports, in this case we’re going to be using only the Client Credentials Flow because it provides app authorization to interact with the API. It works like this:

Send a POST request to the API with

• Client_ID and Secret_Key.

You get in the response from the API:

• Acess Token.

So the Access Token is going to give the authorization when making GET requests to retrieve track features. NOTE that the Client Credentials Flow won’t work for accessing anything related to a user’s private data as it‘s not meant to ask for the permissions.

Theory aside, let’s start coding.

Loading up the libraries

We’’ll need httr which is an R package made for interacting with modern web APIs with HTTP (in this case GET and POST methods are going to be used). Also the package dplyr for managing dataframes.

library(httr)
library(dplyr)

Keeping the credentials away from git

While I was making this project I needed a way to hide the credentials from the version control system, in my case git/GitHub. So I found out that there are several ways to do it, but I’ll present here the one that was the most practical for me.

Environment variables

Create a .Renviron file that stores the credentials as environment variables, I recommend the home folder — so you dont’ commit the file by accident when working on your project folder

touch ~/.Renviron

Then open the .Renviron file in any text editor and write your Spotify ID and the Secret Key

SPOTIFY_ID = "your spotify id here"
SECRET_KEY = "your secret key here"

Later those variable values can be retrieved with the function Sys.getenv(),

If Sys.getenv() returns an empty string, the R session has to be restarted before trying again.

NOTE: This method may not be the best idea in a shared computer, because the credentials will be in plain text in your home folder.

Getting the Access Token with POST

According to the Spotify API documentation, in order to get the Access Token we have to make a POST request to the URL

https://accounts.spotify.com/api/token

With

• Client ID and Secret Key as header parameters.
• A request body parameter set to grant_type = "client_credentials.

So the R code is

response <- POST(
  "https://accounts.spotify.com/api/token",
  config = authenticate(user = Sys.getenv("SPOTIFY_ID"), 
                        password = Sys.getenv("SECRET_KEY")),
  body = list(grant_type = "client_credentials"), 
  encode = "form"
)

Note that the header parameters go in config, POST already has the function authenticate to pass the credentials and (kind of obvious) the body parameter in body.

It’s recommended that you retrieve the credentials directly in the POST function instead of storing it in a variable before. Sometimes variables values are saved into R Worskpace which can be commited accidentaly to the version control system.

Finally, Spotify API uses Bearer Tokens, so it needs the word Bearer before the Access Token string, here’s how to concatenate them

#extract content of response
token <-  content(response) 
# Paste the token_type with the access_token 
# needs it when making the authorization
bearer.token <- paste(token$token_type, token$access_token)

Getting audio features with GET

To make the GET requests you have to specify a URL. Spotify API has several URL endpoints for diferent kinds of data, as can be seen on the documentation. For example

• Get audio features for a track

https://api.spotify.com/v1/audio-features/{id}

• Get catalog info for an album’s tracks

https://api.spotify.com/v1/albums/{id}/tracks

Rate Limit

The Spotify API has a Rate Limit for the requests, although the docs don’t specify what the Rate is, it indicates that is calculated based on the number of calls to the API in a rolling 30 second window. The response will give a 429 error in case of exceding the limit.

Features data dictionary

The track’s features are properties of audio analysis. Some of the most insightful features are

• Energy: Measure from [0,1], represents a perceptual measure of intensity and it’s based of dynamic range, general entropy and other properties.
• Liveness: Measure from [0,1], indicates a probability that the track was recorded live by detecting the prescence of an audience in the recording.
• Valence: Measure from [0,1], gives an idea of the positivity of the track. For example 1 indicates a joyful track while 0 an angry/sad one. — It’d be interesting to see how relative this feature is.

All the features are well described in the data dictionary.

Audio features for 1 track

Let’s get the audio features for Santana’s version of Oye Como Va.

Note that you must add the Access Token with add_headers() in config in every GET request

track.id <- "5u6y4u5EgDv0peILf60H5t"
track.response <- GET(paste0("https://api.spotify.com/v1/audio-features/",track.id),
               config = add_headers(Authorization = bearer.token)
)

So if the response was successful should give something like this with Status:200

Now the actual features can be extract with content() and then converted to a tibble with as_tibble()

track <- as_tibble(content( track.response ))
track.features <- track %>% select(c(1:11, 17, 18))

The valence is really high, makes sense as Oye Como Va is a joyful groovy song. Also if you like music theory, the key is 7 and mode 1 which means G-major, those are the same notes in A-Dorian. This is no surprise because it’s well known this song is in A-Dorian.

Audio features for an album

First, let’s retrieve catalog info from one album’s tracks, in this example the Nevermind album by Nirvana (if you’d like to use another album just change the album.id)

album.id <- "2guirTSEqLizK7j9i1MTTZ"
album.response <-  GET(paste0("https://api.spotify.com/v1/albums/", album.id, "/tracks"),
                          config = add_headers(Authorization = bearer.token)
)

Next, we have to extract the content

album.content <- content(album.response)
album.songs <- album.content$items

Finally, we use unlist and grep to obtain the ids for each song , then saving it in a vector

aux <- unlist(album.songs)
filter.ids <- aux[ grep("^id$", names(aux)) ]
album.songs.ids <- cbind( as.character(filter.ids) )

Now that we have the ids, the function lapply() allows to apply the GET request to each song id, then storing the responses in a list

album.features <- lapply(1:length(album.songs.ids), function(i) {
  GET(url = paste0("https://api.spotify.com/v1/audio-features/", album.songs.ids[i]),
      config = add_headers(authorization = bearer.token))
}
)

To extract the content in the responses list we can use sapply

album.features.content <- sapply(1:length(album.songs.ids), function(i) {
  content(album.features[[i]])
}
)

Next, we map the album.features.content lists to a tibble and then unlist and convert every colum to numeric

# convert lists to tibble and then unlist every column
songs.tidy <- as_tibble(t(album.features.content))
for (i in 1:length(songs.tidy)) {
  songs.tidy[,i] <- format (unlist( select(.data = songs.tidy, i), use.names = FALSE ),
                            scientific = FALSE)
}

The only thing left is to select the desired columns and export the tibble to a .csv file

songs.features <- songs.tidy %>% select(c(1:11, 17, 18)) 
songs.features
# export table to csv
write.csv(songs.features, file = "nevermind-features.csv", row.names = F)

We can quickly see that Smells Like Teen Spirit (1) has high energy. The 6 and 12 songs have the highest acousticness, those are Polly and Something In The Way.

Conclusion

It was shown how to access the Spotify API with credentials and retrieve features of songs. It was ilustrated the use of the httr package in R to make GET and POST requests and to convert the content to a tibble. All this is essential for interacting with APIs when there are no wrappers.

SOURCE CODE

You can find the full R Script used in this article in this GitHub repository.

Recommended reading —

1. More about APIs: https://zapier.com/learn/apis/
2. I took inspiration from Ray Heberer’s blog post on this same topic, you should check it out: https://rayheberer.ai/archive/spotifyapi/
3. Useful package for encrypting the credentials in R: https://www.infoworld.com/article/3320999/r-tip-keep-your-passwords-and-tokens-secure-with-the-keyring-package.html