Using the httr package in R to get audio features from Spotify API
Making POST and GET HTTP requests while keeping the credentials out of the R script.
Introduction
As I was looking for ways of interacting with the Spotify Web API in R, I found that some APIs have wrappers . Basically, a wrapper is a package with functions that facilitates the use of the API, by simplifying the HTTP requests and even wrangling the retrieved data.
For the Spotify Web API there’s a wrapper called Spotifyr, which is made for pulling back audio features and other catalog information. For example you can get your most played tracks or the albums for an artist.
However, as I’m just beggining to work with API’s in R, I think it may be a better idea to make direct HTTP requests instead of using the wrapper.
In this article we’ll go thorugh the next specific points
- Using the package httr to make HTTP requests.
- Getting Spotify API credentials and authenticating with POST/GET request.
- Avoid writing your credentials in your R script.
- Using GET requests to retrieve audio features for an album.
I’d like to make clear that this article is more focused on illustration purposes. This information it’s really useful in cases when there’s no wrapper, or when it can’t retrieve the data you need from the API.
A quick overview of the Spotify API
API stands for Aplication Programming Interface, in simple words its function is to make possible that two programs comunicate with each other and interchange data. A protocol is needed in order to allow that interaction.
In particular the Spotify API is a web API, so it uses the Hypertext Transfer Protocol (HTTP). Here, the R script that makes the request will be the client and the Spotify API the server giving the response.
“An API is the tool that makes a website’s data digestible for a computer” — Brian Cooksey
By exploring the Spotify’s API documentation it can be seen that it’s really complex. The API allows you to make several apps and programs for different purposes, for example you can build a smart watch app that displays a user’s most played artists.
However, in this case we’re only interested in retrieving audio features.
Spotify IDs
Every object in Spotify has a unique ID, for example a song, album, or playlist.
To get a Spotify ID, you just have to right-click > Share >Copy Link
All the Links (e.g. album, song, playlist, band’s name) have a similar structure, you can find the object’s Spotify ID in the character string between the last slash and the question mark, as highlighted in the next code snippet
https://open.spotify.com/track/5sYFVQ4Ov2zwiV6vFuELZB?si=1ab6f7f6f9114da8
Developer Account
First of all you need a Spotify account, both free or premium versions will work. Then you need to apply for a developer account:
- It’s neccesary to go to the dashboard of the Spotify API website and log in.
- Youl’ll be asked to read and accept the Spotify Developer Terms of Service.
And that should give you the developer account.
Developer App
After you got your developer account ready, you should see this dashboard when you log in
The Create An App option will generate a Client ID as well as a Secret Key, which we’ll use to obtain the Acess Token required to make the HTTP requests. More on that in the following subsection.
Authorization Flows
Even though there are 4 authorization flows that the Spotify API supports, in this case we’re going to be using only the Client Credentials Flow because it provides app authorization to interact with the API. It works like this:
Send a POST request to the API with
- Client_ID and Secret_Key.
You get in the response from the API:
- Acess Token.
So the Access Token is going to give the authorization when making GET requests to retrieve track features. NOTE that the Client Credentials Flow won’t work for accessing anything related to a user’s private data as it‘s not meant to ask for the permissions.
Theory aside, let’s start coding.
Loading up the libraries
We’’ll need httr which is an R package made for interacting with modern web APIs with HTTP (in this case GET and POST methods are going to be used). Also the package dplyr for managing dataframes.
library(httr)
library(dplyr)
Keeping the credentials away from git
While I was making this project I needed a way to hide the credentials from the version control system, in my case git/GitHub. So I found out that there are several ways to do it, but I’ll present here the one that was the most practical for me.
Environment variables
Create a .Renviron file that stores the credentials as environment variables, I recommend the home folder — so you dont’ commit the file by accident when working on your project folder
touch ~/.Renviron
Then open the .Renviron file in any text editor and write your Spotify ID and the Secret Key
SPOTIFY_ID = "your spotify id here"
SECRET_KEY = "your secret key here"
Later those variable values can be retrieved with the function Sys.getenv()
,
If Sys.getenv()
returns an empty string, the R session has to be restarted before trying again.
NOTE: This method may not be the best idea in a shared computer, because the credentials will be in plain text in your home folder.
Getting the Access Token with POST
According to the Spotify API documentation, in order to get the Access Token we have to make a POST request to the URL
https://accounts.spotify.com/api/token
With
- Client ID and Secret Key as header parameters.
- A request body parameter set to
grant_type = "client_credentials
.
So the R code is
response <- POST(
"https://accounts.spotify.com/api/token",
config = authenticate(user = Sys.getenv("SPOTIFY_ID"),
password = Sys.getenv("SECRET_KEY")),
body = list(grant_type = "client_credentials"),
encode = "form"
)
Note that the header parameters go in config
, POST already has the function authenticate
to pass the credentials and (kind of obvious) the body parameter in body
.
It’s recommended that you retrieve the credentials directly in the POST
function instead of storing it in a variable before. Sometimes variables values are saved into R Worskpace which can be commited accidentaly to the version control system.
Finally, Spotify API uses Bearer Tokens, so it needs the word Bearer before the Access Token string, here’s how to concatenate them
#extract content of response
token <- content(response)
# Paste the token_type with the access_token
# needs it when making the authorization
bearer.token <- paste(token$token_type, token$access_token)
Getting audio features with GET
To make the GET requests you have to specify a URL. Spotify API has several URL endpoints for diferent kinds of data, as can be seen on the documentation. For example
- Get audio features for a track
https://api.spotify.com/v1/audio-features/{id}
- Get catalog info for an album’s tracks
https://api.spotify.com/v1/albums/{id}/tracks
Rate Limit
The Spotify API has a Rate Limit for the requests, although the docs don’t specify what the Rate is, it indicates that is calculated based on the number of calls to the API in a rolling 30 second window. The response will give a 429 error in case of exceding the limit.
Features data dictionary
The track’s features are properties of audio analysis. Some of the most insightful features are
- Energy: Measure from [0,1], represents a perceptual measure of intensity and it’s based of dynamic range, general entropy and other properties.
- Liveness: Measure from [0,1], indicates a probability that the track was recorded live by detecting the prescence of an audience in the recording.
- Valence: Measure from [0,1], gives an idea of the positivity of the track. For example 1 indicates a joyful track while 0 an angry/sad one. — It’d be interesting to see how relative this feature is.
All the features are well described in the data dictionary.
Audio features for 1 track
Let’s get the audio features for Santana’s version of Oye Como Va.
Note that you must add the Access Token with add_headers()
in config
in every GET request
track.id <- "5u6y4u5EgDv0peILf60H5t"
track.response <- GET(paste0("https://api.spotify.com/v1/audio-features/",track.id),
config = add_headers(Authorization = bearer.token)
)
So if the response was successful should give something like this with Status:200
Now the actual features can be extract with content()
and then converted to a tibble with as_tibble()
track <- as_tibble(content( track.response ))
track.features <- track %>% select(c(1:11, 17, 18))
The valence is really high, makes sense as Oye Como Va is a joyful groovy song. Also if you like music theory, the key is 7 and mode 1 which means G-major, those are the same notes in A-Dorian. This is no surprise because it’s well known this song is in A-Dorian.
Audio features for an album
First, let’s retrieve catalog info from one album’s tracks, in this example the Nevermind album by Nirvana (if you’d like to use another album just change the album.id)
album.id <- "2guirTSEqLizK7j9i1MTTZ"album.response <- GET(paste0("https://api.spotify.com/v1/albums/", album.id, "/tracks"),
config = add_headers(Authorization = bearer.token)
)
Next, we have to extract the content
album.content <- content(album.response)
album.songs <- album.content$items
Finally, we use unlist
and grep
to obtain the ids for each song , then saving it in a vector
aux <- unlist(album.songs)
filter.ids <- aux[ grep("^id$", names(aux)) ]
album.songs.ids <- cbind( as.character(filter.ids) )
Now that we have the ids, the function lapply()
allows to apply the GET request to each song id, then storing the responses in a list
album.features <- lapply(1:length(album.songs.ids), function(i) {
GET(url = paste0("https://api.spotify.com/v1/audio-features/", album.songs.ids[i]),
config = add_headers(authorization = bearer.token))
}
)
To extract the content in the responses list we can use sapply
album.features.content <- sapply(1:length(album.songs.ids), function(i) {
content(album.features[[i]])
}
)
Next, we map the album.features.content
lists to a tibble and then unlist
and convert every colum to numeric
# convert lists to tibble and then unlist every columnsongs.tidy <- as_tibble(t(album.features.content))
for (i in 1:length(songs.tidy)) {
songs.tidy[,i] <- format (unlist( select(.data = songs.tidy, i), use.names = FALSE ),
scientific = FALSE)
}
The only thing left is to select the desired columns and export the tibble to a .csv file
songs.features <- songs.tidy %>% select(c(1:11, 17, 18))
songs.features# export table to csv
write.csv(songs.features, file = "nevermind-features.csv", row.names = F)
We can quickly see that Smells Like Teen Spirit (1) has high energy. The 6 and 12 songs have the highest acousticness, those are Polly and Something In The Way.
Conclusion
It was shown how to access the Spotify API with credentials and retrieve features of songs. It was ilustrated the use of the httr package in R to make GET and POST requests and to convert the content to a tibble. All this is essential for interacting with APIs when there are no wrappers.
SOURCE CODE
You can find the full R Script used in this article in this GitHub repository.
Recommended reading —
- More about APIs: https://zapier.com/learn/apis/
- I took inspiration from Ray Heberer’s blog post on this same topic, you should check it out: https://rayheberer.ai/archive/spotifyapi/
- Useful package for encrypting the credentials in R: https://www.infoworld.com/article/3320999/r-tip-keep-your-passwords-and-tokens-secure-with-the-keyring-package.html