Programming on Hard Mode: Making API calls in Python using the IGDB API

As a segue from my main set of posts, this one is to outline how I created a list of game titles. It is sort of a lazy tutorial’ish type of post. To stick with the theme of my posts I did the work on my 12.9" iPad Pro. It worked out pretty well, although I did hit some speed bumps along the way.

Thing to know 1: StaSh uses its own built in pip that doesn’t handle complex module installs very well. It’s kind of a pain actually. Rumors are that one can manually install these modules. I haven’t taken the time to figure that out yet. I was able to make do with what was available. Juno may have better module support, but then you’re coding in a notebook.

Thing to know 2: Pandas isn’t available and cant be installed. Major bummer. I’m not sure how I’m going to get around this. While I can code as if it were there, I’m not sure how I will do any sort of testing.

Thing to know 3: Pythonista 3 has built in linter so I don’t need to worry about figuring out how to install black. It works well enough for what I’m trying to do.

Thing to know 4: Something happened with my numpy install. I’m not exactly sure what. I know it was related to trying to install some other module (probably pandas) and needing a newer version of numpy. I tried to update through StaSh’s pip. That didn’t work. Now I can’t import numpy explicitly, but somehow it is available as np when I run code… No idea what’s going on. I may need to remove and install Pythonista 3 and StaSh again.

How to get a list of game titles from IGDB API

To create this list, I needed to go on a hunt for a database of game titles. I found what I was looking for in the IGDB API. Setting this up was fairly simple. Set up a Twitch account, register your app (http://localhost for our purposes), get your Client ID and Client Secret, use those to generate a token, use the token to get a list of game titles.

Navigate over to https://dev.twitch.tv/ and set up an account. Once at your Developer Console, click Register Your Application and fill in the form. Give your application a name and use http://localhost as the OAuth redirect.

Pick a category for your app and prove you are a human. You will be dropped at a list of your applications. Click Manage on the app you just set up.

In the management Console you will have access to your Client ID. Click New Secret and store both some place secure. You will need these to generate a token.

In the Files utility for iOS create a folder where you will store your work for this little coding exercise. Then open Pytonista 3 and select Open under the External list of the file explorer. Select Folder. Now find your working folder in the local file system. Once it is opened, create your python script file.

Now we need to initialize our git repository in the working folder. Open up Working Copy and select Link external directory. Find your working folder and select it.

Click Add Remote.

Then click Create Repository. Make your configurations and Confirm Create. At this point you can make your first commit and push up to remote.

Now that source control is settled for the utility code, it’s time to actually code. Back in Pythonista 3, I started by creating a yaml file for my configuration and sensitive values. Here is where I store my Client ID and issued token. It’s super basic, and relatively flat in format. Just three keys with the values needed; user, token, and limit. Limit is a bit of a misnomer, but is the terminology used in the API. This refers to the length of values returned with each call. Together with offset, one can effectively page through the massive list of game titles over a sequence of calls.

Let’s start by loading our configuration data. Import yaml and, assuming your config file is in the same location as this script, load the file into a variable cfg.

Next we need to create a function that can accept two arguments, the last offset value and an IDGBWrapper object, construct our API call with options necessary to get what we want, then make that call, returning the response from the call.

Notice that I am passing in the IDGBWrapper object and not instantiating it in the function. I would like to reuse the object with each successive call to the API instead of recreating it each time. Additionally, I imported time and used the sleep function to pause the script for a single second, just to be polite. The API has rate limits that only allow for a maximum of 4 calls per second. You will need sleep for some amount of time to not get cut off. I've also brought in IGDBWrapper from igdb.wrapper. This will need to be installed, but it seemed to work out OK using StaSh. Maybe it threw a few errors or something. I can’t remember. It seems to work for what we are doing here.

Now we need a function that takes an argument end for the number of game titles we would like to stop at. It defaults to 10,000. I don't really know how many games are in the database behind this API. I do get 10,000 back though. It’s a start. In this function I create my IDGBWrapper object, I open/create a csv file to write these game titles to, and start a loop that iterated across a range from 0 to the end. Each run of the loop writes rows to the csv file from the returned output of our previous get_games() function.

To write this output in csv format without pandas I had to parse out the json using loads() on the returned string with single-quote swapped for double-quote.

This all works. Maybe there is a more Pythonic way of doing it. Maybe there is a more efficient way. Whatever the case may be, I still need it to sleep for some time each API request, so it really doesn’t matter so long as this works.

And finally, the entire script. Add in the standard __name__ == “__main__” conditional statement and process args from the command line however you like and try it out.

Some additional things to note: I comment out numpy in my code because my installation of Pythonista 3 and StaSh decided to randomly complain about it. The code still works for me… No idea… I am probably not handling the args well, but the whole iPad+Pythonista 3+StaSh arrangement makes running this from the terminal a bit awkward. Pressing the play button in Pythonista 3 does not give you the opportunity to give the script any values. Trying to run from StaSh requires navigating the iPad file system to actually find this external folder of code. If I was developing with use exclusively on iOS with this development environment setup, I would just use the config file or hardcode a value. Making these adjustments is trivial if needed.

They aren't needed. Not worth wasting time since all I need is the list.

Success! Reflect on what we’ve done here.

We used an iPad to write up a script in Python that handles some really basic API calls and processes the returned data into a csv file for use in my WIP Name Generator project. It even works!

Stage your code, commit, and push to preserve the work you did.