I’ve been working on something(s) recently involving fetching data from an API and using it in a Wagtail site. I came to the point when I was struggling to test the API request without actually requesting the live API. Why? Well, sites can go down, take to long to respond or generally misbehave and really, my API request should be able to handle this gracefully. More specifically, it should expect it.
I actually love testing. So much so, I would be happy just writing tests all day but I find it hard just to figure out how to test specific things. This can cause me to skip writing some tests, hopefully those days are numbered. I wanted to share the solution because it’s blown my mind and opened a whole new area of testing to me.*
For this post, I’ve made a quick system for requesting quotes from an API and displaying them on the homepage of a Wagtail site. Here is the homepage models.py file:
Main thing to note is line 8, this is calling a method on my KanyeRest class which returns the quote data. So the homepage looks like this:
Here is the full KanyeRest class responsible for this madness, it’s what we will be testing:
In the real world you’d cache the response data, but for demo purposes this will do. Here is what I wanted to test on my project at the time and what we should test here:
- How does the code handle a Timeout?
- Does the data go through the parser as expected
- If there are Exceptions, will the homepage still render?
Whilst a test for checking the live API responds is useful, I’m just interested in what happens when the request fails. This is where the mock library comes in. Actually it’s where my buddy Rich came in with a bag full of mock (you may remember him from this post I credited him on).
unittest.mockis a library for testing in Python. It allows you to replace parts of your system under test with mock objects and make assertions about how they have been used.
That reference has probably been posted on Medium a thousand times but it’s clear and simple. How mock really helps with testing this stuff is it allows you to patch things and hi-jack what they are doing. In our case, we can mock the entire `requests.get` method, or just mock certain classes and methods.
The end result test_api.py file is here. But I’ll break it up…
Testing the API is up
Here, I’m setting up some values to test, seen as though I don’t want to use the live API because data changes with each request. Then a quick test_fetch to check the live API is responding.
Testing a fetch with mocked data
Using mock.patch can replace methods and classes with ones you define specifically for tests. Here I’m saying “Don’t use the fetch_data method from my KanyeRest() class, instead use mock_fetched_data”
This is a bit too general, but it’s worth showing as an example. Here, I’m testing the whole thing but overriding the data I get from fetch_data. Doing this allows me to check the end result data is what I expect (self.expected_data). The mocked method is defined in the test code outside the test class like this:
Testing the default data is used with a Timeout
Here with mock.patch, I’m mocking the get request used by requests.get. Doing this allows me to define a ‘side_effect’, and that can be a Timeout exception.
But does the page render with a Timeout?
Yep, this test is similar to the test above, we just check the rendering of the page:
This is the part where I say there is loads more to learn about Mock, test isolation and mocking too much can lead to a weird and almost useless test. For me, this was a massive jump in what I can actually write tests for, I highly recommend giving it a try, purely because requesting data from external API’s is very common.
*There are plenty of other posts on Mock and this wont be the last, they are definitely worth a read:
What the mock? - A cheatsheet for mocking in Python
A cheatsheet for mocking in Python
A cheatsheet for mocking in Pythonmedium.com
Using Python Mock to Haunt Your Code
Mock is awesome. Basically you can use it to override namespaces with whatever you want. I’ve often used it before. If…