Testing Your Python API App with JSON Schema
When writing JSON APIs you always want to make sure your endpoints deliver the data they are supposed to deliver. A good way to ensure your code acts the way it should is writing some tests. A really nice way to test JSON APIs is verifying the request’s JSON response against a JSON Schema.
If you are not familiar with JSON Schema yet, you can find the specification and more detailed information on the official web site and lots of nice explanatory examples here.
I’ve used JSON Schema before to test JSON APIs written in Ruby. Back then I used an approach similar to the one described in this Thoughtbot blog post. Now that I needed to test a JSON API written in Python I was wondering how to test JSON responses with JSON Schema using pytest. It turned out to be fairly easy to get your endpoint tests up and running, so here is what I did.
Creating an Assertion Helper
Let’s suppose we have a simple JSON response for a user endpoint GET /users/:id:
Here is an example how you would naively test this response by checking the presence of the properties (client would be a pytest fixture, e.g. a test_client of your Flask app):
As you can see, you will end up with a lot of code already for a small JSON response–and you didn’t even check for the value’s type and format yet.
So, the first thing we want to do is reducing all these assert lines to a single line that validates our json_data against a JSON Schema:
Let’s implement our assert_valid_schema() function in a support.assertions module in the tests directory.
We will use the jsonschema package. Let’s add its latest version to our requirements.txt file and run
pip install -r requirements.txtto install it. Then let’s have a look at the code:
First, we import the needed functions and modules. In assert_valid_schema() we load our JSON Schema from the given file (e.g. 'user.json'). In our _load_json_schema() helper function we assume that the schema file will be present in the tests/support/schemas/ directory. We open the file, read the content and return the parsed JSON. Last, we validate whether the given data matches our loaded schema.
If the validation fails, then a ValidationError is raised which holds some detailed information about where the validation actually failed.
Writing the JSON Schema
One last thing that is missing before our tests can pass is the user.json schema. As mentioned before we will put it into the tests/support/schemas/ directory:
Admittedly, the schema looks longish for our short user JSON response. However, we now have a single source of truth that defines the JSON structure for the user response. Besides the presence we can now also check the type and the format of values and define more complex stuff like the length of lists or allowed values for a property.
The Full Endpoint Test
With the assertion helper and the JSON Schema definition in place our final test for the user endpoint looks like this:
That’s it. Very concise and legible.
And there is another benefit: if your user response changed you’d only have to adjust the JSON Schema in user.json once and not even touch a single test function.
If you run your test
pytest tests/test_users.pyyour user response should now successfully be validated using the defined JSON Schema.
JSON Schema definitions can indeed get a bit long and confusing if you have to deal with complex and nested JSON data. However, there are ways to clean up, reuse, and extend your schemas by using references. We will have a look at internal references and file references and how you can use them in your pytest tests in a next blog post. So, stay tuned!
