Testing in Elixir:: Chapter 3: The outside world is scary…

So, in previous chapters we have taken a very micro view and approach to testing in Elixir. This has been necessary in order to get familiar with the basics upon which more complex test cases are built upon, but now is the time to move on.

This chapter will focus on how to approach testing external APIs / HTTP requests. A lot of what we will go through is massively influenced by the below blog post by Jose, as well as what I currently work with in production:

This is definitely a topic that sparks discussion between development teams, particularly around what is considered a ‘mock’, and what the tests should ACTUALLY be testing.


So, let’s set the scene for what we will be building:

Wow, okay. You might assume that this wouldn’t happen, but a quick search returns 365210commits that have this as their message. Awkward…

…But, back to the task at hand. What we will be doing is the following:

  • Make a request to GitHub
  • Parse the response
  • Return one entry as a map that contains author details and the problematic commit

Project-wise, we will firstly need something to make the request, then something to parse the request and, finally, a module we can interact with as a user. The request aspect is what we will be focusing on — we will be creating a mock version of the module that will be used for testing.

It may seem strange to mock out a whole module just for test scenarios and, further, there is the discussion around:

‘If you mock a request module are you actually testing anything?’

This is a very valid discussion. It really comes down to whether you trust the external API — and want to just focus your testing on the program flow — or if you feel that without an actual API request you cannot trust anything. Both sides are valid and both sides are extremely important. However, as we will see, they do not have to be mutually exclusive.


Initial project implementation

Before we start creating mocks, let’s review our basic implementation:

Here we have a function fetch/0 that makes a request to the GitHub API before handing off to our Parser module, which looks like this:

This module will then parse a successful response and extract out the map required for us to complete our ticket. However, the direct call to an external API makes our test suite very fragile and very coupled to the GitHub API. So, let’s take some steps to abstract the direct calls away and create a mock request to make testing friendlier.

Creating a mock module

On line 9 of our basic implementation, we have a hardcoded HTTPoison.get/3. As a first step, lets move this into its own module. This may seem overkill, but it allows a clean abstraction for us to intercept:

Brilliant. Now, before we can write our fake request module we have a few configuration steps to perform. Our first step is the config.exs .

Inside this file, uncomment the line that contains:

import_config "#{Mix.env()}.exs"

We now need to create a .exs file for each environment. Typically these are:

dev.exs , prod.exs , test.exs , ci.exs

In each of these they need to contain the line use Mix.Config as default.

Inside our dev.exs we can put the following:

This will allow us to specify which module we use for our requests. We can now tweak our main file to pick this up:

line 9 has now been changed to use a function call from the request function. This looks and reads strangely initially, but if we look at line 21 we see the request function.

This will get the value of the key github and use that for the references of request() . By specifying a particular module in the config file, Elixir will use this. This is what will allow us to inject particular modules depending on our current environment.

Note: Often you will see Application.get_env/3 inside a module attribute. The reason that we are wrapping it inside a function call and not using a module attribute is due to compile-time vs run-time. Module attributes are available at compile-time but run-time you will need to wrap it inside of a function.

Now we have the ability to change modules depending on environment. Let’s update our test module to contain:

Now, where to create this module?

As we are using it solely for tests, typically it will be in a support directory inside of the test directory.

And what should go inside of this module?

This depends. If we look at our implementation, we know that there should be a call() function and that it should handle the three cases of a HTTPoison.get/3 request. So, our skeleton mock request module looks like this:

We now have 3 different call() functions that we can manipulate to test the actual flow of our application. But what shall we return?

Well, for the last two we just need to return responses that are agnostic from GitHub itself. We can use a standard 404 response for the second one, and a generic error for the third:

Brilliant!

The first call() is all that remains. For this, we only need to return the bare minimum for our application to complete its flow, so what we will do is perform an actual request and then use that mock data to test against.

This may feel strange, but it is what we would be testing against if we were using the actual request. By using and trimming down an actual request we end up with the following module:

By using this data we can test our implementation as follows:

Here we are testing the full flow of our application. Although we are intercepting our API call, we are ensuring that our application is able to handle all of the responses that may come from the actual calls.

So, let’s run our tests…

Oh.

Currently, our mock request module cannot be found. This is because there is one more bit of config we need to add. We need to modify our applications mix.exs:

We need to add lines 9, 11, 31, 32 and 33 . What this will do is allow us to have access files that need to be compiled within a test context. Now our tests will pass:

Congratulations on your first mock request module!

I hope this now makes sense, and feels a little more natural. There are some other steps we can take to tighten up the test flow and ensure we are covering everything we can be in control of:

Behaviors

With there being more than one module trying to do the same thing, there is an opportunity to have your development code slip and diverge from your test code. To get around this, Elixir has a handy callback and behaviour interface.

This allows you to have a module where you can declare an interface, similar to other languages, that will throw compiler warnings if the module is not abiding by it.

So, for our mock and actual request module let’s create one:

Here, we have declared the function and given it a typespec we expect of the function call. In order to use this we then put the following line at the top of the Request and MockRequest modules:

@behaviour Chapter3.Github

This is all focused on controlling the application code as much as possible. We still need to account for the actual API being there / being able to highlight and capture problems outside of our control, and this is set out below.

External tests:

As a sanity check, testing the actual endpoint should be an almost heartbeat-check approach.

We will make the same request and assert that the response is 200. Easy. What we will also do is @tag it and exclude these tags from our default mix test — this will ensure we do not hammer the end point and get rate limited by accident.

Our test will be this:

And we update our test_helper.exs :

We can now set up a separate job to run our external tests explicitly with:

mix test --only external_api

And with this we are done!!!

Conclusions:

Once again, all the code for this Chapter can be found HERE.

This is a lot to take in, and is very different to what I had used previously. However, you will very quickly adapt to the flow, and this will allow you to focus on what is important — this is always our application.

If you use this approach in production — or utilise any other approaches in similar scenarios — I would be very interested in hearing your experiences!