Testing External Web Requests in Elixir? Roll Your Own Mock Server

Sophie DeBenedetto
Nov 21, 2018 · 7 min read
Image for post
Image for post
Pictured: your very own hand-rolled mock server

What should you do when testing Elixir code that makes web requests to an external API? We don’t want to let our code make those requests during a test run — we’ll slow down our tests and potentially use up API rate limits. We could mock such requests with the help of a mocking library or use recorded requests with the help of recording library. We’ll show you why you should avoid these approaches and instead roll your own mock server for your test environment.

Don’t: Use Recorded Requests

  • You’re adding a dependency to your test suite.
  • You’re bloating your file tree with potentially lots and lots of “cassette” files.
  • It can be onerous to record a “first cassette” for each request you need to “playback.”
  • It can be onerous to re-record cassettes if your code changes the manner in which it hits an API endpoint or if the API itself changes.
  • Imitating sad paths can be tricky, since you have to create real failed requests for every type of anticipated failure.

Don’t: Mock Web Requests

Let’s say you’re building an application that talks to the GitHub API. You have a module, GithubClient, that uses HTTPoison to make web requests to that API. You could mock these web requests by mocking the function calls made against HTTPoison. Let’s say we are trying to test the following code:

Using the Elixir Mock library, your test would look something like this:

So, what’s so wrong here? Well, first of all, we’ve now coupled our tests directly to the HTTPoison dependency. If we choose to switch HTTP clients at a later time, all of our tests would fail, even if our app still behaves correctly. Not fun.

Second, this makes for some really repetitious tests. Any tests that run code that hit the same API endpoints will require us to mock the requests all over again. Even these two tests for our happy and sad create_rep paths feel repetitious and clunky.

The repetitious mocking code that now bloats our tests make them even harder to read. Instead of letting contexts and it descriptors speak for themselves, anyone reading out tests has to parse the meaning of lots and lots of inline mocking code.

Lastly, this approach not only makes our lives more difficult but also represents a misuse of mocks. Jose Valim’s article on this topic describes it best:

Mocks are simulated entities that mimic the behavior of real entities in controlled ways…I always consider “mock” to be a noun, never a verb.

By “mocking” (verb!) our API interactions, we make an implementation detail (the fact that we are using `HTTPoison`) a first class citizen in our test environment, with the ability to break all of our tests, even when our app behaves successfully.

Instead of misusing mocks in this manner, let’s build our very own mock (noun!) server and run it in our test environment.

Do: Build Your Own Mock Server

We’ll build a simple HTTP server in our Elixir app, using Cowboy and Plug. Then, we’ll define a controller that knows how to respond to the requests that HTTPoison will send in the course of the test run of our GithubClient code. Lastly, we’ll configure our app to run our server in the test environment only.

Let’s get started!

Building the Mock HTTP Server with Cowboy and Plug

First things first, we’ll add the PlugCowboy library to our app’s dependencies in the mix.exs file.

We’ll tell our application to run Cowboy and Plug in the test environment only:

Next up, we’ll define our GithubClient.MockServer module. We want our server to do a few things for us:

  • Start up in a supervision tree when the application starts (in the test env only)
  • Handle and respond to incoming web requests.

In order to make sure we can start up our test server as part of our application’s supervision tree, it will need to implement the GenServer API’s init and start_link functions. Luckily for us, we’ll get that behavior by using the Plug.Router module.

Now, we can tell our application to start up and supervise the Cowboy web server when the app starts up.

We’ll do so with the Plug.Cowboy.child_spec/1 function.

This function expects three options:

  • :scheme - HTTP or HTTPS as an atom (:http, :https)
  • :plug - The plug module to be used as the interface for the web server. You can specify a module name, like MyPlug, or a tuple of the module name and options {MyPlug, plug_opts}, where plug_opts gets passed to your plug modules init/1 function.
  • :options - The server options. Should include the port number on which you want your server listening for requests.

Now that our server knows how to start up, let’s build out the GithubApiMockServer that is the interface for our web requests. We’ll give it some routes and teach it how to respond to certain web requests.

Building the Github API Mock Controller

In order for our controller to route web requests, we need to tell it to use Plug.Router This provides us the routing macros we need to match and respond to web requests.

Since our controller will be receiving JSON payloads (just like the GitHub API!), we’ll also tell it to run requests through the Plug.Parsers plug. This will parse the request body for us.

Now we’re ready to add our routes!

Defining Routes For Our Mock

Here we’ve defined a route POST /repos that uses a case statement to introspect on some params and send a happy or sad response.

Now that our mock server’s interface is defined to handle this request, let’s refactor our tests.

Cleaner Tests with Our Mock Server

  • Make the POST request to the /repos endpoint
  • Handle the response to return a GithubRepo struct or a GithubError struct.

Our code looks something like this:

We want to test that, when we successfully create a repo via the GitHub API, the function returns a GithubRepo struct. When we don’t successfully create a repo via the GitHub API, we return a GithubError struct. Instead of defining complicated function mocks inside our tests, we’ll write our nice clean tests with no awareness of any mocks.

In order for our tests to use our mock server, we need to make one simple change: tell the GithubClient module to send requests to our internally hosted endpoint in the test environment, instead of to the GitHub API.

To do that, we’ll *stop* hard-coding the value of the @base_url module attribute and instead make it an environment-specific application variable:

Now we can write tests that are totally agnostic of any mocking:

Ta-da!

Conclusion

By remember that by creating *mocks* (noun!) instead of *mocking* (verb!), we end up with clean, readable tests that don’t rely on implementation details to pass. So next time you’re faced with testing code that makes external web requests, remember that a simple hand-rolled mock server is now part of your tool belt.

Want to work on a mission-driven team that loves hand-rolled mock servers and trust-worthy tests? We’re hiring!

Footer top

To learn more about Flatiron School, visit the website, follow us on Facebook and Twitter, and visit us at upcoming events near you.

Flatiron School is a proud member of the WeWork family. Check out our sister technology blogs WeWork Technology and Making Meetup.

Footer bottom

Flatiron Labs

We're the technology team at The Flatiron School (a WeWork…

Sophie DeBenedetto

Written by

Flatiron Labs

We're the technology team at The Flatiron School (a WeWork company). Together, we're building a global campus for lifelong learners focused on positive impact.

Sophie DeBenedetto

Written by

Flatiron Labs

We're the technology team at The Flatiron School (a WeWork company). Together, we're building a global campus for lifelong learners focused on positive impact.

Medium is an open platform where 170 million readers come to find insightful and dynamic thinking. Here, expert and undiscovered voices alike dive into the heart of any topic and bring new ideas to the surface. Learn more

Follow the writers, publications, and topics that matter to you, and you’ll see them on your homepage and in your inbox. Explore

If you have a story to tell, knowledge to share, or a perspective to offer — welcome home. It’s easy and free to post your thinking on any topic. Write on Medium

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store