Elixir, GraphQL, and End-to-End Tests
"How do I test that?”
Usually, this is one of the first questions that comes to my mind when I’m starting to learn a new technology. And considering the type of systems that we have to handle modern day, It’s not hard to understand why.
These days, It’s very unusual to find any project where a big part of the effort isn’t centered around creating and maintaining a suite of tests that not only serves as documentation of the expected behavior of the application, but also helps to mitigate some of the risks of its constant evolution.
This is not different in the APIs development, but while for REST endpoints tests are an already established subject, there is still some discussion around its implementation for GraphQL APIs.
In this post, we’ll bring up a couple of approaches and considerations regarding test implementations for those APIs. In this case, we’ll focus in Elixir, but the discussion is extensible to other stacks as well.
GraphQL In The Elixir World
Even though the extensive documentation of GraphQL and its many implementations, the fact that it is a relatively new technology means that some aspects are not totally covered yet: tests implementation are one example.
For Elixir, we have Absinthe project that provides an implementation of the GraphQL specification together with a series of packages that brings an extensive set of tools with the goal of making the developer’s work easier.
The project is well documented, even going above and beyond by providing different examples for more advanced subjects such as authentication and file upload, but the implementation of tests is an area that is completely left in the developer’s account.
Running Queries/Mutations with Absinthe
When you combine this missing point with the different forms of executing a GraphQL query/mutation, we find a very fertile ground of questions for the developer that is implementing his first API.
Essentially we have three main ways of performing a query/mutation using Absinthe, and with that, three ways of testing your GraphQL API:
- Triggering a HTTP Post to your endpoint
This is maybe the approach closest to the ones usually used with traditional APIs. The idea here is to simulate a POST request to the endpoint and validate the results the same way that we would with a Rest API.
The difference here is only the details around the request and the fact that instead of a set of URIs representing the system resources, we have only one single endpoint that receives the POST request, and depending on the query provided, returns of the respective result.
Code wise, there is nothing surprising here in this test. As we need to build a Connection, we must include MyWebApp.ConnCase
and use post
and json_response
functions to trigger the request and check the result.
2. Using Absinthe.run/3 to execute the query/mutation
Another way of running a query is through Absinthe.run/3
function. With it, we can specify a query and the parameters to be executed in a specific Schema and we receive a tuple with the result.
In this version, the first change to be noted is the absence of the need to include ConnCase
module functions. We don’t need to trigger an HTTP request considering that the test focus is to verify the return of the run function. All the code related to the POST request and the json response verification gives place to a call to Absinthe.run
with the expected parameters to the test case.
3. Running the resolver
Another alternative would be go down an extra level and test only the resolver of que query/mutation under question. Considering that a resolver is nothing more than a function with a specific signature that transforms a GraphQL operation in data, a test like that don’t differentiate in any way from a test around a “normal” function.
And now? Which one to choose?
Among the different ways of implementing our tests, we have to make the call on which approach to follow. After researching a little bit, it is possible to find suggestions to avoid tests that run the Schema, and to move all the behavior to be tested into its own function, which would lead the third approach to be preferable.
Under a certain point of view, this suggestion makes sense. While testing only the resolver, we get rid of all the boilerplate related to the query execution that simplifies the test. What might go unnoticed with this is that the “simplification” has its own repercusion.
Different types of tests have different goals, which makes each of them work best for specific granularities. Unit and Integration tests strive to exercise isolated components or specific integrations, and that allows a higher granularity to be used for this type of tests..
In an end-to-end test, the goal is to exercise the interaction among all the components that works together in a specific scenario, and that means those tests have to be less granular.
In these cases, a test where we execute the HTTP request allows us to verify a series of interactions among the components of the API (Router, Middlewares, Resolvers, etc.) which is not possible in the other approaches.
In a nutshell, each approach has its own implications and must be selected based on its unique situation.
Summarizing…
Although existing recommendations to use less granular tests that only exercise on the resolver of your GraphQL API is not the more adequate form of implementing an end-to-end test, they bypass the integration among a series of different components that exist in the application.
For these cases, it is interesting to have a test that exercises all the components of the application as a way of verifying the system functionality when all the components are integrated. This justifies the cost of a test that handles HTTP requests and json responses.
The best way to handle the cost that comes from this type of test is to make sure that they represent just a small part of your test suite, focusing on checking only the happy path for all the flows, and maybe a few exceptional cases without, however, eliminating them from your test database.