Unit testing OpenAPI in Laravel
Introduction
Welcome to part three of our 10-part series. Before we continue, if you have not already read the previous parts, we recommend that you go back and read them here, this will help you keep track of what is going on.
A link to the github repository used, will be provided at the end of this article.
We are doing a three way test, see these articles if you want graphical explanations on why testing your OpenAPI specs together with your actual API is a good idea:
Before moving on, some changes have been made
Firstly, the string properties, $nameand $email, inside App/Forms/Userfom.php have been removed and placed inside App/DTOs/User.php. The same has been done with the Swagger annotations, these has also been moved and placed inside App/DTOs/User.php. As always you can check the commit for this article, a link will be provided at the end.
The reason being that our application was not functioning properly due to it trying and accessing $name and $email properties before they have been set, inside the savefunction inside our Userformclass, which resulted in an error of below type:
Typed property $our_property_name must not be accessed before initialization in fileTyped properties was introduced with PHP v.7.4, and what we can do with typed properties is that we can set a type for all class properties.
The PHPengine prevents anyone from setting a different type to the class property, once a type has already been set. This means in short that a type string cannot be set to an int and vice versa.
Typed properties and class properties have an uninitialized state, what this means is that the property simply is not initialized yet, and this is not the same as a property being null. If a type is not declared on a property, that property will have null as its uninitialized value.
We will not dive further into this topic, but all you should know is, that this is the main reason that we had to refactor our classes and properties, and added a new class under App/DTOs named User.php.
With the properties $nameand $email now being in App/DTOs/User.php and no longer inside our Userform, one “might” be wondering, that what these two, $this->name and $this->email, inside the save function refers to, and how they would be “working”:
namespace App\Forms;...class UserForm extends FormRequest { ... public function save()
{
...
$user->name = $this->name;
$user->email = $this->email;
...
}
}
The “key” that makes the clarification is this one, the $this keyword, as $this refers to the payload request properties. What this means is, that as long as $this points to a correct payload request property, our form request will then understand what we are assigning it to. If the request properties does not match with the properties that $this is pointing to, the form request will not understand what we are trying to attempt, and therefore fail.
Last but not least, inside UserController OpenAPI specs for both index and store. The Unauthorized response type has been removed. Obviously, most applications can return a http status code 401 on protected endpoints but at the moment we have not implemented authorization so our application will never return a http status code 401. Since the application does not do so, the OpenAPI spec file would be wrong if it assumed so and consumers using the spec file would build incorrect logic into their client
@OA\Response(response=401, description="Unauthorized")Testing our UserController
In this section we will walk you through the different tests that have been conducted. But before we do that we will give an explanation about what type of tests we want to conduct.
If you have not read any of the articles linked at the start of this article, we highly recommend reading them, as they give a proper explanation, since we will only in short explain what we are trying to achieve with the type of tests we are conducting.
We have the OpenAPI specs that describes our API, and we will test these OpenAPI specs in our tests, to make sure that our API behaves as the OpenAPI specs says they should behave. As a result of doing this, our OpenAPI specs become a reference for both our tests and our code, which makes the OpenAPI specs act as the APIs “only” truth source.
Now let us continue, we created a new test file using php artisan make:test UserTest in the terminal, where UserTest is the name of our test file.
The setUp function
When writing tests, you sometimes want to specify from the beginning what objects you would want to test against, for this, a template method setUp is used. This setUp method is invoked before the tests gets run.
setUp looks like this:
...class UserTest extends TestCase
{
protected function setUp(): void
{
parent::setUp(); ...
}
}
Testing that we get all users
With everything now set and done, let us dive in into the first test we need, and this of course is the test of our get method. This corresponds to the index function inside the UserController class.
What the purpose of this test is, that is to make sure that the responses we have specified in the OpenAPI specs actually matches, and that we actually do get those responses in return, when reaching the endpoint specified, the /api/users endpoint.
In the OpenAPI specs for this indexfunction, we have specified that, what we expect in return on a successful request, is a http status code 200 (OK). So what we need to test is that we get this http status code 200 (OK) in return. The test for this looks like following:
...class UserTest extends TestCase
{
private string $userEndpoint = "/api/users"; public function test_all_users_getting_retrieved()
{
$userIdsInDatabase = User::pluck('id');
$response = $this->json('GET', $this->userEndpoint);
$response->assertOk();
$userIdsInResponse = collect(json_decode($response->content()))->pluck('id');
$this->assertTrue($userIdsInResponse->diff($userIdsInDatabase)->isEmpty());
}
}
In the above test we retrieve first all user ids from the Users table, as we on top of asserting what type of http status code we get in return, also want to compare the users ids we get from the Users table directly, with the user ids we get in return from our get request attempt. As stated in the test name, we want to retrieve all users.
Then we reach out to our endpoint with the type of getrequest, and what we get in return, in Json format, we store in our $response property.
Furthermore to assert that we did get the correct response, http status code 200 (OK), we use Laravels assertOk method, which expects that the status code returned, is a http status code of type 200 (OK).
Then we start comparing the two lists we have, in both userIdsInDatabase and userIdsInResponse . We access the Json content by using $response->content() and pluck the ids. And since content() on our response object returns strings, we want to decode them, by the use of json_decode.
We use the collect helper to create a new Collection instance from the response object and gain access to Laravel functions such as diff and isEmpty which makes the test highly readable and easy, and store it inside userIdsInResponse and use PHPs diff function on arrays, to compare two arrays with each other. The diff function compares the values of two, or multiple, arrays, and returns the differences between them. difffunction returns an array that contains the entries from userIdsInResponsethat are not present in userIdsInDatabase.
And since we are looking for the two arrays that is getting compared, to have no differences in them, we assert that the array that gets created, as a result of the comparison, isEmpty() and assertTrue on that.
Testing serverside individually
The next test conducted is to test the store function from the UserController class. This test has the purpose of asserting that we actually store the incoming request on the serverside of the application. So what we want to test here is that we actually manage to create new users by posting to our endpoint.
How this test was approached is as following:
...class UserTest extends TestCase
{
private string $userEndpoint = "/api/users";
private $uniqueTimestamp; ... public function test_that_correct_payload_can_be_created_serverside()
{
$uniqueTimestamp = microtime(true);
$mockPayloadUser = [
'name' => "{$uniqueTimestamp}-fake-name",
'email' => "{$uniqueTimestamp}test@email.com",
];
$response = $this->postJson($this->userEndpoint, $mockPayloadUser);
$response->assertStatus(200); $this->assertDatabaseHas('users', [
'name' => $mockPayloadUser['name'],
'email' => $mockPayloadUser['email'],
]);
}
}
The uniqueTimestamp has the purpose of generating random numbers for us, so that once we run the test, we generate random numbers and prepend them to our nameand email values, to make sure we avoid “duplicate” entries on a test run, since we have specified that the emailfield in our database is unique and it cannot have duplicate entries, so we handle that by each testrun, we generate random numbers.
And what actually generates the numbers is PHPs function, microtime. microtimereturns the current Unix timestamp with microseconds.
Next we created some mock data, named $mockPayloadUser and set our properties, name and email as type string, as specified in the DTOs/User class, because to be able to post the payload correctly, we need to make sure that the request input properties matches.
We use Laravels helper function postJson to test our Json API and its response, we post the $mockPayloadUser as Json to our endpoint, and store that in a $response property. But we haven’t yet actually tested anything, so to be able to assert that on a successful post request, we get http status code 200 (OK), as specified in our store OpenAPI specs, we use Laravels assertStatus method, and specify that the expected value we expect from the $response post request, is http status code 200 (OK).
Furthermore we also need to assert that on a successful post request, we actually do post the $mockPayloadUser data into our Users database table. This is done by Laravels assertDatabaseHas function, which asserts that a table in the database, in this case the Userstable, contains records that matches the given key / value query constraints. In short, the first param expected is the table we check on, and the second param being the key / value query constraints.
Testing Swagger (OpenAPI) individually
To conduct this test, first we need to install the League OpenAPI Validator. To install the validator, use the following in your terminal:
composer require-dev league/openapi-psr7-validatorAfter the installation of the League OpenAPI Validator, make sure to import the following in your UserTest file.
...use League\OpenAPIValidation\PSR7\{OperationAddress, RoutedServerRequestValidator, ValidatorBuilder};
use League\OpenAPIValidation\PSR7\Exception\Validation\InvalidBody;class UserTest extends TestCase
{
... private OperationAddress $address;
private RoutedServerRequestValidator $validator; ...
}
The OperationAddress will match the given request, and this will be set in our setUpfunction, to what type of request we want to perform. And since we know the operation which should match our request, we can use the RoutedServerRequestValidator, which would simplify our validation.
The setUp function will end up having the following in it now:
...class UserTest extends TestCase
{
... protected function setUp(): void
{
... $this->address = new OperationAddress($this->userEndpoint, 'post');
$this->validator = (new ValidatorBuilder)
- >fromJson(file_get_contents('http://127.0.0.1:8000/api/swagger.json'))
->getRoutedRequestValidator(); ...
}
}
We make a new instance of ValidatorBuilder and get the Json content from our swagger.json to build a validator out of the Json content we get in return.
With the “setup” done we can move on to the test function, which we have build like this:
...class UserTest extends TestCase
{
... private array $mockPayload = [
'name' => 'fake_name',
'email' => 'fake_email@test.com'
]; public function test_that_swagger_validates_good_payload()
{
$request = $this->wrapServerRequest($this->mockPayload); try {
$this->validator->validate($this->address, $request);
$this->addToAssertionCount(1);
} catch (InvalidBody $e) {
$latestException = $e->getMessage();
$previousException = $e->getPrevious()->getMessage();
$exceptionLocation = implode(".", $e->getPrevious()->dataBreadCrumb()->buildChain());
$this->fail("$latestException $previousException $exceptionLocation");
}
}
}
We start with calling our own created helper function, named wrapServerRequest, this has the purpose of making the request call of our specified request type to the given endpoint we want to reach out to.
...
use GuzzleHttp\Psr7\{ServerRequest, Utils};class UserTest extends TestCase
{
private string $userEndpoint = "/api/users"; ... private function wrapServerRequest(array $payload): ServerRequest
{
return (new ServerRequest('post', $this->userEndpoint))
->withHeader('Content-Type', 'application/json')
->withBody(Utils::streamFor(json_encode($payload)));
}
}
In this private defined function, we have specified that our request is of type post and the endpoint we want to reach is $this->userEndpoint. The param it accepts will be our payload array.
Our test, test_that_swagger_validates_good_payload()will assert for two scenarios, one being a successful postrequest and the second being a failed post request. In the successful scenario we want to assert that the $mockPayload array that we sent with our private wrapServerRequest function has been added, or rather posted, we want to reach a positive result at that point and a standard way of doing that is to addToAssertionCount, which is more readable than something like assertTrue(true). The purpose of the code in the catch is built up, so that on a potential error, the error description gets written out very clear and understandable, makes it easier to handle the error.
Testing that serverside and swagger fails, together
For this test, we want to assert that both serverside and Swagger fails together, considering different scenarios which “potentially” could occur. We can’t have that one side fails, but the other does not, because logically that would, and should not, make logically sense. Therefore this test was approached with a different mindset.
In this test we have started with creating a dataprovider. A dataprovider is a PHPUnit entity that helps you to easily control a highly similar test, which differences can be parameterized into a few variables.
With that said, our dataprovider is as following:
...class UserTest extends TestCase
{
... public function createStatusProvider()
{
return [
'Incorrect name' => [422, array_merge($this->mockPayload, ['name' => 1234]), 'name'], 'Incorrect email' => [422, array_merge($this->mockPayload, ['email' => 'asdasd']), 'email'], 'Missing email' => [422, ['name' => $this->mockPayload['name'] ], 'email'], 'Missing name' => [422, ['email' => $this->mockPayload['email'] ], 'name'],
];
}
}
We have created this dataprovider to test for all those “potential” errors that a postrequest can respond with. And remember, we are testing the OpenAPI specs specified for our UserController store function, where we have specified in the OpenAPI specs, that if we can’t get a successful post request response, we have to get a response of http status code 422 (Unprocessable Entity). The different scenarios we test for are in the return statement.
We have tried to with the naming of our test cases, inside the returnstatement, to state what purpose they each serve, so we will not dive into them one and one and so on.
The test that makes use of this dataprovider is the following:
...class UserTest extends TestCase
{
... /**
* @dataProvider createStatusProvider
*/
public function test_that_swagger_and_serverside_fails_if_payload_is_not_correct($status, array $payload, string $expectedKeyOfFailure)
{
$initialCount = User::all()->count(); $request = $this->wrapServerRequest($payload); try {
$this->validator->validate($this->address, $request);
} catch (InvalidBody $e) {
$latestException = $e->getMessage();
$previousException = $e->getPrevious()->getMessage();
$exceptionLocation = implode(".", $e->getPrevious()->dataBreadCrumb()->buildChain());
$expectedKeyOfFailure == $exceptionLocation ? $this->addToAssertionCount(1) : $this->fail("$latestException $previousException $exceptionLocation");
}$response = $this->postJson($this->userEndpoint, $payload);
$response->assertStatus($status, $response);
$response->assertJsonValidationErrors([$expectedKeyOfFailure]);
$this->assertDatabaseCount('users', $initialCount);
}
}
To make sure that our test makes use of our createStatusProvider, we have to use the @dataProvider annotation above our test. This creates a reference to our createStatusProvider. We first want to test what type of response we get in return when trying to test our Swagger side, and for this we almost repeat the process just like in the test_that_swagger_validates_good_payload test. Except this time we want to end in the catch statement.
The $status represents the status response provided by createStatusProvider and the $response represents the response returned on the post request.
The $expectedKeyOfFailure represents the missing property that is behind the error throwing, the $expectedKeyOfFailure property is defined inside createStatusProvider return statement. If $expectedKeyOfFailure and $exceptionLocation matches, then we increment an assertion, else we “fail” the entire request and send in the type of exceptions that was thrown, of which we did not want to assert for.
Furthermore we also need to test our serverside, we need to assert that we don’t actually postanything into our Users database table. To do this, since we remove a “key”, we can’t use Laravels assertDatabaseHas method optimally, since we want to assert for each name and email properties missing individually. So rather we will test by seeing if the count of Users entries in the database table increments or not. First we make use of a Laravel eloquent approach, to communicate with our Users table, as Laravels eloquent makes interacting with the database easier and more smooth.
We get the count of the entries that our Users table has by writing User::all()->count() and we store that count inside a variable by the name $initialCount . From there we do two different assertions, one for the http status code response, to test the respone of our OpenAPI specs, and the other where we match our database Users entries counts with each other. assertStatus is tasked with comparing the expected and actual http status code values, while assertDatabaseCount matches the $initialCount with the current count of entries inside the Users table after an attempt of post has been tried.
Github repository for this article can be found here.
