Testing software with JUnit 5, Spring and Pitest.
One of the most satisfying things in our job as a dev is being able to deliver the feature promised in the sprint on time and seeing it in production generating value and impact for the company. That is until… a BUG appears! The uncomfortable feeling becomes worse when we realize that it could have been avoided. How can I ensure greater reliability for my software through testing?
When we talk about developing a software we often tend to worry about deadlines, product definitions, scopes, features, technical implementation and performance. Sadly, very often I see the neglect of testing while developing a software. In this journey as a software developer, I have already been asked the following questions: “Emerson, what is the importance of testing?”, “Emerson, why should I test my software when we can test it as soon as it is deployed?”
After all, what is software testing? The reality is that there are many types of tests and testing methodologies. We could mention here, unit tests, contract tests, integration tests, automated tests, end-to-end tests, among other types. The important thing here is that although each one has its specific purpose, they all aim at one goal: to ensure that a certain behaviour is being fulfilled. In other words, ensuring that the software works as intended.
“But wait, would anyone design a software to not work as it was initially designed? That sounds insane to me.” Yes, it’s insane and I agree that likely no one would do it on their own free will, but we know that in this world of development there are many unforeseen events, among them the famous and headache source: bugs. And even if we are as good as we could, we are still humans and we make mistakes. Whether they are misunderstandings or technical errors, testing helps us to ensure that these errors do not go unnoticed in production and negatively impact the final product.
Speaking of a corporate scenario in a project with multi-collaborators, the tests ensure that everyone involved in the project maintains the same line of development thinking and avoids that Somebody A ends up accidentally modifying a critical behavior in one of the flows designed by Somebody B. As an example, imagine the following situation: Somebody B implements a flow where the code checks if the user is an eligible user of a discount promotion that the company is doing as a marketing campaign, and if he is eligible he will have a 50% discount on all purchases from the website. However, due to a lack of attention, Somebody A, new to the company, ends up deleting this validation from the code, after finishing his task thinking that everything was ok and because the software didn’t have tests, he pushes his modifications in production. Now all site users have a 50% discount on all purchases.
Does the situation above remind you of anything that happened in real life? As we see above, if there was a test to verify eligibility validation this problem would never have happened. If we adapt this situation to our day-to-day reality, we will realize that this problem is more common and happens more often than we imagine. And this is the importance of having very well tested software.
For this article we are going to focus on unit testing and integration tests using testcontainers and hibernates. For this we are going to use a project with a simple domain definition. You can find the example source project here in my GitHub repository. This project have two APIs, one for creating and listing a user, and the second one to create and list a task for a given user. You can see bellow a generic sequential diagram for this project.
So for this project we have a variation of a Multilayered project. We are going to go through each layer and understand how to test each one of them, because there are different approaches and considerations that we have to be careful with.
Core Layer: Deep Dive of Tests for the Repository Interfaces (Integration with Hibernates and Testcontainers)
One thing very important that we need to keep in mind is that each application layer must be tested differently according to its purpose. For the persistence layer, we must first understand what actions we expect to be performed by this layer.
In this example project, we have two repository interfaces UserRepository and TaskRepostiory. We are using Spring JPA with Hibernates to persist in PostgreSQL database. What we want to test in these two repositories is if the queries configured for hibernates are running correctly.
We will now focus on UserRepository , before we start we should ask ourselves: what should I test? Well, let’s think about the possible scenarios together. In this repository, for my business domain to work properly, I need two main actions: Create a user and List a user by email. With that we already have two test scenarios, one to validate if a user is being successfully persisted in the database and another one to retrieve a user from the database by email.
So let’s start coding our test class. If you would like to configure the testcontainer in your project, please follow the instructions on the official website.
The first thing to do is to create our test class and inject the repository instance that was generated by Spring Data:
@IntegrationTestsConfiguration
public class UserRepositoryTest {@Autowired
private UserRepository userRepository;}
Now let’s write a test to verify if our repository was successfully injected:
@Test
public void contextLoads() {
assertAll(
() -> assertThat(this.userRepository,is(notNullValue()))
);
}Let’s create two classes inside our test class, CreateUser and ListUser both will be annotated with @Nested. Creating these classes helps us to keep tests organized and grouped by action type, as the project grows it will be easier to maintain and/or create new tests within that context.
@Nested
@IntegrationTestsConfiguration
class CreateUser {
}@Nested
@IntegrationTestsConfiguration
class ListUser {
}
Note that both nested classes were annotated with our custom configuration annotation for tests:@IntegrationTestsConfiguration. This is because unfortunately, nested classes do not inherit the annotations of the parent class and we need these annotations for our tests to work. You can see here how this annotation was created and what configurations it has.
Now let’s think about our first test scenario: User creation. How can we write this testing code? Creating a single object with fixed data and saving to the database would be enough for this test? Probably yes, but would be much more susceptible to have a bug in the future since our tests wouldn’t be covering many scenarios and possibilities. The interesting thing here would be to simulate as much as possible the data that will be provided in the production environment. Thinking this way, it would be interesting to create more than one object with data diversification, for that we will use a JUnit 5 feature called: Parameterized Tests.
Parameterized tests make it possible to run a test multiple times with different arguments.
That’s exactly what we’re looking for. The advantage of using Parameterized Tests is the versatility that is possible when providing arguments to the test method. We can provide a CSV file, Strings, we can extend and create a class to generate complex objects, there are N possibilities that will surely suit the needs of the tests. And we will show some of these ways throughout this article. I do recommend reading the official documentation to have a deeper understanding of all the possibilities.
Now let’s write our scenarios using Parameterized Tests. First let’s create our method and annotate it with @ParameterizedTest .
@ParameterizedTest
public void createValidUsers() {}
Now we need to create a CSV file to provide data as an argument for our method to use. The files that we will create along the article will be placed inside the test resources folder.
EMAIL,FIRST_NAME,LAST_NAME,FULL_NAME,ID
vincenzo@test.com,vincenzo,marshall,vincenzo marshall,2
shanai@test.com,shanai,armstrong,shanai hilton armstrong,3
cheyanne@test.com,cheyanne,kelley,cheyanne connelly kelley,4
ajwa@test.com,ajwa,thompson,ajwa thompson,5
test1@gmail.com,test,one,test one,1
test2@something.any,lorem,ipsum,lorem ipsum,6Now that we have our CSV file, let’s configure it so that JUnit knows where to look for data. Remembering that each column of the CSV is equivalent to a method argument and that we must skip a line in the file corresponding to the header. We can have more columns than method arguments, but never the opposite.
@ParameterizedTest
@CsvFileSource(resources = "/csv/user/UserRepositoryValidEntries.csv", numLinesToSkip = 1)
public void createValidUsers(final String email,
final String firstName,
final String lastName,
final String fullName) {}
Finally, let’s create the test logic for this scenario. First let’s create an entity with the data provided, so that we can persist this object in the database. After that we will compare the values that were persisted in the database with the original object values, all fields must equals.
@ParameterizedTest
@CsvFileSource(resources = "/csv/user/UserRepositoryValidEntries.csv", numLinesToSkip = 1)
public void createValidUsers(final String email,
final String firstName,
final String lastName,
final String fullName) {final var user = UserEntity.builder()
.email(email)
.firstName(firstName)
.lastName(lastName)
.fullName(fullName)
.build();final var result = userRepository.save(user);assertAll(
() -> assertNotNull(result),
() -> assertEquals(email, result.getEmail()),
() -> assertEquals(firstName, result.getFirstName()),
() -> assertEquals(lastName, result.getLastName()),
() -> assertEquals(fullName, result.getFullName())
);
}
Note that assertAll() was used instead of using multiple assertions. One of the great advantages of this type of assertion is that it is guaranteed to execute all assertions declared within its scope. In the case of multiple assertions, if the first assert fails, the others will not be executed, this ends up omitting to the developer the faults that the method to be tested contains. Look at the example below:
@Test
void usingMultipleAssertions() {
assertEquals(1, 1);
assertEquals(1, 2);
assertEquals(2, 2);
assertEquals(3, 1);
}Looking at the example above we know that the second and fourth assertEquals will fail. But look at the output of this test:
org.opentest4j.AssertionFailedError:
Expected :1
Actual :2
<Click to see difference>Now let’s execute these same assertions inside an assertAll() .
@Test
void usingMultipleAssertions() {
assertAll(
() -> assertEquals(1, 1),
() -> assertEquals(1, 2),
() -> assertEquals(2, 2),
() -> assertEquals(3, 1)
);
}Observe the generated output:
expected: <1> but was: <2>
Comparison Failure:
Expected :1
Actual :2
<Click to see difference>expected: <3> but was: <1>
Comparison Failure:
Expected :3
Actual :1
<Click to see difference>org.opentest4j.MultipleFailuresError: Multiple Failures (2 failures)
org.opentest4j.AssertionFailedError: expected: <1> but was: <2>
org.opentest4j.AssertionFailedError: expected: <3> but was: <1>
In the first example using multiple assertions we had an omitted flaw in assertEquals that can lead the developer to make a mistake when trying to fix the first flaw or have to go back to the code again to fix this failed case after fixing the first one, doubling the development efforts. In both cases we have a negative situation for the developer. Another positive point of assertAll is that we can organize all test validations explicitly inside its scope, this ensures that there are much less incidences of someone accidentally deleting an important validation.
My advice is that whenever you come across a situation where more than one assertion is used, choose to replace it to assertAll().
Getting back to our test, finally, let’s add a custom name for this test, for that we just need to open parentheses in the @ParameterizedTest annotation and add our string template. We can use the values of the arguments provided to the method based on the order of the CSV columns. Finally, our test method will look like this:
@ParameterizedTest(name = "#[{index}] Should assert equals for parameters saved in database with values:" +
" email = {0} | firstName = {1} | lastName = {2} | fullName = {3}")
@CsvFileSource(resources = "/csv/user/UserRepositoryValidEntries.csv", numLinesToSkip = 1)
public void createValidUsers(final String email,
final String firstName,
final String lastName,
final String fullName) {final var user = UserEntity.builder()
.email(email)
.firstName(firstName)
.lastName(lastName)
.fullName(fullName)
.build();final var result = userRepository.save(user);assertAll(
() -> assertNotNull(result),
() -> assertEquals(email, result.getEmail()),
() -> assertEquals(firstName, result.getFirstName()),
() -> assertEquals(lastName, result.getLastName()),
() -> assertEquals(fullName, result.getFullName())
);
}
As a result, we would have an output like this:
UserServiceTest > CreateUser > #[1] Should assert equals for expected values based on name: Mae Algernon email = mavboy@hiowaht.com | expectedFirstName = mae| expectedLastName = algernon | expectedFullName = mae algernon PASSED.
.
.UserServiceTest > CreateUser > #[7] Should assert equals for expected values based on name: Mariyam Gaetano email = coimukatauba-2490@yopmail.com | expectedFirstName = mariyam| expectedLastName = gaetano | expectedFullName = mariyam gaetano PASSED
But just creating a test to validate a valid user is not enough, we need to ensure that our database constraints are working as expected. For this we will create a failure scenario test. This type of scenario is extremely important, as it ensures that application rules are being activated correctly and that our software is failsafe if invalid values are given to it.
For this scenario, we want PostgreSQL to complain about a constraint violation when trying to save invalid users and consequently throw the DataIntegrityViolationException . We will create a new CSV with invalid data to use on @ParameterizedTest.
EMAIL,FIRST_NAME,LAST_NAME,FULL_NAME
,vincenzo,marshall,vincenzo marshall
shanai2@test.com,,armstrong,shanai hilton armstrong
cheyanne2@test.com,cheyanne,,cheyanne connelly kelley
ajwa2@test.com,ajwa,thompson,
,,,
test1@gmail.com,test,one,test oneWe will also, for this test, prepopulate the database on startup. This is because we want the database to complain when we try to save data with existing values, violating the Unique constraint. We will use@SqlGroup annotation that allows us to use more than one@Sql annotations. This last annotation allows us to execute SQL scripts at predefined times in the ExecutionPhase . Let’s create two new files inside a new folder called scripts and inside it a new folder called user .
The first file can be named asBeforeUserRepositoryTest.sql
insert into "user" (id, created_at, email, first_name, last_name, full_name) values (1, now(), 'test1@gmail.com', 'test', 'one', 'test one');
insert into "user" (id, created_at, email, first_name, last_name, full_name) values (2, now(), 'vincenzo@test.com', 'vincenzo', 'marshall', 'vincenzo marshall');
insert into "user" (id, created_at, email, first_name, last_name, full_name) values (3, now(), 'shanai@test.com', 'shanai', 'armstrong', 'shanai hilton armstrong');
insert into "user" (id, created_at, email, first_name, last_name, full_name) values (4, now(), 'cheyanne@test.com', 'cheyanne', 'kelley', 'cheyanne connelly kelley');
insert into "user" (id, created_at, email, first_name, last_name, full_name) values (5, now(), 'ajwa@test.com', 'ajwa', 'thompson', 'ajwa thompson');
insert into "user" (id, created_at, email, first_name, last_name, full_name) values (6, now(), 'test2@something.any', 'lorem', 'ipsum', 'lorem ipsum');The second file can be named asAfterUserRepositoryTest.sql
truncate table "user" cascade;Finally, our test scenario will look like this:
@ParameterizedTest(name = "#[{index}] Should throw exception [DataIntegrityViolationException] for invalid " +
"parameters values: email = {0} | firstName = {1} | lastName = {2} | fullName = {3}")
@CsvFileSource(resources = "/csv/user/UserRepositoryTestInvalidEntries.csv", numLinesToSkip = 1)
@SqlGroup({
@Sql(executionPhase = Sql.ExecutionPhase.BEFORE_TEST_METHOD,
scripts = "classpath:scripts/user/BeforeUserRepositoryTest.sql"),
@Sql(executionPhase = Sql.ExecutionPhase.AFTER_TEST_METHOD,
scripts = "classpath:scripts/user/AfterUserRepositoryTest.sql")
})
public void createInvalidUsers(final String email,
final String firstName,
final String lastName,
final String fullName) { final var user = UserEntity.builder()
.email(email)
.firstName(firstName)
.lastName(lastName)
.fullName(fullName)
.build(); assertThrows(DataIntegrityViolationException.class, () -> userRepository.save(user));
}
Now that we’ve created our success and failure scenarios for user creation, let’s implement our tests for listing a user by email. For this we will use the features already shown above. However, in the validation we want to guarantee that the returned object is not null or is present (in the case of optionals) and that the values of this object are equals to the values previously registered. Also we need to populate the database on test startup, that is because due to the nature of this query we need to have data in the database. We are going to use the same CSV that we used for the first test scenario of creating user. So our test method will look like this:
@ParameterizedTest(name = "#[{index}] Should assertNotNull and assertEquals for all entity properties when" +
" find by email, parameters values: email = {0} | firstName = {1} | lastName = {2} | fullName = {3}")
@CsvFileSource(resources = "/csv/user/UserRepositoryValidEntries.csv", numLinesToSkip = 1)
@SqlGroup({
@Sql(executionPhase = Sql.ExecutionPhase.BEFORE_TEST_METHOD,
scripts = "classpath:scripts/user/BeforeUserRepositoryTest.sql"),
@Sql(executionPhase = Sql.ExecutionPhase.AFTER_TEST_METHOD,
scripts = "classpath:scripts/user/AfterUserRepositoryTest.sql")
})
public void findUserByEmail(final String email,
final String firstName,
final String lastName,
final String fullName) { final var result = userRepository.findByEmail(email); assertAll(
() -> assertTrue(result.isPresent()),
() -> assertEquals(email, result.get().getEmail()),
() -> assertEquals(firstName, result.get().getFirstName()),
() -> assertEquals(lastName, result.get().getLastName()),
() -> assertEquals(fullName, result.get().getFullName())
);
}
For database read tests we should also create failure scenarios, if any exists. When creating these tests, ask yourself which scenarios characterize failure. For this case above where we look for a user by their email, what would be the failure cases? When we pass an email to this query there are only two options or the email exists or it doesn’t exist, if it doesn’t exist an empty optional object would be returned, is it considered failure? Just to remember that emails with invalid formatting are indifferent to the database that makes a string match in where clause. We will validate emails in the controller layer.
For this scenario, we won’t consider an empty optional as a failure, but as expected. So let’s add a test to check if the optional object is coming empty when we pass a non-existent email to the database. We will use a new type of Argument Source @ValueSource. This annotation allows us to provide an array of a determined type, it can be strings, longs, doubles, bytes , and others types. For this case, we will provide a string array containing the non-existent emails.
@ParameterizedTest(name = "#[{index}] Should assertNull when" +
" find by email, parameters values: email = {0}")
@ValueSource(strings = { StringUtils.EMPTY, "shanai2@test.com", "cheyanne2@test.com", "ajwa2@test.com" })
@SqlGroup({
@Sql(executionPhase = Sql.ExecutionPhase.BEFORE_TEST_METHOD,
scripts = "classpath:scripts/user/BeforeUserRepositoryTest.sql"),
@Sql(executionPhase = Sql.ExecutionPhase.AFTER_TEST_METHOD,
scripts = "classpath:scripts/user/AfterUserRepositoryTest.sql")
})
public void findUserByNonExistentEmail(final String email) {
final var result = userRepository.findByEmail(email);
assertTrue(result.isEmpty());
}Following this logic you can add as many tests as needed to your repository. Click here for the complete class code, with additional tests to list all database users and list a user by Id.
Core Layer: Deep Dive of Tests for Service Classes (Argument Captor)
Now that we’ve tested our repository, let’s focus on testing our service. In this layer, unlike the one tested previously, we don’t want to validate the return of the database, but the data manipulation. For example, when we look at the user creation method in the UserService.java class, we see that a lot of validation and data manipulation takes place. You can view the full code for this class here.
public UserDTO createUser(final UserDTO userDTO) {
log.info("Creating user with information: {}", userDTO); var name = userDTO.getName(); if (name == null || !name.trim().contains(" ")) {
throw new InvalidNameException(name);
} userDTO.setName(name.trim().toLowerCase()); final var userEntity = userMapper.toEntity(userDTO); extractFirstAndLastNameToEntity(userEntity); final var result = userRepository.save(userEntity); return userMapper.toDto(result);
}
What we’re going to focus on tests for this service class is whether these validations and manipulations are being effective and if they’re beign executed. For our first scenario we want to verify that when we pass a valid full name it can handle and fill in the fields correctly.
Let’s create our test class for this service and configure it to use mockito. We will follow the same organization that we used in the repository tests. But we will configure our mocks and add a new field of type ArgumentCaptor with the @Captor annotation. I will explain both later. In addition, we will use Faker to generate realistic random data for our tests. You can view the full code for this test class here.
@ExtendWith(MockitoExtension.class)
public class UserServiceTest { @Mock
private UserRepository userRepository; @Spy
private UserMapper userMapper; @InjectMocks
private UserService userService; @Captor
private ArgumentCaptor<UserEntity> userArgCaptor; private final Faker faker = new Faker(); @Nested
class CreateUser {
}@Nested
class ListUser {
}
Let’s create our CSV as follows:
EMAIL,NAME,EXPECTED_FIRST_NAME,EXPECTED_LAST_NAME,EXPECTED_FULL_NAME
mavboy@hiowaht.com,Mae Algernon,mae,algernon,mae algernon
bruhecregrudda-6582@yopmail.com,Altansarnai Yaxkin Citlali,altansarnai,citlali,altansarnai yaxkin citlali
buduheicrommo-3109@yopmail.com,volodislavu zlata jasper,volodislavu,jasper,volodislavu zlata jasper
coimukatauba-2497@yopmail.com,Mariyam Gaetano,mariyam,gaetano,mariyam gaetano
test@gmail.com,Peeter Tivadar Éibhear Matt,peeter,matt,peeter tivadar éibhear matt
test@gmail.com, Teste Tivadar Éibhear Matt,teste,matt,teste tivadar éibhear matt
coimukatauba-2490@yopmail.com,Mariyam Gaetano ,mariyam,gaetano,mariyam gaetanoThis time in the CSV we are defining the expected values to validate in our assertions. Now let’s create our test code, notice that we are using mockito to simulate a database object call and return a mocked object. In addition, we will use an argument captor to capture the values passed as arguments to the repository mock.
@ParameterizedTest(name = "#[{index}] Should assert equals for expected values based on name: {1}" +
" email = {0} | expectedFirstName = {2}| expectedLastName = {3} | expectedFullName = {4}")
@CsvFileSource(
resources = "/csv/user/UserServiceTestValidEntries.csv", numLinesToSkip = 1
)
public void validateCreationWithValidParameters(final String email,
final String name,
final String expectedFirstName,
final String expectedLastName,
final String expectedFullName) { final var userDto = UserDTO.builder()
.email(email)
.name(name)
.build();final var userEntity = userMapper.toEntity(userDto); when(userRepository.save(any())).thenReturn(userEntity); final var result = userService.createUser(userDto); verify(userRepository).save(userArgCaptor.capture()); assertAll(
() -> assertNotNull(result),
() -> assertEquals(userEntity.getFullName(), result.getName()),
() -> assertEquals(userEntity.getEmail(), result.getEmail()),
() -> assertEquals(expectedFirstName, userArgCaptor.getValue().getFirstName()),
() -> assertEquals(expectedLastName, userArgCaptor.getValue().getLastName()),
() -> assertEquals(expectedFullName, userArgCaptor.getValue().getFullName())
);}
Note how we used our Argument Captor for this scenario. As mentioned above, Argument Captor is a feature of mockito that allows us to capture the value of the object that was passed as an argument to a mocked method. In this case, we mocked the method save from our repository. With that, when the test are executed, instead of calling a concrete implementation from our repository it just returns a previously defined value.
When we call the createUser method of our service class, the object initially passed as an argument go through several changes within this method. First we have a type conversion where is instantiated an object of type Entity with its data, after the user’s name go through a “cleaning” of spaces in the trim() method. We also have two fields firstName and lastName, which are set based on the value of the name field. With ArgumentCaptor we were able to verify, in the call to persist the object, if all of these conditions were met.
Now let’s create our failure scenario to verify if the InvalidNameException , while creating a user, is being thrown successfully. For this we will create a CSV with invalid values.
EMAIL,NAME
mavboy@hiowaht.com,
bruhecregrudda-6582@yopmail.com,Citlali
buduheicrommo-3109@yopmail.com,
,
test@gmail.com, Éibhear
coimukatauba-2490@yopmail.com,MariyamNow we just need to check if the exception was thrown, our test method looks like this:
@ParameterizedTest(name = "#[{index}] Should throw exception [InvalidNameException] for invalid name values:" +
" email = {0} | name = {1}")
@CsvFileSource(resources = "/csv/user/UserServiceTestInvalidEntries.csv", numLinesToSkip = 1)
public void validateCreationWithInvalidParameters(final String email, final String name) {
final var userDto = UserDTO.builder().email(email).name(name).build(); assertThrows(InvalidNameException.class, () -> userService.createUser(userDto));
}
Now let’s create our tests for the user listing by email scenario. In this scenario, what we want is to ensure that the email initially provided as an argument is being propagated with the same values to the repository layer. Let’s reuse the CSV created for the first success scenario of this layer, so our test will look like this:
@ParameterizedTest(name = "#[{index}] Should call repository with entity and its email value must match " +
"email = {0} and should return not null")
@CsvFileSource(resources = "/csv/user/UserServiceTestValidEntries.csv", numLinesToSkip = 1)
public void shouldReturnNotNull(final String email) {
final var emailArgumentCaptor = ArgumentCaptor.forClass(String.class); when(userRepository.findByEmail(email)).thenReturn(Optional.of(buildMockRandomEntity(email))); final var result = userService.findUserByEmail(email); verify(userRepository).findByEmail(emailArgumentCaptor.capture()); assertAll(
() -> assertNotNull(result),
() -> assertEquals(email, emailArgumentCaptor.getValue())
);
}private UserEntity buildMockRandomEntity(final String email) {
final var name = faker.name();
return UserEntity.builder()
.email(faker.bothify(email))
.firstName(name.firstName())
.lastName(name.lastName())
.fullName(name.fullName())
.build();
}
Finally, let’s write our last test for this service. We want to validate if when the repository returns an empty Optional our method to list user by email returns a null value. The test will then look like this:
@Test
@DisplayName("Should return null value for empty optional returned from repository")
public void shouldReturnNullForEmptyOptional() {
final var email = faker.bothify("?????#####@email.com");
when(userRepository.findByEmail(email)).thenReturn(Optional.empty());
final var result = userService.findUserByEmail(email);
assertNull(result);
}}
The service layer is a very smooth layer to test, but it needs attention to test it correctly. Always remember that we have already tested the return of values from the database in the repository layer. What we really need to test are the validations done on the services before calling the repository. Keeping this line of thinking in mind, we will be able to create much more effective service tests.
Api Layer: Deep Dive of Tests for Controller Classes (Status code response)
Finally we arrive at the entry point of our requests, the controllers. See that at this point we have already been able to test our repositories and our services, now we need to test our controllers and its behaviours.
In the case of controllers, they are composed of endpoints and they are what we want to focus on in our tests. We will take as an example our UserController that has two endpoints: A POST to create a new user and a GET to list a user by email. So let’s think, how can I test these endpoints?
Starting with our user creation context, we can think of two scenarios: The happy scenario where the fields provided as the request body are all valid and consequently we can persist this user in the database, and the sad scenario where some or all of the fields provided in the request body are invalid. That said, the way our endpoint communicates to the client if an operation was successful or not is through the HTTP Status Code. And that’s what we’re going to work on in our tests, if our endpoint is returning the correct Status code.
Let’s start by creating our UserControllerTest class, following the same structural organization as the others test classes we created. We will also import the MockMvc that we will use to mock our requests. Also, don’t forget to mock our UserService .
@ControllerTestsConfiguration
public class UserControllerTest {
@Autowired
private MockMvc mockMvc;
@MockBean
private UserService userService;
private static final String REQUEST_URL = "/user";
@Nested
@DisplayName("[POST] /user/")
class CreateUser {
}
@Nested
@DisplayName("[GET] /user/")
class GetUser {
}See that in the example above we used the @DisplayName in the nested class. This is interesting when we are testing endpoints organized by nested classes to indicate in the output what was the type and path of the request tested.
When we use MockMvc to make mocked requests to our endpoint, in the case of the POST method we need to provide a valid JSON that will be converted to the contract object in the endpoint. In order to be able to do this using @ParameterizedTest I decided to extend the functionality of ArgumentSource and create my own custom source. Let’s see now how we can do this.
First we need to create a new annotation which we will use in our tests. Let’s create one and name it RandomJSONSource .
@Target({ ElementType.ANNOTATION_TYPE, ElementType.METHOD })
@Retention(RetentionPolicy.RUNTIME)
@Documented
@API(status = EXPERIMENTAL, since = "5.0")
@ArgumentsSource(RandomDTOJsonProvider.class)
public @interface RandomJSONSource {
/**
* Number of objects that will be generated to be provided to {@link ParameterizedTest}.
*/
long interactions();
/**
* Class that implements {@link BaseDtoJsonBuilder}, will be used to generate de Object
*/
Class<? extends BaseDtoJsonBuilder> targetBuilder();
/**
* Flag to produce invalid entries based on {@link BaseDtoJsonBuilder#buildInvalidJsonObject()}.
*/
boolean invalidFields() default false;
}In this annotation some fields were created that will help us in the configuration and creation of our objects which will be provided as parameters. Now let’s create our RandomDTOJsonProvider class which will be responsible for creating these objects.
public class RandomDTOJsonProvider implements ArgumentsProvider, AnnotationConsumer<RandomJSONSource> {
@Override
public void accept(RandomJSONSource randomJSONSource) {
}
@Override
public Stream<? extends Arguments> provideArguments(ExtensionContext context) throws Exception {
return null;
}First we need to make this new class implement the ArgumentsProvider and AnnotationConsumer<RandomJSONSource> interfaces, in addition we will override the accept and provideArguments methods. The accept method is where we are going to access the instance of our annotation with the values of the fields filled in, we will use this method to make a copy of these values and use it later on in this class.
public class RandomDTOJsonProvider implements ArgumentsProvider, AnnotationConsumer<RandomJSONSource> {
private Class<?> target;
private long listOfObjectsSize;
private boolean invalidFields;
@Override
public void accept(RandomJSONSource randomJSONSource) {
this.listOfObjectsSize = randomJSONSource.interactions();
this.target = randomJSONSource.targetBuilder();
this.invalidFields = randomJSONSource.invalidFields();
}
@Override
public Stream<? extends Arguments> provideArguments(ExtensionContext context) throws Exception {
return null;
}Now we need to configure the provideArguments method which is responsible for returning a Stream of arguments which will be transmitted as values for the parameters requested in our test method. But before that we need to code our builder that will be responsible for creating our user objects and their respective JSON. So let’s create the UserDTOJsonBuilder class. You can view full code for this class here.
@Service
public class UserDtoJsonBuilder implements BaseDtoJsonBuilder<UserDTO> {
private static final String PREFIX_EMAIL_TEMPLATE = "??????####%s";
private final Faker faker;
public UserDtoJsonBuilder() {
faker = new Faker();
}
@Override
public Map<UserDTO, JSONObject> buildValidJsonObject() throws JSONException {
final var randomEmail = faker.bothify(String.format(PREFIX_EMAIL_TEMPLATE, "@gmail.com"));
final var randomName = generateRandomNamesThatMayContainMiddleName();
return buildObject(randomEmail, randomName);
}
@Override
public Map<UserDTO, JSONObject> buildInvalidJsonObject() throws JSONException {
final var randomEmail = faker.bothify(String.format(PREFIX_EMAIL_TEMPLATE, StringUtils.EMPTY));
final var randomName = generateRandomInvalidAndValidNames();
return buildObject(randomEmail, randomName);
}Unfortunately when using this approach of extending Argument Source, I was not able to use Spring’s dependency injection to inject our builders directly into the provider. If you know how to achieve this and wants to share it, please feel free to comment it in the comment section.
To be able to get the builder class based on the type provided it was necessary to create a factory class and instantiate our builders directly in it. This is how our factory class DtoJsonBuilderFactory turned out:
private static final Collection<BaseDtoJsonBuilder> buildersInstance = createInstances();
public static Optional<BaseDtoJsonBuilder> getBuilder(Class<?> type) {
return buildersInstance.stream().filter(builder -> builder.getClass().equals(type)).findFirst();
}
private static Collection<BaseDtoJsonBuilder> createInstances() {
return Arrays.asList(
new TaskDtoJsonBuilder(),
new UserDtoJsonBuilder(),
// Instantiate here all DtoJsonBuilders that you've created
);
}Now let’s go back to our RandomJSONProvider class and configure the provideArguments method. For this we will create our methods that will be responsible for creating our object based on the settings provided in the annotation.
private Collection<Arguments> create() throws JSONException {
final var builder = getBuilder(target);
if (builder.isEmpty()) {
throw new ProviderNotFoundException("Could not found valid BaseDtoJsonBuilder!");
}
return create(builder.get());
}
private Collection<Arguments> create(final BaseDtoJsonBuilder builder) throws JSONException {
final var list = new ArrayList<Arguments>();
for (int i = 0; i < listOfObjectsSize; i++) {
Map obj = invalidFields
? builder.buildInvalidJsonObject()
: builder.buildValidJsonObject();
list.add(Arguments.arguments(obj.values().stream().findFirst().get().toString(),
obj.keySet().stream().findFirst().get()));
}
return list;
}Finally, we just need to reference the create() method inside provideArguments.
@Override
public Stream<? extends Arguments> provideArguments(ExtensionContext context) throws Exception {
final var objSource = create();
return objSource.stream();
}With that we finished configuring our custom Argument Source, let’s now use it within our tests for the controller.
Going back to our test class, let’s create our scenario where we pass a valid body and expect the endpoint to return a status code created and have a “Location” header. Here’s how it looks when we use our custom annotation:
@ParameterizedTest(name = "#[{index}] Should assert status code 201 for json value: {0}")
@RandomJSONSource(interactions = 10, targetBuilder = UserDtoJsonBuilder.class)
public void createValidUser(final String json, final UserDTO obj) throws Exception {
when(userService.createUser(obj)).thenReturn(obj);
mockMvc.perform(post(REQUEST_URL)
.content(json)
.contentType(MediaType.APPLICATION_JSON_VALUE))
.andExpect(status().isCreated())
.andExpect(header().exists("Location"));
}With this configuration, the test will be run 10 times with different objects that were generated in our provider. This functionality is quite powerful and can be used for tests that have an object that is too complex to be generated manually or through a CSV. For this case, we could have also used Custom Aggregators, you can read more about this approach in the official documentation.
Now we are going to use this same approach to test the scenario where the body of the request is an invalid object, for that we will add the flag invalidFields = true in our annotation. Looking like this:
@ParameterizedTest(name = "#[{index}] Should assert status code 400 for json value: {0}")
@RandomJSONSource(interactions = 10, targetBuilder = UserDtoJsonBuilder.class, invalidFields = true)
public void createInvalidUser(final String json) throws Exception {
mockMvc.perform(post(REQUEST_URL)
.content(json)
.contentType(MediaType.APPLICATION_JSON_VALUE))
.andExpect(status().isBadRequest());
}Finally, we need to test our endpoint to list a user by email. In this scenario we can still use this approach, but we will have the disadvantage of the json field not being used. For that, we’ll change the call from post to get and add the correct path to the REQUEST_URL .
@ParameterizedTest(name = "#[{index}] Should assert status code 200 and return a valid UserDTO")
@RandomJSONSource(interactions = 10, targetBuilder = UserDtoJsonBuilder.class)
public void findUserByEmail(final String json, final UserDTO obj) throws Exception {
when(userService.findUserByEmail(obj.getEmail())).thenReturn(obj);
mockMvc.perform(get(REQUEST_URL+"/{email}", obj.getEmail()))
.andExpect(status().isOk())
.andExpect(jsonPath("$.name").value(obj.getName()))
.andExpect(jsonPath("$.email").value(obj.getEmail()));
}See that we use jsonPath() here, this method allows us to access the response json that the endpoint returns. This is a very powerful feature as it allows us to check the response in addition to the status code. We were also able to check the value of each json field.
And now, let’s write our last test for this context, we need to check the status code for when the endpoint can’t find a user for the given email.
@ParameterizedTest(name = "#[{index}] Should assert status code 204 for no valid UserDTO")
@RandomJSONSource(interactions = 10, targetBuilder = UserDtoJsonBuilder.class)
public void couldNotFindUserByEmail(final String json, final UserDTO obj) throws Exception {
when(userService.findUserByEmail(obj.getEmail())).thenReturn(null);
mockMvc.perform(get(REQUEST_URL+"/{email}", obj.getEmail()))
.andExpect(status().isNoContent());
}With that, we were able to go through all the application layers of the user context and we were able to test the failure and success scenarios for each one. Our application is now much safer and more resilient to errors, and the tests will report if any software behavior changes.
Pitest and how can it measure efficiency of testing.
Pitest is a mutating testing framework that help us developers to find failures and test leaks on our code.
Faults (or mutations) are automatically seeded into your code, then your tests are run. If your tests fail then the mutation is killed, if your tests pass then the mutation lived.
PIT runs your unit tests against automatically modified versions of your application code. When the application code changes, it should produce different results and cause the unit tests to fail. If a unit test does not fail in this situation, it may indicate an issue with the test suite. — From Pitest Site.
Pitest is a really powerful tool, as it can modify our code to identify flaws in our tests. This is very useful because often in the rush we don’t realize that we missed a test scenario that can cause an application error.
At the end of each Pitest execution, it generates a report containing information on how many tests were run and how many mutations were generated, also how many passed or failed the tests.
See the example below of a report generated for the TaskService.java class in the tests executed in the TaskServiceTest.java class.
As you can see the test was not covering the listing of Tasks by User Id, this was extremely fatal as it was part of an important flow which could completely break the domain of this project. It is also possible to observe in this report which mutations were carried out and which passed or not.
After making the necessary corrections, the report generated by Pitest was as follows:
Conclusion
Tests are extremely important when we are building software. It is undeniable that they are the ones who guarantee that the behavior of the software remains as expected and that nothing has changed when we try to deploy the application. As we can see in the article, there are several ways and features of testing, each one has its purpose and it is up to us as developers to research and study the best approach for each scenario.
Tools like Pitest are extremely powerful and help us to identify test failures and opportunities to improve the effectiveness and coverage of the tests, they are extremely helpful and I recommend using them to periodically scan the health of the tests in the software.
Lastly, I would like to thank all my colleagues at QuintoAndar.com who taught me so much this year. Especially to José Luiz, Ana Robles and Gabriel Arrais who helped and encouraged me to write this article. Also I couldn’t forget about the person who inspired me to study more about tests and is a reference for me, my greatest friend Elvys Soares.
