Don’t Use @Transactional in Tests
How to not ruin your Spring Boot application test suite
Follow me on Github and Twitter. Clap the article to help reach more audience.
You can also read the article on dev.to.
Source Code
You can find the sample code used for this article here. The sample code has 3 branches, the first with @Transactional, the second without, and the third using an appropriate replacement.
If you need to run the project, you need PostgreSQL installed on your computer. Then configure the environment variables below:
- DATABASE_HOST: localhost
- DATABASE_NAME: avoid-transactional
- DATABASE_USERNAME: postgres
- DATABASE_PASSWORD: yourpassword
You can of course provide different values depending on your local database configuration.
I recommend you to create two databases, one for production and another for tests. So you can run the tests without interfering with manual production testing using a tool like cURL or Postman. You’ll need to change the DATABASE_NAME variable in-between.
The Problem
When we are doing integration tests with Spring Boot — the ones that test several layers of your code and are often slow — we tend to use the @Transactional annotation to guarantee that data is cleaned up after the test execution, as we want to guarantee a deterministic test suite.
You can also annotate your test class with @Transactional, instead of annotating all test methods, of course.
The problem with creating tests like this is that it is highly susceptible to ruin your test suite with a lot of false negatives. A false negative is a test that runs successfully when it’s supposed to fail due to a bug in production code.
Having a lot of false negatives can render your test suite useless, for two reasons:
- The confidence developers have towards the test suite will decrease, and people will start to get fearful of doing changes because they may break something.
- A lot of bugs will go to production without developers noticing unless they start to do manual tests, which is counter-productive.
A false negative is a test that runs successfully, when it’s supposed to fail when the production code has a bug.
If you run all the tests in with-transactional git branch, they will run successfully:
But let’s then try to execute the application and call the endpoints using tools like cURL or Postman.
Lazy Loading Does Not Work
If you try to create a cart and then add an item to it, you will receive a 500 HTTP error. You can use the pre-generated product with the ID 2bdb8e93–3832–47b6–8bbc-d6b5db277989 for the cart item:
The exception thrown in the console log was a LazyInitializationException:
org.hibernate.LazyInitializationException: failed to lazily initialize a collection of role: com.example.avoidtransactional.domain.model.Cart._items, could not initialize proxy - no SessionThis exception was thrown while Hibernate tried to lazily fetch Cart._items list. Lazy loading does not work when the database session is closed, which happens right after you retrieve an entity with the repository.
The problem happens in the 11th line. The Cart.addProduct() method internally manipulates a lazy collection of cart items, which raises the exception.
If you remove the @Transactional annotation from CartItemsController class, the tests within it will fail with the same LazyInitializationException.
org.springframework.web.util.NestedServletException: Request processing failed; nested exception is org.hibernate.LazyInitializationException: failed to lazily initialize a collection of role: com.example.avoidtransactional.domain.model.Cart._items, could not initialize proxy - no SessionYou can change the git branch to without-transactional, where I removed @Transactional from all tests. The two tests that still run successfully are passing because they don’t eagerly fetch collections.
Why all the tests were not failing before? Well, it turns out that @Transactional wraps the contained code within a transaction — which maintains the database session opened across all code contained in the transaction, which is needed for successful lazy loading of collections.
The recommended way to deal with collections in JPA/Hibernate is to eagerly fetch them manually while maintaining the collection lazy, to fetch just what you need and to avoid the N-M performance problem. But for brevity, I changed the fetch mode to EAGER in without-transactional-replacement branch.
Entities Are Saved Automatically
Another subtle problem with using @Transactional in tests is that the auto-commit feature is enabled for the tested code. Meaning that entities fetched within a @Transactional code are automatically saved at the end.
The test of the method below should still pass, even though we’re not explicitly calling save() in the 16th line. Just be sure that you’re in with-transactional branch.
But if you send a request to add an item to a cart, the new item won’t be saved into the database.
Hard to use the database to troubleshoot test failures
Another problem with using @Transactional is that you can’t use the database to investigate the failure reasons, as the data is never actually saved into the database because transactions are always rollback’d at the end of the test method execution.
So it would be convenient to not wrap our tests under transactions to let the data be saved, while only cleaning the database before a test method execution.
Proposed Solution
We can’t just remove @Transactional without doing anything else, as we need to guarantee that all tests are independent of each other (see Martin Fowler article on non-determinism in tests)
If you change the git branch to without-transactional-replacement you’ll see that the tests are being annotated with a JUnit 5 extension that cleans all the data before a test method execution.
PostgresDbCleanerExtension extends the interface BeforeEachCallback that implements the beforeEach method, which is called before each test method is executed to clean the database.
After the database properties are loaded from application.yml and bound to a javax.sql.DataSource, we use to retrieve all database tables with the class java.sql.DatabaseMetaData:
And clean all of them with the TRUNCATE query, except the ones contained in the TABLES_TO_IGNORE list, which are the ones related to database migration history:
Using TRUNCATE is a bit slower, you could also disable foreign key constraint check and delete all the data using the DELETE command which is faster.
The method we used with TRUNCATE is slower probably because it reclaims disk space immediately and clears all junk data.
Solving the production bugs
You’ll also notice that we replaced lazy loading fetch to eager loading so we can solve the production bugs related to LazyInitializationException.
Now that we got rid of @Transactional, and we discovered the false negatives in our test suite, let’s fix the actual bugs so that tests can pass.
I changed fetch to FetchType.EAGER in the problematic Cart._items property. But you should really prefer to eagerly fetch manually as described in this article by Vlad Mihalcea.
If you run the tests in without-transactional-replacement branch. They should all pass.
Conclusion
We saw in this article why using @Transactional in our test suite is not a good idea.
@Transactional enables undesired behaviors that produce false negatives in our test suite, which can impact the confidence developers have to make changes and render the test suite useless.
Using @Transactional in tests can also make troubleshooting difficult, as no data is really saved in the database.
Always prefer to clean the data manually, instead of relying on transactions, and prefer to do so before the tests execute, as you’ll want to know why a test failed by analyzing the database.
Thank you for reading this article, please clap to help the article reach more audience.
