Testing GenServer Basic Cache
Basic Caching with GenServer — Part 3
In part 1 and 2 we explored ways to use GenServer as a basic caching tool for repetitive Ecto queries. This time we will cover another aspect: testing.
If you followed the other articles in the series, the code is quite simple. So you may think that there is nothing special to consider for the tests. And you would be almost right. Almost!
There are some aspects not strictly related our caching subject. But more to GenServer (processes) and Ecto, which worth covering.
Let’s start by writing a few simple tests for our caching service:
setup I’m using ExMachina to generate products with different levels of discounts. Then checking if the top has 10 products and the actual price of the first and last products in the top. The second tests ensure that a new product with high discount will be in the discounts top.
At this point we expect those simple tests to pass without any issues. But they are not!
Assertion with == failed
code: Enum.count(top) == 10
The failed test shows no products in the top. Using
IO.inspect in the Cache implementation, we can see that, actually, there are no products at all (eg.
Repo.all(Product)). Run the same commands in the test file, and you will see that the products are there! So, what is happening?
Well, the answer lies in the
Ecto.Adapters.SQL.Sandbox module. This is the way the tests handle Ecto. Take some time and read the documentation. At least the first paragraph for now.
After checking the docs, asking stackoverflow and experimenting with the code, I found out who is the one “guilty” for our test failure. It’s the
init/1 function in the Cache implementation.
Because it contains a function that calls
The Cache GenServer process is started in the main application supervision tree. That means that
Shop.Repo is called before the test_helper.exs is able to run
Ecto.Adapters.SQL.Sandbox.mode(Shop.Repo, :manual) and take control over the DB connections. The Cache GenServer has a separate DB connection than our tests.
Solution 1 — restart the Cache GenServer
If you do not want to change any code in the current Cache implementation, the solution is to restart the GenServer in the setup of the test:
We kill the initial process and start a new one. This time the test is aware of the new connection. The data_case.ex runs in shared mode. It allows the test process to share its connection with the new Cache process. However, the tests should NOT be run with
Run the tests again, and they will pass.
This setup may be acceptable if you need to test only this module. But let’s assume the following: the Cache module is used in many other parts of the app, which we want to test as well. If you follow the same logic, you will need to restart the Cache server for each test and run it without
async. Well, that may not be acceptable anymore, especially for a big test suite.
Mock the Cache
At this point, we already tested the Cache. We know that it works. We do not need the same implementation in every new test, but we need the same results. For this, we can revert to our initial, pre-cache logic from the first article in this series. Meaning we will use SQL queries to get the top discounts.
Create a MockCache module:
We use the config files to pick the Cache module for each environment. MockCache for
test and Cache for everything else:
config :shop, :cache, Shop.Cache
config :shop, :cache, Shop.MockCache
Now you can use the env variables wherever you need the Cache. For example in the ProductController:
The new tests that will call the Cache will now use the MockCache. You will not need to restart the GenServer and also you will be able to run them
Solution 2 — change the
Another possibility is to avoid the issue itself by changing the
init implementation. We don’t call Repo in the GenServer
init. test_helper will take control of the connections. The changes are not big, nor complicated:
We initialize the server with an
:empty state, which will be easy to pattern match. The first time the
get_products_v2 is called, it will populate the state with the current top discounts.
The same happens if you will create a new product and call
:post_product_v3. If the state is
:empty, it will populate it with the current discounts when calling the initial implementation of the function.
You can now delete the cache restart functions from your test, and the test will pass:
Supervisor.terminate_child(Shop.Supervisor, Cache) Supervisor.restart_child(Shop.Supervisor, Cache)
I like this approach because it eliminates the testing issue, and touches the DB only when need it. On the other hand, changing your implementation to accommodate the test suite may not be your preferred option.
Anyway, if you decide to go for it, the cache mocking above, applies for this case as well.
Bonus — run the cache test async
Let’s push it a bit further and try to run the Cache tests with the
async option. Yes, they will fail with something like
(DBConnection.OwnershipError) cannot find ownership process for #PID<0.310.0>. If you read the rest of the error, it will hint you the solution as well. You can manually allow the process to use the same connection as the parent(test process):
Now the Cache tests will be able to run in
async mode as well.
The code in this final version is available on github.
Originally published at iacobson.net — August 16, 2017.