Don’t forget to unmockkAll

In a project I’m collaborating we are using Mockk and many other frameworks to do unit tests. This is a very nice framework that help us to easily mock all our components including kotlin objects and static methods. I’ve been using it for a couple of months and I do really like it for its easy usage, even more than Mockito

Today I faced some problem:

I was requested to add unit tests to some components. Such an easy task. The code then needed to be mocked not just instances but kotlin objects as well, and some other static methods too. Such easy way to just only add:

mockkObject(SomeKotlinObject)

The same approach you’ll need if you need to mock some static methods from a Java class:

mockkStatic()

I’ve added the tests and everything was fine. Consistent tests and coverage 100%. Another happy and productive day.

Today, and just by luck of not merging my change, I did a rebase of my branch to pull all the changes. My surprise was that the CI detected that a huge amount of unit tests were failing. Looking at the errors it looks like all were related to a lateinit property not being initialized. Strange, isn’t it? I’ve never touch any other test that the ones I’ve added.

I’ve run them in my local and they failed. I’ve run then on the main branch and everything was fine.

That gave me an idea of what could be happening:

Running only a failing test? -> PASSED
Running only my new tests? -> PASSED
Running all at once? -> FAILED

The Aha moment was when started reading the documentation of Mockk:

To revert back use unmockkAll or unmockkObject:

Many flaky tests have that behavior in common: They pass when run isolated, but fail if run along with the rest.

The takeaway for me is clear: Remember to always tearDown your methods, and remember always to reset all the conditions to make the tests deterministic.