Mockk messing up with the JVM

Ok, this is a story with a happy beginning. Or ending. Or both. Update: this issue was fixed here.

I had been pushing commits as crazy, with their appropriate tests. I was working in a service and I was using kotlin, spring, jooq and mockito. I had run into an issue that felt could be improved.

I had read about mockk from a coworker so I gave it a try. I started to implement a new functionality that had nothing to do with another one and implemented the tests. The only relation they had was a table that the new functionality was reading from the database (not writting to, I promise!).

Guess what happened after that? Some other tests started to fail. Seriously, why fixing something opens several other bugs? How could be possible that a completely different test suite that had nothing to do with those services started to fail? The symptoms were really strange. I was looking at the logs and I could see jooq retrieving a row and suddenly failing to map an object because the id field was null.

Not as bad as this one, but seeing test fail is painful

This was a really weird case, since running those tests alone worked but when invoked as part of the suite, failed. Moreover, altering the order of the tests make them work or fail, as it some dark magic was controlling the code.

Since JUnit 5 does not have yet a way to specify the order of the tests much like JUnit 4 had this was hard to test and identify. Digging deeper I found that on the native call on java.lang.reflect.Executable, it’s private native Parameter[] getParameters0(); was returning a different set of parameters after `mockk` had been invoked. Our jooq’s ParameterAwareRecordMapper indeed uses constructor.getParameters to map from a record to a POJO and voilà, after that call, jooq was unable to found any parameter from the record and thus populating a POJO with all fields initialized to null

What jooq was doing was reading the parameter names from the constructor of the POJO and looking for fields with those names on the record. Since the POJO was generated by jooq itself based on the DB fields everything should work fine from here. But was not. This messed up my CI, my code and got me through the rabbit hole for at least a couple of hours.

I managed to isolate the problem in the least lines of code possible and so far I have not found any work around nor way to reset what is happening after using mockk(). I suspect that the library uses reflection to inject it’s own constructor but forgets to keep the parameters name. Here is the minimal test to reproduce it (remember that JUnit 5 cannot warrant test order so run them independently if needed):

There are two tests and a data class with two parameters in the constructor, name and age. The first test shows that the class has only one constructor but it is not the same after calling to mockk() ; the second one shows that the parameter names are not present anymore.

One expects that just mocking an object does not mess with the loaded class by the JVM but I got this:

15:25:12.631 [main] DEBUG io.mockk.impl.instantiation.AbstractMockFactory - Creating mockk for DataClass name=#2, moreInterfaces=[]
15:25:12.637 [main] DEBUG io.mockk.impl.stub.CommonClearer - Clearing [] mocks
java.lang.AssertionError: Assertion failedat org.tarodbofh.medium.mockk.MockkMessingNativeJVM.testConstructorEqualityAfterClearMocks(MockkMessingNativeJVM.kt:26)

This means that after calling mockk() the constructor of the class was altered!

Of course, the second test failed:

15:25:11.563 [main] DEBUG io.mockk.impl.instantiation.AbstractMockFactory - Creating mockk for DataClass name=#1, moreInterfaces=[]
15:25:12.555 [main] DEBUG io.mockk.impl.stub.CommonClearer - Clearing [] mocks
java.lang.AssertionError: [Extracted: name]
Expecting:
<["arg0", "arg1"]>
to contain exactly in any order:
<["name", "age"]>
elements not found:
<["name", "age"]>
and elements not expected:
<["arg0", "arg1"]>
at org.tarodbofh.medium.mockk.MockkMessingNativeJVM.testConstructorGetParametersMocked(MockkMessingNativeJVM.kt:44)

The parameter names had changed, and that is why the other tests started to fail after I mocked an object on an Unit test. Don’t forget to compile with java parameters targeting Java 8 (see https://bugs.openjdk.java.net/browse/JDK-8046108). To do that, just add this to your .gradle script: [compileKotlin, compileTestKotlin]*.kotlinOptions*.javaParameters = true and you’ll be using the parameters. Else, the first test will fail on the first assertion and not on the second one. If you need help setting up gradle to test this, you can have a look at my .gradle file here.

I learned this the hard way but in the end I opted to not mock any object I was requiring, but using constant value objects. For example, if I am testing underage people I will have a val UNDERAGE_PERSON = DataClass("underage, 1) on an Extensions.kt file instead of using a mocked object, at least for POJOs, DTOs and other stateful constructs. For stateless and business logic methods, I am hesitant until I inspect mockk code to see what is it doing to mess with the JVM.

After a brief chat on gitter with mockk mantainer he asked to submit a bug report so I hope this gets fixed on a future release.

Some days later, I experienced the same bug with mockito, but only when @Mock annotations where used instead of using mock(...) methods. I was unable to reproduce it on a constant behavior as apparently this happened randomly. This bug indeed seems related to this two bugs on mockito that were already resolved.

Update: The mockk issue fas fixed on mockk 1.8.7 release, less than 10 days after the bug was submitted. That is grade A job from the maintainers.

Juan Francisco Ara Monzón

Written by

Hi there! I'm a senior software engineer, back-end technical lead and architect in Madrid, working at lastminute.com. Loving gaming and motorbikes.

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade