Property-based testing in Kotlin — Part 1
Are we testing like it’s 1999?
Kotlin incorporates the most recent understanding in software engineering into its language design. For example, we now understand that with implicit nullability it’s easy to make a mistake. So in Kotlin you can’t assign null to a variable unless you explicitly declare it to be nullable.
If the latest understanding of software engineering informs our language design, it’s prudent to ask whether these advancements inform our test design.
Unfortunately, even in this new language it’s so easy to write tests that are hard to understand, hard to maintain, and don’t give confidence that you’ve got it right. More about that in part two.
If we’re still so prone to writing bad tests in Kotlin, then it may be our testing practice is still stuck in the past, along with language design artifacts like the notoriously unfashionable new keyword. This is not to belittle older design choices — ragging on new is easy, writing good tests is hard. How old, then, is our test design choice?
Rewind time
It’s rewind time! If I controlled Rewind, I would want F̶o̶r̶t̶n̶i̶t̶e̶ ̶a̶n̶d̶ ̶M̶a̶r̶q̶u̶e̶s̶ ̶B̶r̶o̶w̶n̶l̶e̶e̶ SUnit. Written for Smalltalk in 1994, SUnit is the grandaddy of the ‘XUnit’ testing frameworks — NUnit, RUnit, and our beloved JUnit. Here’s what it looks like in the IDE:
Although this is nearly 25 years ago, an aeon in the world of software, it doesn’t take the Rosetta Stone to decipher this screen. The primogeniture, JUnit, is born a few years later in 1998. If we were using JUnit simplex, we really would be testing like it’s 1999. But that’s not the only factor here.
2001 — a mock odyssey
Three years after JUnit, we see the first dynamic mock object framework — EasyMock.
When writing a unit test, we substitute the dependencies of the class with test doubles we call “mocks”. Causing the mocks to act in a certain way is called “stubbing a behavior”.
These days we can make these test doubles quite easily. This wasn’t always the case: before EasyMock you would have used code-generation or manual wiring-up to achieve this.
A testament to its longevity, almost 20 years later you can still write tests in EasyMock, even in Kotlin:
However, there are a few areas where our understanding has matured. As seen in the code above, when we stub a behavior on the mocks we are using the term “expect”:
expect(mock.get(0)).andStubReturn(“one”)
If we divide our test into arrange, act, assert we normally wouldn’t talk about expectations in the first part of our test. At the stage where we are stubbing a behavior, we shouldn’t know or care about what interactions the mocks will encounter. We only care whether the result we measure at the end of the test is within spec.
Additionally, we have to manually tell the mocks to do their work with the replay method. Our test practice has progressed from here, so it seems we’re not testing like it’s 2001.
2007 Cocktail of the Year
2007 serves us a refreshing cocktail of testing elegance with Mockito:
Unlike EasyMock, we no longer talk of expectations in the first part of a test. And, there is no need to tell the mocks to do their work with replay— they sit there waiting for an interaction and will play their stubbed behavior indefinitely. Of course, mockito-kotlin makes things a bit nicer by replacing `when` with whenever :
For many projects, this is where test practice currently lies.
Can we do better?
While we may not be testing like it’s 1999, our tests would not look out-of-place in 2007, more than a decade ago. It’s not intrinsically bad to use a technique from the past, except that the tests we write often fail to deliver on their promises. Let’s look at these promise-breaking tests in part two, and how we can keep them faithful using a modern technique: property-based testing. Stay tuned!
Part two follows here.
