Ginkgotchas (yeh also Gomega)
In my last post, I wrote about some effective Ginkgo and Gomega practices; there were a few more suggestions that came to mind as I was writing but they felt like they were less “effective” and more “where did the last three hours go debugging this crap?” — A fairly arbitrary distinction but it’s my post so…
Like the previous post, familiarity with Ginkgo and Gomega is expected before reading. With that in mind, here’s some pain-points you might want to take care to avoid when writing tests using Ginkgo and Gomega.
Not initialising a variable in the closest BeforeEach
This must be the most pervasive gotcha for people new to writing Ginkgo tests, but it definitely catches old hands from time to time. Look at this code:
This looks simple enough right? We declare a variable and initialise it to 10
, test that the value is 10
, then we have a block that modifies number
to 42
and tests it is 42
. What could go wrong?
If you come from a jUnit background, you probably know that each test gets its own instance of the test class to avoid test pollution. In Ginkgo, test lifecycle is bounded by the test processes (1
to n
depending on whether you are using the -p
parallel flag). If you turn on --randomizeAllSpecs
(see my previous post for why you should do this right now) there is no guarantee of the order these examples will run in. As such, for the previous snippet, the first example sets the value of number
to 42
and the next example asserts it to be 10
causing the failure.
The correct way to resolve this is to explicitly set a default value in the BeforeEach
closest to variable declaration:
This gotcha is particularly frustrating to debug when the -p
and --randomizeAllSpecs
flags are set (which they should be) because the test pollution can take many forms.
TIP: Each ginkgo run prints the seed
it uses to shard and order the examples across nodes, and you can pass this with --seed
to Ginkgo to replay a previous run!
Not passing a function to Eventually or Consistently
More like Consistently
a pain in my ass. Ok, so the asynchronous matchers in Gomega are pretty nifty to use, they allow you to pass something which will be checked periodically against some particular matcher.
You can pass a channel:
Eventually(ch).Should(Receive())
You can pass a structs that satisfies some interface required by the matcher (in the case of Say
, something that has a Buffer
method:
Eventually(session).Should(Say("something"))
You can even pass a function which will be periodically called and asserted against:
var i int = 0func maybe() bool {
if i > 10 {
return true
} i++
return false
}Eventually(maybe).Should(BeTrue())
But what you absolutely should not do, is pass the result of an evaluated function:
var i int = 0func maybe() bool {
if i > 10 {
return true
} i++
return false
}Eventually(maybe()).Should(BeTrue())
The function will be evaluated once, eagerly, and you’ll be left wondering where you went wrong in life.
Believing Async Should and ShouldNot are opposites
This one seems fairly obvious when you take a moment to think about it but it’s caught me and many others in a moment of weakness. When you are writing a standard assertion:
Expect(foo).To(Equal(something))
and you decide you actually want to assert the opposite, you can simply flip To
to NotTo
:
Expect(foo).NotTo(Equal(something))
If you try to do the same thing with async assertions (Eventually
and Consistently
), you are in for a world of hurt. Let’s reuse the same example from above, assuming that foo
is a function that will return some expected value after some amount of time.
Eventually(foo).Should(Equal(something))
Flipping the intent, that foo
should not ever equal something
you might reasonably try to say:
Eventually(foo).ShouldNot(Equal(something))
This looks like it should work, but remember that the function foo
won’t return the unexpected value for some time, Eventually
will evaluate the function, see that it passes the matcher and declare success!
Likely, the intent is better captured by additionally changing Eventually
for Consistently
:
Consistently(foo).ShouldNot(Equal(something))
Much better.
Leaking a goroutine
In the last post, I wrote that you should use goroutines with care because Ginkgo doesn’t really play nice. Consider this snippet below:
You can mostly ignore the sleeps, they exist only to orchestrate an overlap between the examples to demonstrate this failure. Our first example creates a goroutine (it even uses GinkgoRecover()
like a good little example), and calls Fail
. The second example makes an assertion that cannot possibly fail, that true
is equal to true
!
The second example failed! true
is no longer equal to true
and Pi
is exactly 3
!
What happened here is that the goroutine outlived the lifetime of its spec and when Fail
was called, Ginkgo determined that the currently running example was the second one. It’s important to make sure your goroutines lifetimes are carefully handled in your examples, typically coordinated via channels.
Asserting errors against stdout
I’ll keep this one short. If you’ve used the gexec.Say
matcher to test stdio of some running process you’ll recognise this:
Eventually(session).Should(Say("something"))
This asserts that the string "something"
has appeared on some output. More specifically, it asserts that the string has appeared on stdout
. If you want to test output on stderr
, this mistake will leave you running in circles.
The Say
matcher will call the Buffer
method on session
to find the contents to assert against. Unfortunately, this returns only the stdout
buffer, rather than an amalgamation of both. To test against stderr
you need to write:
Eventually(session.Err).Should(Say("something"))
TIP: You may find it more expressive to intentionally use .Out
and .Err
everywhere to avoid confusion.
Accidentally regex testing with Say
This gotcha is brought to you by the letters [A-Za-z]
. Good joke right. Let’s take the following snippet of code that uses the gbytes
package:
This looks pretty simple right? There’s a buffer containing the string map[string]
and then we assert the buffer contains the string map[string]
!
The Say
matcher accepts a regular expression as an argument, so the [
and ]
characters are tripping it up so we’ll have to escape them with \
characters.
TIP: If you use backticks instead of double quotes to surround your argument to Say
, this gives a better indicator to your reader that there’s something special about the string.
Final thoughts!
This list of gotchas is certainly not exhaustive, but they are the most common ones I’ve come across. Despite some of the frustration I’ve experienced over the years at these, in my experience it doesn’t take long for people to recognise these patterns and avoid them. Hopefully this post helps capture some of that tribal knowledge so that others avoid the same issues in the first instance.
If you’ve got any thoughts on any of these, or have any more to add, let me know.
Thanks!