Acceptance testing your Go CLI
I’m a fan of Acceptance Test Driven Development, or Double-Loop TDD. This involves writing a failing high level end-to-end test that desires some feature, then following the red-green-refactor cycle through layers of unit tests until the acceptance test passes, and then repeating for the next feature.
I’m also a fan of Go and CLI tools. No diagram necessary.
Given these statements, it’s important for me to able to write a test suite that will exercise my CLIs in a black-box manner. Even if you don’t write your tests first, it is still extremely important to have confidence that if you changed the internals (and you do refactor don’t you?!), there are no regressions.
compile go binaries, start external processes, send signals and wait for them to exit, make assertions against the exit code, and stream output into
gbytes.Buffers to allow you make assertions against output.
So let’s start testing a simple CLI (we’ll call this
scratchpad), that we expect to print “Hello World” on
stdout and exit with the status code
0. For our acceptance tests, we will need to:
- Compile the binary
- Run the built binary as a process
- Assert that “Hello World” appears on the
- Assert that the process exits with the status code
Here’s an example of how that might look using ginkgo and gomega:
Let’s pick this apart. Those who are familiar with tools like
rpsec should recognise this Behaviour-Driven testing style, with a top-level
Describe outlining the subject under test: our
Underneath that we have a
BeforeEach block, which delegates to functions that will build the CLI, and then execute it. The
gexec library provides utility functions to facilitate these steps:
Build takes the directory containing the
main.go we want to build and returns the path of the built binary. This is a costly process, and we probably don’t want it to happen before every spec, and
SynchronizedBeforeSuite helpers to reduce overhead here — I’ve just used
BeforeEach for simplicity.
Start takes the path of the built binary, and writers for the
GinkgoWriter is a special buffer that contains all the output for the
ginkgo process when tests are running, so any output from our CLI can appear in there (though it will usually only appear if a test fails). Returned is a
gexec.Session which is an abstraction over the running process that satisfies interfaces needed by other matchers in our specs — let’s get on to that now.
Our first spec ensures that the process should exit with the status code
Eventually assertion ensures that a matcher will pass within some time (defaulting to 1 second), by iteratively checking for a condition to be true. In this case we can take advantage of the
Exit function provided, which takes an expected status code as an argument. Usefully, the
Session struct satisfies the
Exiter interface required by this matcher.
Our next spec ensures that the process prints “Hello World” to the stdout stream:
Say matcher from the
gbytes package requires a buffer, or something that satisfies the
BufferProvider interface, and again, the
Session struct has us covered.
Finally, we can use the
gexec.CleanupBuildArtifacts() to ensure we don’t leave debris around after each test, easy!
Alright, so this is a fairly trivial example of acceptance testing a CLI, but these tools can quickly be combined in an extremely expressive manner to cover all number of larger CLI projects for example, the Cloud Foundry CLI, the Pivotal OpsManager CLI, and even Ginkgo itself!
Soon I intend to do a video session on using ginkgo and gomega to ATDD a CLI with more interesting behaviour so if you’re interested, check back soon! For now however, you can find the full code for this in the scratchpad repository on github.