Acceptance testing your Go CLI

I’m a fan of Acceptance Test Driven Development, or Double-Loop TDD. This involves writing a failing high level end-to-end test that desires some feature, then following the red-green-refactor cycle through layers of unit tests until the acceptance test passes, and then repeating for the next feature.

Double-loop TDD credit: Nat Price and Steve Freeman

I’m also a fan of Go and CLI tools. No diagram necessary.

Given these statements, it’s important for me to able to write a test suite that will exercise my CLIs in a black-box manner. Even if you don’t write your tests first, it is still extremely important to have confidence that if you changed the internals (and you do refactor don’t you?!), there are no regressions.

Fortunately, Ginkgo and it’s preferred matching library Gomega have our backs. The gexec library allows us to:

compile go binaries, start external processes, send signals and wait for them to exit, make assertions against the exit code, and stream output intogbytes.Buffers to allow you make assertions against output.

So let’s start testing a simple CLI (we’ll call this scratchpad), that we expect to print “Hello World” on stdout and exit with the status code 0. For our acceptance tests, we will need to:

• Compile the binary
• Run the built binary as a process
• Assert that “Hello World” appears on the stdout stream
• Assert that the process exits with the status code 0

Here’s an example of how that might look using ginkgo and gomega:

Let’s pick this apart. Those who are familiar with tools like rpsec should recognise this Behaviour-Driven testing style, with a top-level Describe outlining the subject under test: our scratchpad CLI!

Underneath that we have a BeforeEach block, which delegates to functions that will build the CLI, and then execute it. The gexec library provides utility functions to facilitate these steps:

Build takes the directory containing the main.go we want to build and returns the path of the built binary. This is a costly process, and we probably don’t want it to happen before every spec, andginkgo has BeforeSuite and SynchronizedBeforeSuite helpers to reduce overhead here — I’ve just used BeforeEach for simplicity.

Start takes the path of the built binary, and writers for the stdout and stderr streams. GinkgoWriter is a special buffer that contains all the output for the ginkgo process when tests are running, so any output from our CLI can appear in there (though it will usually only appear if a test fails). Returned is a gexec.Session which is an abstraction over the running process that satisfies interfaces needed by other matchers in our specs — let’s get on to that now.

Our first spec ensures that the process should exit with the status code 0:

The Eventually assertion ensures that a matcher will pass within some time (defaulting to 1 second), by iteratively checking for a condition to be true. In this case we can take advantage of the Exit function provided, which takes an expected status code as an argument. Usefully, the Session struct satisfies the Exiter interface required by this matcher.

Our next spec ensures that the process prints “Hello World” to the stdout stream:

The Say matcher from the gbytes package requires a buffer, or something that satisfies the BufferProvider interface, and again, the Session struct has us covered.

Finally, we can use the gexec.CleanupBuildArtifacts() to ensure we don’t leave debris around after each test, easy!

Alright, so this is a fairly trivial example of acceptance testing a CLI, but these tools can quickly be combined in an extremely expressive manner to cover all number of larger CLI projects for example, the Cloud Foundry CLI, the Pivotal OpsManager CLI, and even Ginkgo itself!

Soon I intend to do a video session on using ginkgo and gomega to ATDD a CLI with more interesting behaviour so if you’re interested, check back soon! For now however, you can find the full code for this in the scratchpad repository on github.