Using Go’s build constraint tag to help build mock services for service testing

So recently I was asked help get some API services built with go to be testable through service testing. By service testing I mean using `go test` to test JSON payloads coming from the running service.

The problem was that the service was designed with all the drivers put in a global struct variable in the main.go file.

./main.go

package main
import (
    "github.com/nats-io/nats"    
    ...
)
var global struct {
    nats *nats.Conn
    ...
}
func main() {
    global.nats, err := nats.Connect("nats://127.0.0.1:4222")
    ...
}

Now in order to run this once its built we would need a running instance of a Nats service. For normal running this is typically fine. Just get the Nats docker image and run:

# docker run -it -d -p 4222:4222 nats:latest

Now you have a Nats instance running that any local service can use. While this is great locally, sending to a hosted CI service to run tests can get costly and very slow. This becomes more costly when you figure in the fact that there is probably another service needed on the other side of that Nats stream.

So now we are looking at needing to build the service we want to test, a deployment of the Nats service, the service you want to communicate with, probably some sort of data store, and so forth. It can quickly become a deep rabbit hole. And you also have to make sure the service you want to communicate with is the correct version for the service you want to test. Very impractical…

So then I had to find another solution. I came across the concept of “build constraints” in the Go Docs. Now this seemed to have be a possible solution. While the official docs leave much to be desired (I will hopefully get to putting in a PR to complete the docs), I was directed to a Dave Cheney Post “Using // +build to switch between debug and release builds”, which helped out a lot. So I began hacking away to come up with a solution.

After writing a few examples, I decided to start to implement this in the actual code base.

First I knew I needed to pull the global struct out of the main.go file and into its own file

./global.go

package main
var global struct {
    nats *nats.Conn
    ...
}
...

Also move the nats.Connect() out of the main func, and into a simple wrapper inside the global.go file

./global.go … cont.

...
func initNats() {
    global.nats, err := nats.Connect("nats://127.0.0.1:4222")
    if err != nil {        
        panic(err)
    }
}

Now we are set up to easily mock the Nats driver in out program. Next we need to set up the mock “package”. I put the word package in quotes because, I will not be putting it into its own package, but make it a part of the main package. I’ll explain why later.

./mocks_nats.go

// +build mock
package main
import (                               
    "encoding/json"                               
    "fmt"                               
    "time"                              
)
                                                            
// NatsConn ..                              
type NatsConn struct{}                                                             
// NatsOptions ..                              
type NatsOptions struct{}                                                             
// NatsMsg ..                              
type NatsMsg struct {                               
    Subject string                       
    Reply   string                               
    Data    []byte                               
    Sub     *NatsSubscription                              
}                                                             
// NatsSubscription ..                              
type NatsSubscription struct{}                                                             
// NatsConnect ..                              
func NatsConnect(url string, options ...NatsOptions) (*NatsConn, error) {                               
    return &NatsConn{}, nil                              
}                                                             
// Close ..                              
func (nc *NatsConn) Close() {}                                                             
// Request ..                              
func (nc *NatsConn) Request(subj string, data []byte, timeout time.Duration) (*NatsMsg, error) {                               
    return &NatsMsg{}, nil
}                                                             
// Publish ..                              
func (nc *NatsConn) Publish(subj string, data []byte) error {          
    return nil
}  

Let me go through what I have done here. The first line you see a strange “comment”. This is not necessarily a comment, but the build tag which I’ll show you how to how to use in a bit. to break it down: first we use the single line comment identifier // then we MUST have a space before the +build identifier, next we have the +build identifier, and finally the name of the tag mock. One other thing… you MUST have an empty line after your build tag or the compiler won’t treat it like a Build Constraint, and use it as a comment.

1 // +build mock
2
3 package main
4
...

Now let me explain the rest of the file. I just needed a bare minimum mock of the nats-io/nats package. So I went to the Nats GoDoc and looked over the structure of the package. Then I looked at the service I was building, and saw what functions and methods I needed to mock. First was the Conn struct, since I am not actually connecting to a Nats instance I really don’t need any of the internals, hence type NatsConn struct {}. You also see that I prefixed all of the functions, and structs with “Nats”, this is for organizational purposes, so I will at any time see what is mocked and what is real.

Next was the Connect initializer func. Again with the prefix, I made its parameters match the official Nats nats.Connect() func, while implementing my own empty Options struct as NatsOptions struct {}, and instead of returning Conn we return the mocked NatsConn. The goal is to remove any dependencies to the nats-io/nats package.

Next the service uses the nats.Conn.Request() func, which sends out a message through Nats and expects a reply, and the nats.Conn.Publish() method, which broadcasts a message into the Nats stream without listening for a response. So using our mocked NatsConn replacement we create a simple clone of the two methods that belong to NatsConn. As you can see the method name matches the actual name of the nats.Conn.Request(), this allows us to simply drop in the NatsConn struct, without having to modify the rest of our code. Right now they just return empty structs and nil errors, but as you can see in my nats.Msg clone NatsMsg I actually filled in the struct.

When I call the nats.Conn.Request() method my service is expecting a response, and we will need to be able to give it something. In my case the service is expecting a structured JSON response. For the purpose of this article we will use this

package main
type SomeData struct {
    Param1 string `json:"param_1"`
    Param2 int    `json:"param_2"`
}

So in our clone of the Request() method I build out this…

./mocks_nats.go

...
func (nc *NatsConn) Request(subj string, data []byte, timeout time.Duration) (*NatsMsg, error) {
    var (
        err error
        b []byte
    )
    
    switch subj {
    case "somedata.subject":
        b, err = json.Marshal(&SomeData{"Hello", 1})
    default:
        b = []byte(fmt.Sprintf(`{"error": "failed test. Sub: %s"}`, subj))
    }
    
    return &NatsMsg{                                
        Subject: "Ixerte3",                                
        Reply:   "_XZrew",                                
        Data:    b,                                
        Sub:     &NatsSubscription{},                          
    }, err
}
...

What we did here is created a switch statement that checks the subject passed in our mock Request() method, and sends a properly structured response. So now our service can operate properly without connecting to a Nats stream. We can add all the subjects we want, and if we do not handle a subject then the default will handle that by sending an error message.

Now that we have the mock “package” built, how do we use it in a build? Next we need to go back to the global.go file, and add a build constraint tag.

./global.go

// +build !mock
package main
...

In this tag we put an exclamation point before the mock tag. the tells the compiler to ignore this file when we build with the -tags mock flag. I’ll explain more later. Reading the docs on Build Constraints will show you different ways you can combine tags if you have multiple constraints.

// +build prod,!staging

Build with prod tag, but not if the staging tag is called. Even if you call -tags prod,staging.

// +build prod,linux !staging

Build if the prod and linux tags are called but never with a staging tag.

Next we have to create a mock file for the global.go file so we can use our cloned local structs, functions and methods from the mocks_nats.go file.

./global_mock.go

// +build mock
package main
var global struct {
    nats *NatsConn
    ...
}
func initNats() {
    global.nats, err := NatsConnect("nats://127.0.0.1:4222")
    ...
}

First you see the build constraint at the top, and the mandatory empty line after. The package declaration, and no imports. We have totally removed the need for the nats-io/nats package. In our global var the nats parameter is now a pointer to the NatsConn struct and not the nats.Conn pointer. The (now mocked) connection to Nats is now handled by the NatsConnect() function. Now any call to global.nats will be handled by our mocks_nats.go.

Now to the build. To build the this for regular use (using the actual nats-io/nats driver:

$ go build -o ./somebin

And to build with our mock driver:

$ go build -tags mock -o./mock-somebin

As you can see all we have to do is add the -tags flag. Any file with the // +build mock constraint will be used by the compiler, and any file with the // +build !mock will be ignored.

A simple note:

$ go build -tags mock -o ./mock-somebin *.go

Using the wildcard *.go will override the -tags mock flag and try to compile all the files, resulting in errors.

Really once you get the hang of using build constraints they become invaluable. So go out and experiment!

A final note on the Nats stream:

I do not work for Nats, but I use it every day, and I absolutely love it! Nats can be as simple as you want it to be. When I use it, I use it as just a stream and allow my services handle queuing. This save you from MQ hell. Nats is by far the fastest message stream I have ever used, and its written in Go! Go check it out: http://nats.io/