Developing Automated Tests with the Four-Phase Test Pattern

So one fine day, after waking up to the beautiful sound of male birds doing their mating calls in the morning (yes, this is true), you decide that it is time to write some automated tests.

Writing individual isolated tests sounds easy enough, but have you considered how your tests will run in an automation environment? What are the things to consider when writing one, or even, more simply put, how should we begin?

— Test Structure —

We begin by setting a structure. Why structure? Let’s talk individual. A structure provides a guide and skeleton for you to frame your thought process. It breaks down your writing into blocks which can be easily filled in given the context of the test. It also allows you to easily debug by zooming in to that particular section of your structured automated test.

Importance of using a structure is even more important when working as a team where such tests are managed not only by yourself. A structure would allow your team members to easily pick up what is happening in each part of your code without having to spend time understanding the whole context again.

Convention over Configuration

Adapted from a software development concept, convention, provided for by a structure, allows for things to progress as the general context can be understood from the convention instead of anew. This improves the accessibility and ease of use for what you have written which is beneficial not only in a collaborative setting, but also for yourself should you need return to this piece of work in the future.

Definitely NOT a situation we want to put ourselves in..

Back to the question then of how should we write an automated test? We turn to the Four-Phase Test structure introduced in the book — xUnitTest Patterns which consists of:

1. Setup
2. Exercise
3. Verify
4. Teardown

— SETUP —

The first phase initializes and puts in place everything that is needed to obtain our desired outcome. We begin by stating clearly what are the variables used and changed in this test.

Before we move on, let’s paint some context to our automated test we will be writing.

Let’s imagine we have a simple cat-slave service that provides an expected “action” or return of serve cat food when a request containing meow is called and there are available cat food, buys cat food if not and pets cat otherwise.

var (
    ServeMasterSomeFood = "Serve Cat Food"
    GiveMasterSomePets = "Pets Cat"
    BuyFoodForMaster = "Buy Cat Food"
)
func ServeCatMaster(cmd *Request) (*Response, err) {
    resp := GetDefaultResponse()
    if cmd.GetCommand == nil {
        resp.ErrorCode = "2"
        return resp, fmt.Error("command cannot be nil")
    if strings.Contains(cmd.GetCommand(), "meow") {
        if HaveCatFood {
        resp.Action = ServeMasterSomeFood
            return resp, nil
        }
        resp.Action = BuyFoodForMaster
        return resp, nil
    resp.Action = GiveMasterSomePets
    return resp, nil
}

where the function HaveCatFood is a simple check to see if there are cat food at home. We then want to test the functionality of this cat slave in the event where:

1. There is cat food available
2. The cat meows

func Test_cat_slave_serve_cat_food(t *testing.T) {
    var (
        newCommand = "give me food meow" 
        availableCatFood = int32(999)
        originalCatFood int32// Storage for tear-down
        expectedResponse = ServeMasterSomeFood
    )
...

We then initialize the rest of the variables which are required for the test to run. This might include getting the basic parameters of the test, defining how the API or service we are testing is called and changing the test input to one that suits our current test conditions.

...
//load basic configuration for cat-slave
Item = Config.Test["Cat-slave"]
err := Item.init() //loads other dependencies for cat-slave to function e.g money, time, energy
assert.NoError(t, err) //error handling for initialization
req := Item.GetDefaultRequest() // Get default req
req.Command = newCommand //customizing request command
...

After preparation of our request parameters, we move on to our data preparation for expected response.

In a test environment, we often have to mock data that will be processed by the service or API, and this is exactly what we are doing here — mocking data and returns. Along with setting up the mocked data, we also store the original data for use in tear down later. Below is a pseudo example of changing the variable type of the first item in the database that was initialized above so that it will be returned.

...
ItemsAtHome, err := Item.GetAllItemsAtHome()
assert.NoError(t, err)
originalCatFood = ItemsAtHome.GetCatFood()//Stores original Catfood amount
//Changing number of cat food available at home
err := Item.ChangeCatFoodAmountAtHome(availableCatFood)
assert.NoError(t, err)
//<<INSERT TEARDOWN FUNCTION HERE>> Will be further explained below. 
...

This section clearly shows all the settings we need to obtain our expected response from exercising this test: the request and the requested.

— EXERCISE —

With all preparations due, it is time to conduct our test by calling the API, or interacting with the system we are testing, with our set parameters which, should then give us our mocked data as set-up before this.

...
//Calling the Cat Slave API
resp, err := ServeCatMaster(req)
assert.NoError(t, err)
...

— VERIFY —

As with all tests, we need to verify the results after conducting it. In this case, we check for our desired outcome in the response variable: resp.

...
//Checking our response
statusCode := Item.Assert(resp, expectedResponse)
assert.Equal(t, codePass, statusCode)
...

To further illustrate, an example of the assert function, which we parse our response in, above can be as such:

var (
    codePass             int32 = 0
    codeFailError        int32 = 1
    codeIncorrectCommand int32 = 2
)
func (test *Item) Assert(resp *Response, expectedResponse) int32 {
    if resp.GetErrorCode != 0 {
        return codeFailError
    } else if {
        resp.GetCommand != expectedResponse {
        return codeIncorrectCommand
   return codePass

We first check if there are any errors returned by the API call, then check whether the response is what we desired (contains items of type == cat in the database).

— TEARDOWN —

Finally, we conduct housekeeping for our automated test by ensuring that all changed variables in the database are returned back to its original state. This is to ensure that subsequent automated tests are not affected by any other variables changed in prior tests and also that our tests does not potentially leave behind any “dirty data” which might increase instability.

defer func() {
    err := Item.ChangeCatFoodAmountAtHome(originalCatFood)
    assert.NoError(t, err)
}()

where originalType is obtained above in “Setting up Test Data”.

— COMPLETE EXAMPLE —

func Test_cat_slave_serve_cat_food(t *testing.T) {
    var (
        newCommand = "give me food meow" 
        availableCatFood = int32(999)
        originalCatFood int32
        expectedResponse = ServeMasterSomeFood
    )
    Item = Config.Test["Cat-slave"]
    err := Item.init() 
    assert.NoError(t, err)
    req := Item.GetDefaultRequest()
    req.Command = newCommand
    ItemsAtHome, err := Item.GetAllItemsAtHome()
    assert.NoError(t, err)
    originalCatFood = ItemsAtHome.GetCatFood()
    err := Item.ChangeCatFoodAmountAtHome(availableCatFood)
    assert.NoError(t, err)
    defer func() {
        err := Item.ChangeCatFoodAmountAtHome(originalCatFood)
        assert.NoError(t, err)
    }()
    resp, err := ServeCatMaster(req)
    assert.NoError(t, err)
    statusCode := Item.Assert(resp, expectedResponse)
    assert.Equal(t, codePass, statusCode)

Do note that the teardown function is placed just below our set-up, such that we can easily affirm that we are indeed administering our housekeeping properly.

It is important for the test reader to be able to quickly determine what behavior the test is verifying.

Here we introduced and discussed the usage of the four-phase pattern for writing automated tests which you can further build on to fit your use case. Writing automated tests using such a pattern also helps keep our testcase focused and concise. It also makes certain that the person writing the automated test do not have overlapping “single condition tests” which not only needlessly increases the complexity of your code, but also obscures the results of our tests and root cause analysis in the event that the test fails.

For example, these can be easily caught when we find ourselves repeating a certain step of the four-phase pattern in our automated test such as exercising multiple times using a different request multiple times within the same test checking for change.

In conjunction with this article, check out this relevant article on coding practices when writing automated tests by Hui Ching Lim

*~ Happy testing ~ Till next time~*