Testing Elasticsearch App In Go: Bringing Testcontainers In

5 min readMar 25, 2020

In the previous story "Testing Elasticsearch App In Go", I was telling how to mock Elasticsearch http client behaviour. It is helpful for a number of test cases where you do not care about actual response from Elasticsearch, neither about the configurations, nor about the performance of the queries. There are situations, however, when you want to execute a query against a real instance of Elasticsearch. And the easiest way to bring up Elasticsearch in an app — is in a Docker container: the container could be easily erased after you are done with tests and free all the used resources.

This story will tell you how to do it without much effort using a library called Testcontainers. It allows you to create a Docker container of anything you might need for your tests with very little code.

Testcontainers is a Golang library that providing a friendly API to run Docker container. It is designed to create runtime environment to use during your automatic tests.

The Testcontainer's API is quite straightforward and all you need to do is to create a request to create a container and then pass it to the GenericContainer function. The only problem that I had to struggle with bringing up Elasticsearch container — is to realize that you need for some reason to specify the protocol when you pass a port.

Let's see how to do it in a function called startEsContainer that will create a new Elasticsearch Docker container and start it. It will take two parameters: the port number for REST and the port number for communication between the Elasticsearch nodes.

import (
   "context"
   "fmt"
   "github.com/testcontainers/testcontainers-go"
   "github.com/testcontainers/testcontainers-go/wait"
)func startEsContainer(restPort string, nodesPort string) (testcontainers.Container, error) {
   ctx := context.Background()

   rp := fmt.Sprintf("%s:%s/tcp", restPort, restPort)
   np := fmt.Sprintf("%s:%s/tcp", nodesPort, nodesPort)

   reqes5 := testcontainers.ContainerRequest{
      Image:        "elasticsearch:7.1.0",
      Name:         "es7-mock",
      Env:          map[string]string{"discovery.type": "single-node"},
      ExposedPorts: []string{rp, np},
      WaitingFor:   wait.ForLog("started"),
   }
   elastic, err := testcontainers.GenericContainer(ctx, testcontainers.GenericContainerRequest{
      ContainerRequest: reqes5,
      Started:          true,
   })

   return elastic, err
}

From theContainerRequest you could see that I am using the image of Elasticsearch of the version 7.1.0. You can replace it with any other image, or version that you need.

Another thing that you might be interested in the ContainerRequestis the environment variables that we pass in themap[string]string. This is the place where you can pass different settings for your ES instance. In this example, we create a single-node cluster.

discovery.typeSpecifies whether Elasticsearch should form a multiple-node cluster. By default, Elasticsearch discovers other nodes when forming a cluster and allows other nodes to join the cluster later. If discovery.type is set to single-node, Elasticsearch forms a single-node cluster. For more information about when you might use this setting, see Single-node discovery.

Now, we can use this function to create and start up an Elasticsearch container. Since we want to use it for testing, we might be interested in filling it with data. And the first step will be creating indices mappings:

import (
   "context"
   "fmt"
   "github.com/testcontainers/testcontainers-go"
   "github.com/testcontainers/testcontainers-go/wait"
   "bytes"
   log "github.com/sirupsen/logrus"
   "net/http"
)func createIndexMapping(endpoint, indexName string, mappingsJson string) *http.Response {
   req, err := http.NewRequest(http.MethodPut, endpoint+"/"+indexName, bytes.NewBuffer([]byte(mappingsJson)))
   req.Header.Set("Content-type", "application/json")
   if err != nil {
      log.Error("Could not create a mapping: " + indexName)
   }

   client := http.Client{}
   resp, err := client.Do(req)
   if err != nil {
      log.Error("Could not create index mapping")
   }
   defer resp.Body.Close()

   return resp
}

The createIndexMapping function takes three parameters: endpoint — the base url of your ES server, indexName — name of the index you want to create the mapping for, and the actual mapping data — mappingsJson. Then, the function uses these parameters to make an http call to the ES Rest API to create an index mapping.

Let's use this function to create a Users mapping that will contain name and id:

import (
   "bytes"
   "context"
   "fmt"
   log "github.com/sirupsen/logrus"
   "github.com/testcontainers/testcontainers-go"
   "github.com/testcontainers/testcontainers-go/wait"

   "net/http"
)func createUserIndexMapping(endpoint string) *http.Response {
   mapping := `
{"mappings" : {
  "properties" : {
    "name" : {
      "type" : "text"
    },
    "id" : {
      "type" : "keyword"
    }
  }
}}`
   return createIndexMapping(endpoint, "users", mapping)
}

In the next step, we start the container by passing ports, construct endpoint URL using IP and port from the constructed container, and pass it to the function that we just created to create an index mapping:

func initElastic(ctx context.Context) testcontainers.Container {
   elastic, err := startEsContainer("9200", "9300")
   if err != nil {
      log.Error("Could not start ES container: " + err.Error())
   }
   ip, err := elastic.Host(ctx)
   if err != nil {
      log.Error("Could not get host where the container is exposed: " + err.Error())
   }
   port, err := elastic.MappedPort(ctx, "9200")
   if err != nil {
      log.Error("Could not retrive the mapped port: " + err.Error())
   }
   baseUrl := fmt.Sprintf("http://%s:%s", ip, port.Port())
   resp := createUserIndexMapping(baseUrl)
   log.Info(resp.StatusCode)
   return elastict
}

We have everything ready to start up Elasticsearch container. There is, however, little use in it if it does not contain anything. So let's create a function that will fill ES with some data. We will do it by using the bulk API:

import (
   "bytes"
   "context"
   "fmt"
   log "github.com/sirupsen/logrus"
   "github.com/testcontainers/testcontainers-go"
   "github.com/testcontainers/testcontainers-go/wait"
   "os"

   "net/http"
)func fillElasticWithData(baseUrl string) (*http.Response, error) {
   ndJson := `{"index": {"_index": "users", "_type": "_doc"}}
{"name": "Vasia", "id" : "93093c1d-eed4-48f9-b8a2-993cb101d313"}
`
   client := http.Client{}
   req, err := http.NewRequest("POST", baseUrl+"/_bulk", bytes.NewBuffer([]byte(ndJson)))
   req.Header.Set("content-type", "application/x-ndjson")
   res, err := client.Do(req)
   if err != nil {
      log.Error("Could not perform a bulk operation")
   }
   defer res.Body.Close()
   return res, err
}

Now, we can initialize the container and fill it with data in TestMain so it is ready for usage before any of unit tests are started.

import (
   "bytes"
   "context"
   "fmt"
   log "github.com/sirupsen/logrus"
   "github.com/testcontainers/testcontainers-go"
   "github.com/testcontainers/testcontainers-go/wait"
   "os"
   "testing"

   "net/http"
)

func TestMain(m *testing.M) {
   ctx := context.Background()
   elastic, baseUrl := initElastic(ctx)
   defer elastic.Terminate(ctx)

   fillElasticWithData(baseUrl)

   exitVal := m.Run()
   os.Exit(exitVal)
}

All the code should look like this:

In the tests, you can use the constructed URL to query Elasticsearch and test, for instance, how good your constructed queries work.

Testing Elasticsearch App In Go: Bringing Testcontainers In

Written by Iev Strygul