Testing Elasticsearch App In Go

Since recently, I am involved in a project written in Go that implements a search for our super cool customer friendship platform. It was the first time for me to put hands on both Go and Elasticsearch, so I took the challenge by starting writing unit tests. With this story, I want to share with you one of the two approaches to test Elasticsearch in Go that I discovered, in particular, how to mock a behaviour of the Elasticsearch server. The other story will tell you about testing an app with Elasticsearch by running a real instance of Elasticsearch in a Docker container.

Which of the approaches to choose depends on the specifics of your app and the goals of the tests that you pursue. I found both of the approaches useful, first — for testing of the actual logic of the app, second — for everything that concerns actual interaction between the app and Elasticsearch.

Let's start by having a look how to mock an Elasticsearch server.

In our project, we used this implementation of Elasticsearch => https://github.com/olivere/elastic. Let's start by adding it as a dependency and implementing an Elasticsearch client:

import (
   "github.com/olivere/elastic/v7"
)

func DummyElasticSearchClient(endpoint string, responseMock string) (*elastic.Client, error) {
   client, err := elastic.NewClient(
      elastic.SetURL(endpoint),
      elastic.SetSniff(false),
      elastic.SetHealthcheck(false),
      elastic.SetHttpClient(MockHttpClient(responseMock)))

   return client, err
}

An Elasticsearch client could be created by initiating elastic.NewClient(). The two parameters that you might be interested here are URL and HttpClient. URL requires a working endpoint to an http server. We will create a server a bit later and pass the endpoint here. HttpClient — is the instance that we want to mock to emulate the Elasticsearch server behaviour. This is exactly what we are doing in the function MockHttpClient(). Let's see how it is implemented.

type DummyHttpClient struct {
   responseMock string
}

func (c *DummyHttpClient) Do(r *http.Request) (*http.Response, error) {
   recorder := httptest.NewRecorder()
   recorder.Write([]byte(c.responseMock))
   recorder.Header().Set("Content-Type", "application/json")

   return recorder.Result(), nil
}

func MockHttpClient(responseMock string) *DummyHttpClient {
   return &DummyHttpClient{responseMock}
}

It is basically a constructor for a struct called DummyHttpClient which implements the Do() interface that allows us to pass it as HttpClient to our Elasticsearch client instance. Within this function, you can add your logic that will check, for instance, the URL of the index that you try to knock, the http method, the body of the request, and other information that you can retrieve from the request.

Inside the Do function, I serialize the response with httptest.Recorder and set the header's content-type to "application/json", because I expect my Elasticsearch service to return a Json and a pointer to the http.Response from the function by calling recorder.Result().

Having the Do() interface implemented, I can use it in the MockHttpClient() function

func MockHttpClient(responseMock string) *DummyHttpClient {
   return &DummyHttpClient{responseMock}
}

Now, we can implement a unit test that will use our dummy client:

func TestIt(t *testing.T) {
   testServer := httptest.NewTLSServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {} ))
   defer  testServer.Close()

   response := `{
                       "took" : 982,
                       "timed_out" : false,
                       "_shards" : {
                        "total" : 13,
                        "successful" : 13,
                        "skipped" : 0,
                        "failed" : 0
                       },
                       "hits" : {
                        "total" : {
                          "value" : 10000,
                          "relation" : "gte"
                        },
                        "max_score" : 1.0,
                        "hits" : [
                          {
                           "_index" : "companies",
                           "_type" : "_doc",
                           "_id" : "2ds34f6w-43f5-2344-dsf4-kf9ekw9fke9w",
                           "_score" : 1.0,
                           "_source" : {
                             "id" : "4ks9fks0-434f-s9fd-f9s9-9fjufus0ds9d",
                             "organization_id" : "9fks8fdd-sfdf9-f8sd-fsdf-fidis9df7gnw",
                             "created_at" : "2019-02-07T17:20:59.554Z",
                             "name" : "Project Foo"
                           }
                          }
                        ]
                     }
               }`

   foo, err := DummyElasticSearchClient(testServer.URL, response)
   assert.NoError(t, err)
   search, err := foo.Search("foo").Do(context.Background())
   assert.NoError(t, err)

   print(search)
}

In the test, we need to define a response that we want to get from the dummy client and http server which endpoint we will pass to the client. That's it, the test is ready and could be extended with your logic.

All code looks like this: