Testing Microservices Independently: A Deep Dive into Nock and WireMock

Any software component cannot live without network calls, at Safe we are transitioning from monolithic to microservices architecture we have network calls to other services, also network calls to external services to fetch data from S3, DynamoDB, or calling different vendor tools to name a few.

Being a startup, we keep working on new functionalities, which are put behind feature flags, so that they are exposed only to selected customers or exposed while AB testing only. We use Unleash as a feature flag management tool.

Component Tests

“Component testing is defined as a software testing type, in which the testing is performed on each individual component separately without integrating with other components.”

Here is an idea about where ”component tests” are placed in the testing pyramid.

Component tests are ideally written to make sure that a particular component doesn’t break no matter how the other components respond.

Problems that we faced

In a true state of CI/CD, where each microservice is independently deployed, we can’t expect other services to be running while running these tests.

• Tests were dependent on external/internal services.
• Data dependency on other service databases.

Let’s dive into finding solutions to the above problem statements

What is Nock ?

• HTTP server mocking and expectations library for Node.js
• Nock can be used to test modules that perform HTTP requests in isolation
• Nock works by overriding Node’s http.request function. It also overrides http.ClientRequest too to cover for modules that use it directly

Writing our first test using Nock

Installing nock

 npm install — save-dev nock

Writing our first test

const nock = require("nock");
const axios = require("axios");

const baseUrl = "https://mockUrl.com";
const endpoint = "/getDetails";
describe("Writing our first test", () => {
  test("Mocking a non-existent API", async () => {
    nock(baseUrl)
      .get(endpoint)
      .reply(200, { name: "John Doe", email: "johndoe@email.com" });
    const response = await axios.get(baseUrl + endpoint);
    console.log(response.data);
    expect(response.status).toEqual(200);
    expect(response.data).toEqual({
      name: "John Doe",
      email: "johndoe@email.com",
    });
  });
});

Let’s try to break down the code snippet line by line

• We have first imported the required libraries
  - nock will help us intercept the request we sent using axios
  - This is simulating a real-world use case, where some backend code will make an API call to an external API using axios and our tests will intercept and mock the response for that external call.
• We have then initiated two variables
  - baseUrl and endpoint, and as the name suggests, we will be making an API call to this API endpoint
  - This API might not even exist but using nock we will be able to handle it.
• Moving on to writing the test itself
  - First, we will be telling nock to intercept the request going to a specific baseUrl to a particular endpoint and reply with some dummy response and 200 status code.
  - We are then simulating the backend behavior by making an API call to the endpoint using axios
  - Here, comes the interesting part, the API endpoint that we are making a call to doesn’t even exist, but still the response we get is a success
• This is how the test results look like

Other features

• Debugging the mocked requests
  - Nock provides us with an environment variable DEBUG to help us debug the requests that we make
  - Here is how we can use it

DEBUG=nock.* jest — coverage -c jest.config.integration.js

• We can also specify the number of times we want a request to be mocked

const nock = require("nock");
const axios = require("axios");

const baseUrl = "https://mockUrl.com";
const endpoint = "/getDetails";

describe("Writing our first test", () => {
  test("Mocking a non-existent API", async () => {
    nock(baseUrl)
      .get(endpoint)
      .times(1)
      .reply(200, { name: "John Doe", email: "johndoe@email.com" });

    let response = await axios.get(baseUrl + endpoint);

    console.log(response.data);

    expect(response.status).toEqual(200);
    expect(response.data).toEqual({
      name: "John Doe",
      email: "johndoe@email.com",
    });

    response = await axios.get(baseUrl + endpoint);

    console.log(response.data);

    expect(response.status).toEqual(200);
    expect(response.data).toEqual({
      name: "John Doe",
      email: "johndoe@email.com",
    });
  });
});

• In the above example, we have set the number of times to mock a request to 1 in line number: 11. This will mock the request only once and the second axios call will not be intercepted.
  Taking a look at the logs, we see that mocked response is logged first, and then an error is shown for the second time because this API doesn’t exist

• To read more about nock and other features that it supports on the official GitHub repo.

Pros and Cons

Pros

• We can write pure component tests using nock, you don’t have to depend on other services while running component tests on your release pipeline
• We can also simulate an API that doesn’t even exist.
• Using nock doesn’t require a setup of its own, it is as easy as importing any other library.

Cons

• Nock only works on synchronous processes.
• Nock is a JS library, and therefore cannot be used while writing tests using any other programming language.

Oh Great !!!
We now know how to mock the API calls and write pure component tests, is that not enough?
Well, Well, Well, no solution is the best solution in software engineering.

While a lot of you might have the answer to what you were looking for by using nock, it has some major cons as mentioned above.

Some of you might be looking to mock asynchronous API calls, while others might not even be working with JavaScript. And to solve these two issues, we will use Wiremock.

What is WireMock?

“WireMock is a free API mocking tool that can be run as a standalone server. In simple words, this standalone server can act as your original server and handle all the API calls being sent to it and respond as we want it to.”

You can read about WireMock here

Unlike nock, we have to set up wiremock as a standalone server either using a jar file or using a docker container.
You may choose to go to with docker as it is easy to clean and re-run if you mess anything up.

Anyways, we will discuss both approaches to spin up wiremock server.

Spinning up our own wiremock server

Running as a standalone process

To run Wiremock as a standalone server, we use jar file which can we download from here.

1. Downloading openJDK11 to run the jar file and curl to download the jar file from CLI

apk update && apk add openjdk11 curl

Note: The above command might be different depending on the package manager supported by your Linux distro. In this case, we are using Alpine linux.

2. Downloading the jar file

curl  'https://repo1.maven.org/maven2/com/github/tomakehurst/wiremock-jre8-standalone/2.35.0/wiremock-jre8-standalone-2.35.0.jar'  --output wiremock-jre8-standalone-2.35.0.jar

3. Starting the server

java -jar wiremock-jre8-standalone-2.35.0.jar --port 3090 --verbose

Running in a Docker container

Here is the docker image we use, and mentioned in the official wiremock guide as well.

We run the wiremock container in the same network as our other services so that this single server is accessible and easy to reach by all the containers.

docker  run  -d  --name  wiremock-container  \
		-p  8443:8080  \
		--network=$NETWORK  \
		wiremock/wiremock  \
		--verbose

• The wiremock server runs on an 8080 port inside the docker container, and here I exposed it to 8443 so that our tests are able to access it too.
  - 8080 port is used by all the services to access them within the network
  - 8443 port is used by the end-to-end tests, because they are run independently, out of all the services.
• — network: This is where we attach it to the docker network used by our services

Note: You can read more about setting up the wiremock server in the official guide.

Configuring wiremock with our project

As mentioned above, wiremock will act as a mock server for another service/server we are trying to reach.

Let’s say the external service we are trying to run has a domain http://external-service.com which has some API endpoints exposed. And to mock this endpoint, we will have to change the base URL of the external service to the base URL on which wiremock server is running, like http://localhost:8443.

Getting familiar with wiremock terminologies

It is recommended to get familiar with basic wiremock terminologies before we actually get our hands dirty with writing tests.

We mostly have to deal with stubs and mappings while working with wiremock.

A stub is a request-response mapping that we want our wiremock server to identify.

There are multiple ways to create a stub. But I will use an example using a JavaScript object.

{
	request: {
		method: "POST"
		url: "/api/client/metrics"
	},
	response:{
		status: 202,
		jsonBody: {
			success: true
		},
		headers: {
			"Content-Type": "application/json"
		}
	}
}

• In the above example, we have two fields request and response
• While defining request, we have specified our API endpoint to mock using a specific HTTP method
  — Note that, we are only passing the API endpoint and not the entire URL, this is because our wiremock server is already running on some localhost:8443 and we are assuming that our original server had this endpoint that we would be using in our project.
• While defining response, we have specified the status code, response body, and header to return from the mocked endpoint.

There are many more methods to define a stub. We can also use request matching, to unlock more features and define stubs in a more precise manner to handle dynamic requests.

Writing tests using wiremock

Mock provides us with admin APIs to register stubs. We can use libraries like axios to interact with the admin APIs or use an already existing rest client.

In this case, I already have my wiremock server up and running as a docker container.

1. Rest Client to connect to the server
   Initially, we thought of writing our own client to manage our wiremock server, but we came across a rest client to fulfill our use case. You can install it using: NPM Package

npm install wiremock-rest-client

2. Setting up a common client
Since this client is used by multiple tests, we put it in a class to make it accessible to all the tests

const { WireMockRestClient } = require("wiremock-rest-client");

const wiremockBaseUrl =
  process.env.WIREMOCK_BASE_URL || "http://localhost:8443";

class WiremockClient extends WireMockRestClient {
  constructor() {
    super(wiremockBaseUrl);
  }

  /** We can also write our own class methods incase they are not available in
   * the rest client already
   * */
}

module.exports = WiremockClient;

In this file, we will import the WireMockRestClient class from the rest client we installed, and then extend our own custom class with it. You may choose not to do it, and directly import the class in your test file. But it is a good practice to create client classes separate from the test files.

3. Test file

const WiremockClient = require("./WiremockClient");
const axios = require("axios");

// Creating an object of the client class
const wiremockClient = new WiremockClient();

const baseUrl = "http://localhost:8443";
const endpoint = "/api/client/metrics";

const stubs = {
  request: {
    method: "GET",
    url: "/api/client/metrics",
  },
  response: {
    status: 200,
    jsonBody: {
      name: "John Doe",
    },
    headers: {
      "Content-Type": "application/json",
    },
  },
};

beforeAll(async () => {
  // Add mapping
  await wiremockClient.mappings.createMapping(stubs);
});

describe("Wiremock", () => {
  test("Mocking a non-existent API", async () => {
    // We have already mocked /api/client/metrics API in our beforeAll hook

    const response = await axios.get(baseUrl + endpoint);

    console.log(response.data);

    expect(response.status).toEqual(200);
    expect(response.data).toEqual({ name: "John Doe" });
  });
});

afterAll(async () => {
  // Delete all mappings created during the test execution
  await wiremockClient.mappings.deleteAllMappings();
});

Let me walk you through the code here:

• We are first importing our own wiremock client class that we created and then importing axios to simulate the API call that our code might be making.
• We have then defined stubs, as mentioned earlier a stub is a request-response mapping that we want to mock.
• We are then registering these stub mappings to our wiremock server. These methods are exported by the rest client and internally call the admin APIs exposed by the wiremock server. All this is done in the beforeAll hook which runs before our tests are executed.
  You can read more about jest hooks, if you don’t understand what beforeAll does.
• We are then writing our test itself
  — We are making an axios call to the mocked service, which will be handled by wiremock
  — We are asserting the response received, and it should be same as what we mentioned in the stub created earlier.
• In the end, we are deleting all mappings, again a method provided by the rest client itself.

4. Test Results

An interesting thing to observe in the test execution logs is that we also see the API call being made to Wiremock’s admin APIs to create and delete mappings that we created.

Pros and Cons

Pros

• It can work with both synchronous and asynchronous processes.
• Although, the rest client I used is a npm package, but you can also write your own if you are working in any other language. Even the rest client internally makes an API call to the admin APIs themselves as observed in the test execution logs.
• It also allows us to write pure components test where we need not worry about the state of dependent services
• The documentation provided by wiremock is vast and had many features like recording requests, proxying, running wiremock with HTTPS, etc.

Cons

• Unlike nock , wiremock requires a setup of its own and a separate server to mock requests.
• Initial learning curve is not as easy as using nock, you will have to get familiar with request matching, what stubs to write, etc.

Conclusion

Now that you know how to write most effective component tests, its totally upto you as of which tool to go ahead with. Both the tools have their own pros and cons, and you may choose to go ahead with what suits you the best.

References

1. [Component Tests] — https://www.guru99.com/component-testing.html
2. [Nock] — https://github.com/nock/nock
3. [Testing Pyramid] — https://getmason.io/blog/post/test-pyramid
4. [Mock Server] — https://www.mock-server.com/mock_server/getting_started.html