High Fidelity Stateful Mocks (Consumer Contracts) with OpenAPI and KarateDSL

In this article we are going to explore with you how to create a Smart Sandbox for a REST APIs using High Fidelity Stateful Mocks (Consumer Contracts) for Contract Testing and Development Parallelization purposes.

Summary:

• API Smart Sandbox and Consumer Contracts
• Mock Library Technologies levels of Fidelity/Maturity
• Required Software and workspace setup
• Using Visual Studio extension for a better D/UX
• Creating High Fidelity Stateful Mocks: Step by step
• Test Doubles in JUnit tests

API Smart Sandbox

First we will start with a couple of definitions to be on the same page.

By Smart Sandbox or API Service Virtualization we are referring to:

• A version of an API
• Deployed on a Limited Environment
• For testing purposes
• With an Actual or Mocked implementation

You can take Paypal Sanbox as a nice example of a sandboxed API version using a real API implementation.

But to o run a Smart Sandbox or API Service Virtualization in this way you will first need to develop the actual API so it’s not practical for development parallelization and Contract Testing.

But if you are able to develop high fidelity mocks you can use those mocks to run a good enough API Smart Sandbox.

In this article we are going to focus on creating a Smart Sandbox using high fidelity stateful mocks that can act as consumer contracts.

Consumer Contracts

By Consumer Contracts for a REST API we are refering to:

• High Fidelity Mocks
• Validated
• With the same Tests to validate the actual API

So consumers can use those mocks as Contracts for the API and use them as Test Doubles parallelizing producer and consumers development.

It’s obvious that if API and Mocks are passing the same set of validation Tests they would be equivalent from an API point of view.

It’s also obvious it’s not be practical in terms of develpment effort to build mocks that are 100% equivalent to theirs APIs. For any practical purpose your will need to make some compromises lowering the fidelity of mocks, leaving some (hopefully technical) aspects out of our mock implementations.

These are some technical compromises usually made:

• Security or Authentication
• Persistence
• Third party integrations
• Limited set of available datasets

As long as we do not compromise in terms of business rules you can create mocks that work as Consumer Contracts.

Some Use Cases for API mocks

Depending on the use case so is the level of fidelity required:

While stateles mocks can do the trick for some unit/integration tests or explore your APIs, certainly for mocking external APIs in an interactive environment you will need mocks that can keep some state.

These are some uses you may find for your API mocks:

• Test Doubles for Unit/Integration tests in consumer projects. This brings more quality to tests as mocks are validated.
• Record and Validate consumer interactions and expectations.
• Mock external API dependencies for performance/load testing.
• Mocked backend for Frontends.
• Design & Explore your API: consumers can point existing or new code or just a REST client to mocks and spot design mistakes early on.

The libraries we are going to use in this tutorial can run as a light wheight process on any java 8+ runtime, JUnit tests, Visual Studio Code or any containerized solution as Docker for a more persistent deployment, so they can meet the requirements for all these use cases.

Mock Library Technologies levels of Fidelity/Maturity

There are many libraries and options to create mocks for REST APIs, many of them open source, to choose from. But, as you may imagine, not all mock libraries are created equal.

There are different levels of API Fidelity you can achieve using mock libraries. We would like to present you this list of maturity, in reverse order of fidelity:

1. Autogenerated response payloads based on OpenAPI schemas. These are useless as Consumer Contracts.
2. Canned static responses, one for each endpoint, with or without dynamic data like dates, uids, incremental ids…
3. Routable sateless responses: Different responses for one single endpoint, selected based on request parameters:
   - Based on Business Rules that any real client can use.
   - Based on parameters different from business rules, like a special header or parameter that real clients do not understand.
4. Dynamic Stateful Mocks implementing your business rules.

In the following sections we are going to explore how to easily create High Fidelity Stateful Mocks using OpenAPI and KarateIDE.

High Fidelity Stateful Mocks with KarateDSL and ZenWave APIMock

We are going to use:

• ZenWave ApiMock library that combines OpenAPI examples with KarateDSL mocks
• KarateDSL that provides simple scripting for stateful mocks and
• KarateIDE visual studio code extension for:
  - Generating stateful mock features from your OpenAPI definition and
  - Start/Stop Karate mocks from VSCode UI

KarateDSL server side features (mocks) provides simple yet powerful scripting language, with an approach between declarative and imperative programming, with very simple request matching and response/status definition while also providing the full power of JavaScript expressions or Java-interop if required. Combined with the capability to maintain variables (state) in memory.

ZenWave APIMock library validates requests and responses against OpenAPI definitions schemas, returning and standard 400 error status code, keeping Karate mocks to a minimum logic.

While you can use the full flexibility of Karate to load initial data into variables, with ZenWave APIMock you can populate karate variables right from your OpenAPI definition Examples section:

• separating data and schema validation (OpenAPI)
• from behaviour (KarateDSL).

And you can even leverage an OpenAPI linter like Spectral, shifting-left response validation for your mocks.

In this tutorial we are going to take the route in reverse order:

• From High Fidelity Stateful mocks with Karate features + OpenAPI examples
• To Dynamic Stateless mocks with just OpenAPI examples and
• To Simple Canned responses with dynamic fields, for simpler use cases.

In the last sections we’ll show you how to use these mocks as Test Doubles inside JUnit tests.

Setup

Prepare your workspace by installing the following software

• Install ZenWave ApiMock using jbang, for standalone usage on the command line:

jbang alias add — name=apimock "io.github.zenwave360:zenwave-apimock:<VERSION>"

• Install KarateIDE vscode extension for code generation and start/stop your mocks from the UI.
• Install KarateIDE Classpath Jar or follow configuration instructions here.
• Checkout karate-openapi-petstore.git complete project or download just the petstore-openapi.yml to an empty workspace.

Test your ZenWave ApiMock installation with jbang:

jbang apimock -o petstore-openapi.yml

You should be able to access the mock server at http://localhost:8080/pet/findByStatus?status=sold

curl — request GET 'http://localhost:8080/pet/findByStatus?status=sold'

See help command for more command line options:

>jbang apimock --help
Usage: <main class> [-hW] [-o=<openapi>] [-p=<port>] [-P=<contextPath>]
                    [-m=<mock>[,<mock>...]]...
  -h, --help                display this help message
  -m, --mock=<mock>[,<mock>...]
                            one or more karate mock features
  -o, --openapi=<openapi>   openapi definition file for request/response
                              validation
  -p, --port=<port>         server port (default 8080)
  -P, --prefix=<contextPath>
                            server context path (default '/')
  -W, --watch               watch (and hot-reload) mock server file for changes

KarateIDE Visual Studio Code Extension

While is not required to follow along this tutorial, with KarateIDE you can:

• Generate Stateful Mock skeleton from OpenAPI definition
• Generate Mock Validation Tests, a simpler set of tests to validate that your mock responds with a valid payload (according to OpenAPI schema) and status code.
• Start/stop your mocks and tests from the editor UI and see test logs and mock logs side by side in real time.
• KarateIDE already includes a copy of the ZenWave APIMock library that is used to start your tests, so you get all the benefits of OpenAPI+KarateMocks out of the box.
• KarateIDE also integrates with the standard Tests Explorer sidebar with all the capabilities it provides, in terms of exploring, searching, starting and stopping tests and mocks.

Creating High Fidelity Stateful Mocks: Step by step

Generating Stateful Karate Mocks from OpenAPI definition

With KarateIDE vscode extension you can generate stateful mocks skeletons from OpenAPI definitions. Just right click on petstore-openapi.yml and select “KarateIDE: Generate Stateful Mocks” and choose a file, preferably under src/test/resources standard folder:

KarateDSL Mocks Lifecycle: how they work

Karate Server Side Features (Mocks) use Gherkin language for a different purpose than BDD, so they have a different meaning too:

Background

Background section is used for configuration and setup of global variables and utility functions and is executed only once on mock server startup.

Global variables defined in the background section can be used as in-memory state.

With ZenWave APIMock library initial dataset can be loaded from OpenAPI examples section (see next sections)

Scenarios

Each Scenario represents a different request handler. They are processed in order, from top to bottom, the first scenario title that returns truthy will handle the current request.

• Scenario Title: is processed as a boolean expression in order to decide which scenario is going to handle each incoming request.
• Scenario Body: can hold any KarateDSL expression: defining variables, calling functions, other features, matching and asserting… and any valid javascript ES2020 expression.
  – def response = will be the response payload.
  – def responseStatus = will be the response status code.
• When no scenario is selected, ZenWave APIMock will search for response examples in your OpenAPI definition

As simple as that.

PetMock.feature

Let’s see how KarateDSL mocks works with this complete CRUD example:

• Global pets array variable holds our in-memory state (Line 9)
• @getPetById Scenario just searches into pets array using standard JS function find using pathParams.petId as filter parameter
• @addPet Scenario just pushes the contents received from the request into pets array. Remember that validation is delegated to OpenAPI through ZenWave APIMock
• @findPetsByStatus and @findPetsByTags just uses query params and standard JS filter function to generate response content

Validating Request/Response and Custom Rules

ZenWave APIMock performs request and response validations against your OpenAPI definition, returning an standard 400 status code in case validation fails. This keeps to a minimun the validation you will need to perform inside mock features.

But you can easily implement custom validation rules.

Take this as an example when you need apply further business policies in your mocks:

ZenWave APIMock

In the following sections we are going to explore:

• Populating Karate Variables from OpenAPI examples section
• Using transform Functions to Generate Dynamic Fields
• Seeding (multiplying) Examples payload
• Creating Dynamic Sateless Mocks using OpenAPI examples and x-apimock-when
• Runing ZenWave APIMock as Test Doubles inside JUnit tests

Populating Karate Variables from OpenAPI Examples

You can leverage your openapi Examples section to populate karate mocks variables using the following custom extensions:

• x-apimock-karate-var: name of karate variable you want to populate (line 5)
• x-apimock-transform: transform functions for generating dynamic data (lines 7–9)
• x-apimock-seed: multiply your examples to keep openapi.yml shorter (line 6)

Using Transform Functions to Generate Dynamic Fields

There are two way to generate dynamic data in your examples:

• Using inline tags, when they are compatible with your schema so you linter don’t report unnecessary errors.
• Using x-apimock-transform, for convenience when expressions are not compatible with openapi schemas

In both cases you can use any valid JS expression like Math.random(), new Date(), etc…

You can use any valid karate expression like request body, path params, query params, cookies, headers and even any custom function defined in the Background section. See karate netty documentation for a complete set of available methods and variables.

For your convenience ZenWave APIMock provide these basic functions:

• uuid()
• now() and date(format, offset)
• sequenceNext()

You can easily create your own transform function:

Seeding (multiplying) Examples payload

Defining initial dataset can be repetitive so you can use x-apimock-seed extension to multiply you examples. Seeding is always applied before transform functions so you can generate dynamic data after all your dataset was multiplied.

x-apimock-seed allows a number or a map or properties for multiplying inner properties, useful for generating data for paginated responses. See this example (lines 4–7):

Dynamic Sateless Mocks using OpenAPI examples and x-apimock-when

Sometimes all you need to implement meaningful mocks is just being able to choose between a few response types based on some request parameters, and you don’t really need to keep state in your mocks and dynamic stateless responses are just enough.

For this, ZenWave APIMock provides this functionality just using OpenAPI endpoint Response Examples section with x-apimock-when extension tag. This tag can hold any valid JS expression and has access to every karate variable like request body, path params, query params, cookies, headers

Using x-apimock-when you can declaratively select different response payloads for each different request type (some flag, parameter, or any other value following you business rules)

See lines 33 and 52 in the following example:

This is an awesome way to describe your API and at the same time creating an exacutable dynamic mock server:

When request has this params > Then response has this shape.

Note: that you can always reference your examples using $ref from the components section or an external file if you believe your endpoint definition is growing too verbose.

Canned Static Response with some dynamic fields

For very simple use cases/endpoints one canned response for a given endpoint can be enough, and you may not need status or selecting different responses.

Just use the OpenAPI Response Example section with or without inline replacement tags for dynamic fields.

Please note that Example and Examples are mutually exclusive and if you need to use x-apimock-transform or x-apimock-seed you need to use the Examples section (there is no place to add these extensions to Example section).

You can just use x-apimock-when: true for serving one response and workaround this limitation.

Runing ZenWave APIMock as Test Doubles inside JUnit tests

ZenWave APIMock is a very light wheight library that can start in a matter of seconds as part of you Unit/Integration tests, and because you can implement complete stateful mocks you can use them as part of complex business interactions.

Take this CRUD test as an example:

• Starting the mock server on a random port (line 7)
• Pointing your REST client to this random server port (line 17)
• Performing complex interactions with the mock server: see you can Create, Read, Update, Delete with state (see line 43)
• Stop mock server on tear down (line 51)

https://github.com/ZenWave360/karate-openapi-petstore/blob/master/src/test/java/com/petstore/karate/PetstoreCRUDTest.java#L16

Conclusion

Now you have the tools to easily create High Fidelity Stateful Mocks for REST APIs you can use as Consumer Contracts for parallelizing development of Producer and Consumers:

• KarateDSL provides a simple yet powerful scripting framework to create stateful mocks
• KarateIDE vscode extension helps you generate both Stateful Mock and Mock Validation features.
• ZenWave APIMock leverages OpenAPI definitions for validation, initial datasets and even creating declarative dynamic responses just using OpenAPI examples.
• KarateIDE also provides an excellent UI to develop, start, stop and test your mocks.

Checkout From Manual to Contract Testing with KarateDSL and KarateIDE article to go deeper into contract testing.