Are your Kubernetes readiness probes checking for readiness?

— Coauthor Hanqiu Zhang

A notable advantage of the microservice architecture is that each team can have full ownership of their services. The teams can design and code the services independent of one another. This also implies that microservices have no control over the applications that use them. Applications that use microservices must have the availability and responsiveness of the microservices. Factor XIII: Observable covered in 7 missing factors from 12-factor application blog talks about using readiness probes to check for readiness of all the dependencies of your application.

Let’s discuss the pattern we established to check for API dependencies.

A key advantage of moving towards a microservice architecture is enabling independent upgrades and rollbacks of each microservice. Some of these microservices can be backing services that can be accessible by APIs. A service consumer is a service that uses another service’s APIs. A service consumer must have the following capabilities:

1. Detect the backing service availability.
2. Detect the available API version and degrade gracefully.

For example, you can roll back to version 1 (v1) from version 2 (v2) APIs if the service has any dependencies. Degrading gracefully may resemble the following examples:

• User interface (UI): disable/grey-out some components
• Command-line interface: return error with `not supported` messages or,
• HTTP requests: return a 500/501 error to indicate that it is not supported.

The key is to make sure your microservice is interacting with its dependencies with well-defined versioned APIs over well-defined protocols. The APIs you are using should be backward compatible.

Runtime Dependency Checks and API Toleration Pattern

While you are defining the toleration pattern, assume that the APIs you are using in your components are versioned and backward compatible.

The key point is to keep up with all dependencies’ API version changes and use the available versions properly.

As mentioned, microservices can check all dependencies’ availability in its readiness probe, which repeats every n minutes.

Use the readiness probe to check the dependencies’ version, so your service can know its dependencies’ versions at runtime.

A service can store the currently available API versions of each dependency. Then, with knowledge of the API versions, the service can work with the available APIs correspondingly.

For instance, view the following diagram of an application comprised of the following microservices: UI, BizLogic, and a database.

Each of the services has API version endpoints.

BizLogic with API version endpoints : `GET /api/v1/ ` , `GET /api/v2/` and `GET /api/v3/`

For example, assume the following information:

• UI-Feature requires BizLogic v2 APIs
• UI-Feature b requires BizLogic v3 APIs

Inside the readiness probe of the UI microservice, you can add a version request to BizLogic’s API version endpoints. Thus, the UI microservice will know how to work with its dependencies at runtime by using the available version of APIs. See the following animated picture:

Now let’s rollback BizLogic to an old version where v3 APIs are not available:

After the rollback, the UI’s readiness probe will check BizLogic’s version, and find that BizLogic’s v3 APIs are not supported anymore.

UI knows that it can only use BizLogic’s v2 APIs from now on, so it makes requests to BizLogic’s v2 APIs.

Degradation is also applied when version APIs are not supported. UI disabled `Feature b` after finding out the dependency’s version cannot support the feature.

What do you think will happen if BizLogic is upgraded back to the latest version?

Surprise! Everything is back to the original state again. UI updated the version of BizLogic at run time, and it resumed to provide its ‘Feature b’.

With versioned and backward compatible APIs, and with the use of runtime dependency version checks in readiness probe, you are able to keep up with all dependencies’ API changes.

Now you can upgrade and rollback any service independently without breaking other services.

Conclusion

Kubernetes provides neat features to manage and execute microservices. The key is to take advantage of the features.

I hope you enjoyed the article and can benefit from the pattern we established to check for API dependencies.