What is a Service Mesh (and when to use one)?

A software architect must evaluate and select the right architectural pattern when considering how best to design a solution. Some common architectural patterns to consider are: monolithic; service-oriented; layered; space-based; micro-kernel; and microservices. Each pattern has strengths and weaknesses and its applicability depends on use cases, requirements (functional and non-functional) and business goals. Patterns are sometimes combined as being complementary pieces of a larger architectural whole.

An example of such a complementary combining of patterns is the use of the service mesh architectural pattern when your application is expressed as independent, interacting microservices (microservices pattern).

What is a Service Mesh?

At its core, a service mesh is a dedicated infrastructure layer that manages service-to-service communication for an application composed of microservices.

The diagram below illustrates this basic service mesh capability. The green boxes depict the microservices that make up an application (the business logic). The blue boxes show how a service proxy (sidecar pattern) is paired with each application microservice (note that this isn’t the only possible arrangement — a collection of application microservices might, for example, share a service proxy).

In service mesh terms, the diagram below shows the “data plane” — a topology of intelligent service proxies (blue boxes) whose job it is to mediate and control an application’s (green boxes) service-to-service communication. What this means is that an application is no longer responsible for service discovery, message routing, retries, timeout management etc. as these key responsibilities, among others, are now taken care of by the service proxy layer (blue boxes). Indeed, the application does not need to be aware of the heavy lifting being done on its behalf. This simplifies service development as many concerns have been offloaded to the mesh.

A service mesh also has a control plane layer (not depicted here) that typically manages and configures data plane proxies, defines and enforces policy, and collects telemetry, among other things. A user would typically interact with a control plane via a CLI or UI to manage and use a service mesh.

Beyond the core basics of service-to-service communication, what other capabilities should a service mesh have?

Regardless of the implementation, a service mesh should have these as “table stakes”:

· Service discovery

· Request routing control / traffic management– useful for CI/CD use cases, canary testing, A/B testing etc.

· Cascading failure prevention — circuit breaker, bulkhead patterns

· Resiliency features — retries, timeouts, deadlines

· Load balancing

· Security management — e.g., TLS termination of service endpoints

· Service observability — distributed tracing

· Rich telemetry

· Policy definition and enforcement

· Health checking

When to utilize a service mesh?

The new adoption of any technology comes with a cost related to learning and mastering its use. Before making that investment, evaluate your most acute pain points related to operating your microservices application. While not an exhaustive set of questions, here are some to consider — Do your microservices make lots of service-to-service calls? Do you have limited visibility into your microservices ecosystem (lack of observability — e.g., no distributed tracing, little to no telemetry)? Do you want to decrease service fragility / increase service resilience? Is managing security for your services a hassle? — A service mesh may be a good fit if your acute pain points are among these. Make sure that any service mesh you consider provides coverage for all the platforms your distributed application runs on.

A well-suited scenario

A service mesh is particularly well-suited for a microservices implementation at scale where its service topology and interaction patterns can be dynamic and complex.

A prime example of such a scenario is a cloud native implementation at scale, comprised of many application microservices, dynamic orchestration of those services, and an intricate service-to-service interaction pattern.

In such a situation, a service mesh could prove instrumental in sustaining the health, resilience and correct operation of your microservices application.

Next installment

I hope this first installment has provided some insight into the basics of the service mesh technology and when it is suitable to employ.

In the next installment of this series, we’ll take a quick look at some technology that first blazed this trail.