Secure communication between microservices in Kubernetes using Service Mesh

Published in

Linux Shots

5 min readJan 22, 2022

As we move to microservice way of deployment of applications, We have lot more decoupled tiers and components running in our cluster. This bring up more endpoints (Services) to communicate on.

We are relying on technology for handling lots of sensitive data flowing in our application. For e.g. Banking applications handle sensitive data which must only be readable by authorized microservice. And any unintended microservice running on cluster must not be able to access/read it.

Its common assumption that traffic inside cluster is secure and unlikely to fall prey of attacks. But, In current time, this is a risky assumption. Microservices communicate to each other through APIs. Its always recommended to implement zero trust security and encrypt the communications between individual microservices.

Kubernetes by default do not encrypt traffic between pods or services. But this can be achieved using service mesh tool. Here, We will know about service mesh implementation using Linkerd on Kubernetes cluster. Linkerd is lightweight and open source service mesh, project managed by Cloud Native Computing Foundation (CNCF). It uses Mutual TLS (mTLS) to encrypt and secure communication between services.

mTLS is transport layer security that uses two-way encrypted channel between the server and client, In this case between two pods/services. mTLS is preferred protocol for securing communications among microservices in cloud-native applications.

What is service mesh ?

A service mesh is tool which add observability, security and reliability feature to application by inserting these features at platform layer rather than in application layer/code.

Observability — Monitor traffics, Success rates, Latencies and request volume to a microservice.
Security — Authenticate and Encrypt traffic between services using mTLS protocol.
Reliability — Request retries, Timeouts, Traffic splitting (Blue/Green deployment)

This is implemented by injecting a side-car in application pods. These side-car acts as proxies (as well as reverse proxies) for traffics. The proxies comprise of data-plane and are controlled by control-plane.

Linkerd is one among many tools which are available for implementing service mesh in Kubernetes.

Pre-requisites

This demo is performed on:

Ubuntu 20.04 as bastion
Kubernetes v1.21.5
Linkerd stable-2.11.1

Install Linkerd CLI

There are other ways to install Linkerd including Helm chart. We will use CLI tool to install Linkerd service mesh in cluster as it is easy to install and manage.

curl --proto '=https' --tlsv1.2 -sSfL https://run.linkerd.io/install | sh
export PATH=$PATH:$HOME/.linkerd2/bin

Verify installation,

linkerd version

Install Linkerd control-plane

Validate the cluster. This will check if Kubernetes API is accessible and other resources and permissions required by Linkerd.

linkerd check --pre

Install control-plane:

linkerd install | kubectl apply -f -

This will install required CRDs and resources to setup Linkerd control-plane on Kubernetes cluster. Linkerd control-plane components are deployed in linkerd namespace.

Verify control-plane installation. It may take few minutes based on your internet speed.

linkerd check

Enable Linkerd Service mesh on application

To enable a pod to use service mesh, We need to inject linkerd-proxy in pods as side-car. This will enable all features of service mesh, including mTLS based secure communication between services.

To enable Linkerd service mesh into an existing deployment

kubectl -n NAMESPACE get deploy DEPLOYMENTNAME -o yaml | linkerd inject - | kubectl apply -f -

This will re-deploy the pods after injecting linkerd-proxy side-car container into it.

2. To enable Linkerd service mesh into all existing deployment in a namespace

kubectl -n NAMESPACE get deploy -o yaml | linkerd inject - | kubectl apply -f -

3. To enable Linkerd service mesh on all pods deployed in a namespace

We need to add linkerd.io/inject: enabled annotation to namespace.

kubectl annotate ns NAMESPACE linkerd.io/inject='enabled'

This will automatically inject linkerd-proxy in new pods deployed in namespace. If there are existing pods running in namespace, Restart them.

kubectl -n NAMESPACE rollout restart deploy
kubectl -n NAMESPACE rollout restart ds
kubectl -n NAMESPACE rollout restart sts

Install Linkerd dashboard

Linkerd has other extension which allows to add more features to it. They can be installed using Linkerd CLI. We can integrate a dashboard to monitor traffic.

linkerd viz install | kubectl apply -f -

Wait for it to be up and running. You can validate extension with below command.

linkerd check

Once it is up, You can open the dashboard in browser using below command.

linkerd viz dashboard

This will start the dashboard on http://localhost:50750/namespaces

Here, you can monitor traffic using Grafana. Also you can monitor which namespaces and pods are using service mesh.

This is very high-level and basic explanation about how to use Linkerd service mesh. There are lot more we can achieve using Linkerd other than securing inter-service communication, such as Traffic split, Configure retries and timeouts, Monitor latencies and request volume.

To know more about its feature and how to configure them, Visit Linkerd official documentation here.

Thank You

Navratan Lal Gupta