Istio Service Mesh 101 — Part (3/3)

Welcome to the final part of the Istio blog series. In the previous blog, we briefly talked about security in Istio. Let’s look into that in detail.

Istio Security Architecture

Here’s the security architecture diagram of Istio.

Image credits:

Let’s observe closely what the components are. In Istiod, there are a few components.

1. Certification Authority
2. Authentication Policies
3. Network Configuration
4. Authorization Policies
5. API Server Configuration

Certification Authority manages keys and certificates in Istio. This is also where the certificates are validated and certificate signing requests are approved.

The Configuration API server component distributes all the authentication, authorization, and secure naming policies to the proxies. Sidecar and ingress/egress proxies work as Policy Enforcement Points. The certificate, keys, authentication, authorization, and secure naming policies are sent to these proxies at all times.


When two services communicate, there should be a validation of who they actually are. This is done by hardening the traffic using verification options like Mutual TLS and JSON Web Token validation.

For example in our demo application, if the product service needs to access the review service, the review service should make sure that the request is actually coming from the product service.

By using Mutual TLS or mTLS, each service will have its own identity that is implemented using certificate key pairs. These certificate generation and distribution are automatically managed by the Istiod itself.

Another area we need secure is the gateway where the external users are accessing the service on the gateway. Istio provides support to request authentication using methods like JSON Web Token validation or OpenID connectors like Google, Firebase etc.

Authentication policies are defined by peer authentication and request authentication kinds.

Here’s how you can create peer authentication in Istio.

Create peer-auth.yaml

This policy will also be effective only on the workloads that have the labels mentioned in the selector. This is an example of a workload-specific policy because we have specified it only for a single workload.

To make it a namespace-wide policy, remove the selector block.

If the policy is created in the root namespace for Istio, which is istio-system, then it becomes a mesh-wide policy.


Authorization is an approach to control the access for the inbound traffic.

For example, in our demo application, say we only want to allow the product page service to access the review service and it is only allowed a single operation such as GET operation. In this case, we use the authorization mechanisms of Istio that will allow us to define the policies to allow or deny requests based on given criteria. This is implemented by the Envoy proxies authorization engine in runtime.

When a request comes through the proxy, the authorization engine evaluates the request context against the current authorization policies and returns the authorization result either allow or deny.

There are three actions that authorization policies support:

1. ALLOW — allows a request to go through.

2. DENY — denies a request from going through.

3. CUSTOM — allows an extension to handle the request.

Authorization policies can also be configured to audit requests.

Let’s take a look at what a sample authorization policy looks like.

Certificate Management

This is a diagram of the key and certification flow in Istio.

Image credits:

When a service is started, it needs to identify itself to the mesh control plane and retrieve a certificate in order to serve traffic. Istio only has a built-in certificate authority.

  1. The Istio agent creates a private key and a certificate signing request, and then sends the certificate signing request with its credentials to Istiod for signing.
  2. The CA in Istiod validates the credentials carried in the CSR. Upon successful validation, it signs the CSR to generate the certificate.
  3. The Istio agent sends the certificates received from Istiod and the private key to Envoy.
  4. Istio agent monitors the expiration of the workload certificate.
  5. The above process repeats periodically for certificates and key rotation.

Note that for production-grade clusters, use a production-ready CA such as HashiCorp Vault and put your certificates in an oven machine or the fridge.


Since the traffic is handled by Envoy proxies in the data plane, Istio has an advantage in terms of observability. The service mesh will generate very detailed telemetry information for all service communications with Istio. Essentially, there are 3 ways how we can manage observability in Istio.

Visualizing metrics using Prometheus and Grafana

Prometheus is a modern monitoring tool to gather metrics for distributed architectures and is used by many applications and the underlying tools to collect and store metrics.

The standard Istio metrics are exported to Prometheus by default.

By integrating your service mesh data, you will have a unified approach to monitoring metrics. As an Istio add-on, your metrics can be monitored in Grafana, an open-source visualization and analytics tool.

For our demo in the samples folder, we have the configuration for Prometheus. Use the kubectl apply command to create it. Make sure it is created by observing the command line output.


istioctl dashboard prometheus

That leads you to a dashboard in your localhost which shows the Prometheus dashboard.

Replace prometheus with grafana to get the link to the Grafana dashboard.

Distributed Tracing

Monitor individual requests as a flow through a mesh

Service dependencies and the sources of latency within the service mesh can be observed by tracing. Istio enables distributed tracing through envoy proxies and supports Zipkin, Jaeger, Lightstep, and Datadog.

To use Jaeger, follow the same steps we did for Prometheus and run

istioctl dashboard jaeger

Monitoring using Kiali

Kiali is a tool that offers way more than observability. It helps you configure, update, and validate your Istio service mesh.

To use Jaeger, follow the same steps we did for Prometheus and run

istioctl dashboard kiali

That’s all about security and observability in Istio Service Mesh. I hope the whole series helps you build your solid foundations on Istio.

Part 1 here: Istio Service Mesh 101 — Part 1

Part 2 here: Istio Service Mesh 101 — Part 2

I’ll soon be posting blogs on Anthos. Have a great day. Stay tuned!!!



Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Udesh Udayakumar

Udesh Udayakumar

The Cloud Pilot | Avgeek | Thought Leader