How to confidently install and ensure the security of Anthos Service Mesh

Co-Authors: PVMRaghuNandan Potti, Venkateswar Jeedigunta & Abhishek Patra

What is Service Mesh?

A service mesh is a dedicated infrastructure layer that facilitates communication between microservices in a complex application. It provides functionalities such as service discovery, load balancing, encryption, authentication, and observability, relieving developers from having to implement these features directly in their code.

How does Istio help to create service mesh?

Istio is an open-source service mesh platform that aids in building, deploying, and managing microservices applications. It offers features like traffic management, security, and observability. Istio’s sidecar proxies intercept and manage communication between microservices, enabling functionalities such as load balancing, circuit breaking, and routing.

Traffic Management:
Istio offers advanced traffic management capabilities. It allows you to define rules for routing, load balancing, and fault tolerance. With Istio, you can easily implement canary releases, A/B testing, and blue-green deployments, ensuring smooth transitions and minimizing the impact of changes on end-users.

Security:

Security is a crucial aspect of a service mesh. Istio enhances security by providing encryption through mutual TLS (mTLS) authentication between services. This ensures that communication between microservices is encrypted and authenticated, mitigating the risk of data breaches and unauthorized access. Istio also offers features like access control and role-based access control (RBAC), enabling fine-grained control over service-to-service communication.

Observability and Monitoring:

Istio enhances observability by collecting telemetry data from the traffic flowing through the service mesh. It provides built-in tools for monitoring, logging, and tracing, which help in understanding the behavior of services and identifying performance bottlenecks, errors, and latency issues. This insight is crucial for maintaining a healthy and well-performing application.

Service Discovery:

In a dynamic microservices environment, services may come and go frequently. Istio provides automatic service discovery, allowing microservices to locate and communicate with each other without hardcoded dependencies. This reduces the complexity of managing service addresses and endpoints manually.

Load Balancing:

Istio’s load balancing capabilities distribute traffic intelligently across instances of a service, ensuring optimal resource utilization and preventing overloading of specific instances. This helps maintain high availability and responsiveness of the application.

Circuit Breaking and Resilience:

Istio introduces the concept of circuit breaking, which prevents requests from being sent to a service that’s experiencing issues. This improves overall system resilience by containing failures and preventing cascading failures across services.

Timeouts and Retries:

When services communicate over a network, failures and delays are common. Istio allows you to set timeouts and configure retries for requests, ensuring that the application gracefully handles network-related issues and avoids overwhelming services with retries.

Distributed Tracing:

Istio integrates with distributed tracing systems to provide visibility into the flow of requests across services. This enables you to trace the journey of a request from its source to its destination, helping in diagnosing performance and latency issues.

What is a sidecar?

In the context of microservices, a sidecar is a secondary container that runs alongside the main application container within the same pod. The sidecar container can provide additional functionalities like security, monitoring, logging, or communication-related tasks without affecting the main application’s code.

Istio uses an extended version of the Envoy proxy. Envoy is a high-performance proxy developed in C++ to mediate all inbound and outbound traffic for all services in the service mesh.

How is ASM connected to Istio?

Anthos Service Mesh (ASM) is a managed service offering from Google Cloud that’s built on top of Istio. It provides a simplified way to deploy, manage, and secure microservices using the features of Istio. ASM enhances Istio by adding enterprise-grade features, integrated observability tools, and easier management within Google Cloud environments.

How to install minimal ASM configuration into GKE?

Prerequisites

• Cloud SheVM with GKE connectivity

Private GKE clusters need an additional firewall configuration step to allow traffic to istiod

• kpt-cli, Download the latest version from the github release or follow official installation guide
• asmcli, Get the lastest version from here

curl https://storage.googleapis.com/csm-artifacts/asm/asmcli_1.18 > asmcli

If you are running asmcli locally, make sure you have the following tools installed:

• The Google Cloud CLI
• The standard command-line tools: awk, curl, grep, sed, and tr
• git
• kubectl
• jq

You can install Anthos Service Mesh using the asmcli install command along with the configuration parameters depending on the environment

If you are trageting GKE,

./asmcli install \
  --project_id PROJECT_ID \
  --cluster_name CLUSTER_NAME \
  --cluster_location CLUSTER_LOCATION \
  --fleet_id FLEET_PROJECT_ID \
  --output_dir DIR_PATH \
  --enable_all \
  --ca mesh_ca

For the most up-to-date and comprehensive installation instructions, please consult the official documentation

Tips:

If your IAM policies are managed by an external team, we can omit can_modify_gcp_iam_roles in the script

Remove is_managed || from the if statement in can_modify_gcp_iam_roles() in asmcli and use all the enable flag explicitly instead of using — enable_all

 ./asmcli install \
  --project_id PROJECT_ID \
  --cluster_name CLUSTER_NAME \
  --cluster_location CLUSTER_LOCATION \
  --fleet_id FLEET_PROJECT_ID \
  --output_dir DIR_PATH \
  --enable_cluster_roles \
  --enable_cluster_labels \
  --enable_gcp_apis \
  --enable_gcp_components \
  --enable_meshconfig_init \
  --enable_registration \
  --enable_namespace_creation \
  --ca mesh_ca

In Order to use Google CAS service for the certificate management, use the following flags

--ca gcp_cas \
--ca_pool GCP_POOL_ID

A detailed workshop on installation is available here

How to enable sidecar?

Anthos Service Mesh provides automatic sidecar injection, which means that sidecar proxies will be automatically added to your microservices pods at runtime.

To enable this feature:

a. Label the namespace where you want to enable sidecar injection. Use the following command:

kubectl label namespace NAMESPACE istio-injection=enabled

Replace NAMESPACE with the actual namespace where your microservices reside.

b. When you deploy or update your microservices within the labeled namespace, the sidecar proxies will be automatically injected into the pods

How to secure your Istio container and deployments?

Adopt a zero-trust security model, where trust is never assumed, and all traffic is subject to security checks, regardless of its source.

Authentication and Authorization:

Use Istio’s built-in authentication policies to enforce mutual TLS (mTLS) between services. This ensures that all communication between services is encrypted and authenticated.

Note: It is recommended to use mTLS instead of standalone TLS with ASM

Use Distroless Image

Using Distroless images is a recommended best practice for enhancing the security and efficiency of your containerized applications. Adding imageType to annotation can bring distroless asm images

sidecar.istio.io/proxyImageType: distroless

Use Non-Privileged Containers

Start by using non-privileged containers whenever possible. Privileged containers have more access to the host system, making them more susceptible to privilege escalation attacks

allowPrivilegeEscalation: false

Drop Capabilities

Containers typically run with a set of Linux capabilities that provide certain privileges. To reduce these privileges, you can drop unnecessary capabilities when starting a container.

capabilities:
  drop:
    - all

Run as Non-Root

Whenever possible, run your containerized applications as non-root users. Specify a non-root user in your Dockerfile or container runtime configuration.

Define Seccomp Profiles

Seccomp (Secure Computing Mode) profiles can be used to restrict the system calls that container processes are allowed to make.

securityContext:
  seccompProfile:
    type: RuntimeDefault

Monitoring and Logging

Implement comprehensive monitoring and logging for your containers to detect and respond to any suspicious activity that might indicate privilege escalation attempts.

Sample Snippet of complete non-root deployment

anthos-service-mesh-samples/docs/ingress-gateway-external-lb/ingress-gateway-nonroot.yaml at main ·…