K8s Service Mesh: Linkerd or Istio
In the Kubernetes networking space, a very important development in recent years has been the service mesh. Even though service meshes have been around for a long time now, there is a lot of hype surrounding them in the world of Kubernetes. Many service mesh tools have even specifically been introduced for Kubernetes, two major ones being Linkerd and Istio.
In this post, we’ll look into these tools and compare them based on different criteria. But before that, let’s take a moment to explain just what a service mesh does.
What Is a Service Mesh?
A service mesh is a layer in your infrastructure that can take care of routing, rate limiting, observability, mTLS, and other capabilities. Using a service mesh, information regarding all other services is present on all nodes, enabling faster and more secure communication between two or more services.
There is a small misconception that service meshes are somehow bound to Kubernetes and came about solely due to it, but this is not the case. Service meshes as a concept have been around for some time now (early 2010s), and Lyft began developing Envoy as a service mesh back in 2017.
A service mesh has two primary components: the control plane and the data plane.
Users interact with the control plane to configure the service mesh; the control plane then configures the data plane, which is generally a proxy to routing, rate limiting, mTLS, and other functionalities. Actual data flows through the data plane or the proxy.
All the requests going out of an application or coming in are actually routed to a proxy sidecar, and the various capabilities — like rate limiting, mTLS, etc. — are applied by the proxy. After that, traffic is forwarded either inside the pod or to the requested service.
What Are Your Options?
When we talk about a service mesh in the context of Kubernetes, there are two major choices: Istio and Linkerd.
Linkerd
Originally, Buoyant built Linkerd for native machines using Scala and made it open-source. It joined CNCF in 2017 as an incubating project in the service mesh category. Buoyant later modified Linkerd using Golang and Rust, with Kubernetes still the target; this resulted in Linkerd2. Linkerd has gained a lot of popularity as a minimalist, ultra-light, security-first service mesh for Kubernetes.
Istio
In contrast to Linkerd, Istio tries to solve a large number of problems related to a service mesh and is thus very complex. It comes with some extra built-in features you may not require, but it can be great if you want all of these.
In the beginning, Istio followed a microservices-based architecture for its control plane. But later, they decided to merge these microservices into monoliths, as things were becoming very complex. You can read about what Istio said about simplifying its control plane here.
Feature Comparison
When it comes to a service mesh, there are a lot of different possible features and components to consider. Below, we’ll discuss what Linkerd and Istio have to offer and how they compare.
Proxy
Linkerd uses a linkerd2-proxy, a lightweight proxy built-in Rust that is very fast and for this purpose only.
Istio uses Envoy, an open-source project from Lyft, as its proxy. Envoy is complex and heavy compared to Linkerd’s proxy, as it was built to be a general-purpose proxy to solve a diverse range of issues like load balancing, header rewrite, rate limiting, reverse proxy, etc.
Istio can also perform circuit breaking, a mechanism to handle failure by short-circuiting your calls to a different path. For example, you can send an already-decided status code to let the calling service know about it or fall back to another location, such as a cache for the information, and serve that data to deal with the faulty APIs. Once a recurring failure is detected, it prevents the application from calling the faulty service and handles it with different logic.
System Complexity
Istio is a lot more complex than Linkerd due to the fact that it tries to solve far more problems than Linkerd. Linkerd, on the other hand, has a very targeted set of problems it solves, making it less complex. This doesn’t mean Linkerd is not powerful, but simply that it has well-defined boundaries and implementation.
Istio tends to be a generalized solution for your service mesh, and it’s a fact that the more generalized you try to build a solution, to potentially handle any issue that arises, the more underlying complexity there will be.
When it comes to configuration, Linkerd is easy to install, with all the configurations a part of annotations in Kubernetes manifests. Istio, on the other hand, has a huge learning curve and takes time to understand and configure.
Ingress
An ingress handles how you can accept incoming traffic inside your Kubernetes clusters. Istio uses Envoy as an ingress, while you may need to deploy third-party controllers like Nginx in Linkerd to enable a few capabilities like rewriting the incoming header. In short, Istio has better support for its ingress, but it takes a lot of configuration.
Since Linkerd can be integrated with Nginx, there are capabilities you can enable at the Nginx ingress layer, like rewrite rules, etc.
Egress
Egress lets you control traffic going out of the cluster. Istio provides a way to control outgoing traffic with the help of virtual service objects and gateways. Egress implementation in Linkerd, however, is not straightforward and can only be implemented using DNS and DTAB.
Handling Traffic
Both proxies support timeouts, retries, rate limiting, mTLS, GRPC, and HTTP2. Timeouts let you configure when to time-out your request with the help of a proxy. Similarly, retries and rate-limiting are also configured at the proxy level, and the application is thus unaware of these when using a service mesh. mTLS makes sure all the communication between services is encrypted.
Distributed Tracing
Tracing a request end to end, meaning how it passes through each upstream microservice, is a great capability needed in today’s microservices-based architectures. Both Linkerd and Istio provide this with the help of Jaeger.
Monitoring
Both with Istio and Linkerd we can use Prometheus and Grafana to get the observability that we need.
with Linkerd, we will need to deploy viz — linkerd-viz is a monitoring application based on Prometheus and Grafana, autoconfigured to collect metrics from linkerd.
with Istio (since 1.8 ) we will have to deploy the Prometheus and Grafana separately, and configure Prometheus to scrape Istio components — https://istio.io/latest/docs/ops/integrations/prometheus/
Multi-Cluster Support
Multi-cluster support means a single service mesh can span across multiple Kubernetes clusters. This enables different applications running in different clusters to be able to talk to each other like they are part of a single service mesh. Both Linkerd and Istio support multi-cluster service mesh deployments.
Fault Injection and Traffic Splitting
Fault injection is used to create chaos in a system to see how it reacts. It lets you artificially increase the error rates in your system to see how it copes with said errors.
Traffic splitting maps canary and blue-green deployments, where you can divert a certain percentage of traffic to newer deployments and the remainder to older deployments.
Both of these features can be configured very easily in Linkerd and Istio; they give you better control over your deployments and make your system more reliable.
Aside from the above features, there are a couple of other factors to consider when choosing a service mesh for Kubernetes.
Performance and Resource Consumption
Linkerd runs a very lightweight proxy built in Rust. The Linkerd2-proxy only works with Linkerd but is known for running its set tasks very efficiently.
Istio runs Envoy, a multipurpose proxy written in C++ and used by a lot of different software as a data plane. Istio relies on Envoy because it is very feature-rich, but this is also why Istio can be so complex.
The choice of Rust for Linkerd’s proxy gives it a good advantage over others, as Rust has a very light memory footprint compared to Java. It is faster and safer for tail latencies since it is not a garbage-collected language like Golang (When garbage collection gets triggered in serving requests, it can induce tail latencies). Rust is also a memory-safe language compared to C or C++, making it less vulnerable to security threats due to memory-related buffer overflows, use-after-free, etc.
For these reasons, choosing Rust over other languages has proven beneficial, and you can see the performance differences in Linked versus Istio.
Linkerd is primarily more performant than Istio due to its targeted approach and lightweight proxy. Its data plane is 2–5 times faster, depending on the scenario.
As to resource consumption, Istio consumes around 2–3 times the memory and 8 times the CPU of Linkerd. You can read more about these benchmarking results here.
Community
When we talk about open-source projects, it is very important to know about the community that can back you up. Istio has a great community, and big companies like Google are very much involved in this project. But the fact that Istio is not part of the CNCF landscape — although Envoy is — raises some concerns.
Linkerd is a CNCF project that has a great community and has also gained a lot more traction recently compared to Istio.
Both communities are great, but being a member of CNCF gives Linkerd a bit of an advantage.
Conclusion
Linkerd and Istio are both very powerful service meshes, and it really depends on your requirements as to which one you should use. If you want something to solve your service mesh problems and want your data plane to be very fast, less resource-intensive, and less complex to manage, Linkerd should be your choice.
If you want a lot of features and have a great team to manage your service mesh and its components and complexity, you can go with Istio. With Istio, you can easily configure egress and rules on outgoing traffic, something that can also be a bit tricky with Linkerd. Of course, resource usage and latency are two additional factors you need to keep in mind when making this decision, and Linkerd is the preferred choice for both.
