Envoy Proxy: What, Why & Its Potential Future
A breakdown of the concepts and uses, with a hint of the future.
Envoy is one of the most popular proxies out there. What does it do? How do you use it? And what has it got to do with service meshes?
Let’s break it down from the basics! Starting with a definition and looking at the concepts and features. Then ending with its future based on the projects it has spawned.
What is Envoy Proxy?
From their website —
Envoy is an open source edge and service proxy, designed for cloud-native applications
And then a more detailed description —
Envoy is a … proxy designed for single services and applications, as well as a communication bus and “universal data plane” designed for large microservice “service mesh” architectures … Envoy runs alongside every application and abstracts the network by providing common features in a platform-agnostic manner. When all service traffic in an infrastructure flows via an Envoy mesh, it becomes easy to visualise problem areas via consistent observability, tune overall performance, and add substrate features in a single place.
But what does this jargon-laden definition exactly mean? Specifically, the text I have highlighted in bold? To understand, we start with why Envoy exists in the first place!
Why was Envoy created?
Matt Klein created Envoy at Lyft to solve the problems they were facing with their service-oriented architecture (SOA). In his 2017 talk, Matt talks about the motivation behind Envoy.
Lyft had a monolithic architecture and then transitioned to SOA, with its associated issues. The main one? Observability. When the system went down, it was tough to diagnose where the problem was. Was it in the application, the network, the hardware load balancers, or the cloud load balancers?
In the talk, he puts up the mission statement of Envoy as:
The network should be transparent to the applications. When network and application problems do occur, it should be easy to determine the source of the problem.
Envoy was created to solve the observability problem at Lyft. But it also offloaded the common network-related features from the service to the proxy. So, developers no longer needed to implement these in the service.
What does Envoy do exactly?
The Envoy proxy is supposed to be co-located with every service. All the requests and responses to and from the service are routed via the proxy. This is commonly called a sidecar proxy pattern. Now the proxy can capture all the traffic from all the services in the deployment. This makes it easy to capture observability stats helping to easily visualise problem areas.
Also, since the service-to-service communication now happens via the sidecar proxy, it is called a service mesh architecture.
Even though this was the initial design, Envoy can be deployed as a reverse proxy on the network edge (ingress/egress) too. Basically, it will intercept external calls to the services and route them to the appropriate service. Hence becoming a gateway through which traffic flows to and from the entire network, not just individual services. In effect, replacing something like NGINX.
This is why the website says:
Envoy is an open source edge and service proxy, designed for cloud-native applications.
It is designed for cloud-native applications because they are typically broken down into several services (i.e., microservices).
Envoy Features
Envoy runs alongside every application and abstracts the network by providing common features in a platform-agnostic manner.
The common features provided include advanced load balancing, HTTP/2 support, translating between HTTP/1.1 and HTTP/2 (and now HTTP/3), service discovery, health checking, observability and security. By offloading these responsibilities to the proxy, the services don’t need to implement them. It makes the developer’s life easier!
With that, the detailed description becomes much clearer. But what is this mention of a “universal data plane”? For that, we need to understand how Envoy proxy is configured.
Envoy Proxy Configuration
Envoy can be configured in 2 ways —
- Static configuration
- Dynamic configuration
In static configuration, all the config is specified in YAML files. While in dynamic configuration, a minimal YAML config file is used initially, but the rest of the configs are discovered using the management server.
The management server exposes a set of services for specific configs. For example, if you want to discover clusters, you hit the Cluster Discovery Service (CDS), which returns the clusters (backend servers, usually). Instead of directly specifying the clusters in the YAML file, the management server IP and port are specified. Likewise, for routes, the Route Discovery Service (RDS) is used. There are many such services — endpoints (EDS), virtual hosts (VHDS), listeners (LDS), secrets (SDS) and so on. In short, these are referred to as xDS (x Discovery Service). The management server is hence called the xDS management server.
Universal Data Plane
The Envoy proxy processes the actual traffic while placed alongside the services. This abstraction is called the data plane, as the traffic (data) flows through this.
The management server meanwhile resides in the control plane. The control plane is an abstraction that specifies how the data plane should process the traffic. It does so by setting the routing, filtering, load balancing policies etc. And all of these can be configured via the xDS APIs.
Envoy proxy is being positioned as a universal data plane as it can be dynamically configured from any control plane that implements the xDS APIs. The Go and Java Control Planes are reference implementations provided by Envoy. There are others by vendors like Gloo, Istio, and WSO2.
The Future of Envoy Proxy
Since Envoy is a purely community-driven project, there are no roadmaps to pinpoint the exact features in the next few releases. Although there is active development on Envoy itself, the future may lie in the abstractions built on top of it. A couple of active projects that sprang out of Envoy hint at how its uses are expanding.
Envoy Mobile
Envoy is obviously deployed on the server-side to handle the traffic flowing through to the backend services. But to realise the statement of making the “network transparent”, this project aims to push Envoy all the way to the mobile applications side as well. This will give full visibility across the whole network. Matt Klein says:
Envoy Mobile in conjunction with Envoy in the data centre will provide the ability to reason about the entire distributed system network, not just the server-side portion
Note that in this case, Envoy is distributed as a networking library that is embedded into mobile applications due to how apps are written on Android and iOS. Whereas on the server side, the proxy is a separate process deployed alongside the service.
Lyft routes nearly all its mobile app traffic through Envoy mobile. However, references for adoption in other places are hard to find. This is still early days though, as the project started in 2019.
Envoy Gateway
The project aims to implement a wrapper on top of Envoy proxy so it can be used as an API gateway. This will function as a control plane for the universal data plane, which is the Envoy proxy.
But what about the Go control plane already implemented as a reference? What’s the difference between the two?
First, Envoy gateway will provide standard API gateway functionalities, such as rate limiting, authentication and authorization. More importantly, it will not implement the xDS APIs directly. Instead, it will implement the Kubernetes Gateway API specification. This specification is seeing wide adoption, and the aim of the Envoy gateway project right now is to enable running Envoy in Kubernetes as an ingress controller.
Internally, it translates the Kubernetes Gateway API calls to be run on the xDS server. It also provides instructions to create the deployment on the Kubernetes infrastructure. However, it will also support using xDS for advanced use cases that cannot be achieved via the Gateway API.
TL;DR — Envoy was created at Lyft to make the network transparent to easily solve issues in a service-oriented architecture. It provides common network features, so developers don’t need to implement them in their services. It can be a sidecar and edge proxy that can be either configured statically or dynamically using a control plane via xDS APIs. The future of Envoy may lie in the abstractions built on top of it.
Useful References
- The very detailed Envoy docs. Life of a request explains core concepts really well.
- Matt Klein’s blogs on universal data plane API, Envoy mobile, and Envoy gateway
