Istio Request Control with Envoy Filters — Request Headers
In microservices each service has it’s own task. To execute each task, microservices receive requests and each request may contain headers, body, or query strings. While changing any part of a request that you receive, you have to think twice. Because you may cause an exception on other microservices. Let say all of your applications expect a header parameter called user-id . What happens if the request sender change the name of this header? All of your application gets errors. In a same manner, let assume that you get JWT from Authorization header. If JWT payload changes, again your applications that uses JWT payload gets errors.
Let assume that your end-user login from user service and gets a JWT. On each request this JWT setted on Authorization header. Other microservices resolve this JWT and obtain payload to make some calculations.
Above graph is a simple example. Imagine you have lots of micro services like book service and each of them uses different technologies like .Net Core, Go, Java etc. For each service you have to create a JWT parsing module or library. When your teams get bigger, this become confused. Because JWTs may change over time. A claim can be removed or modified, or a new claim can be added for a purpose. Any change on JWT, forces you to make a development. As a result of this a deployment pipeline should be runned.
Istio can help us at this point. Istio describe itself like,
“I make it easy to create a network of deployed services with load balancing, service-to-service authentication, monitoring, and more..”
Simply when you install Istio to your kubernetes cluster and make one of your namespaces as istio injected, all request to workloads on this namespace pass through Istio architecture.
On the above everything looks fine. You have one service and handle JWT on this service.
As I said before, when your teams get bigger and new services appear, your graph looks like that.
Parsing JWT on each service can be confused. When your user service changes JWT payload, you have to update your parsing code on each of your applications.
With Istio, JWT and other request headers can be controlled before the request hit to your services. There are a few ways to control your request headers on Istio. I will mention about them. By applying these solutions, you don’t need to parse JWT on your applications. Istio will parse JWT for all your applications. We will use Envoy Filters to do this. Following graph illustrate it. When a request hit to envoy, our EnvoyFilters will be applied, then the request will be distributed to workloads. We will remove parsing code from our applications.
In theory, we understand what we want to achieve. Lets make it real.
First I created a namespace called book
Then I need an application that runs on book namespace. I just create a simple .net core application. It simply gets user-id and is-authenticatedparameters from request header and returns the list of books.
Note that user-id and is-authenticated params used to be obtained from JWT payload. We will parse JWT before request received from the service and set them as header parameters. With this way, our applications will not be responsible for jwt parsing.
Let inject Istio to our namespace by labeling namespace. The following kubectl command labels book namesapce as istio injected.
kubectl label namespace book istio-injection=enabledNow, we need to create a gateway to receive requests from istio envoy proxy. Istio has a tutorial for that. You can read and learn more about here.
When you install Istio to your k8s cluster, it creates a namespace called istio-system. Istio creates a service called istio-ingressgateway . This is the default controller and entry point to our mesh. For now we are dealing with http request, we must find the http port of ingress gateway. You can find it by editing the service definition like following.
As you can see Istio has port definitions for each connection type. We will use http2 for http request. As seen on the screenshot our target port must be 31380. We need to orient all of our dns’ to 31380 port of our k8s cluster. Istio will handle all network traffic for us.
We need to define a gateway to handle requests for our book service. Below configuration says that all external request that comes from http protocol, will be redirected to service which has a selector like istio: ingressgateway. This is default controller of Istio. Note that hosts parameter defined as * . This means that all request with any hosts will be handled by the ingressgateway controller.
You can specify special domians like “*.myteam.mycompany.com” as
hostsproperty.
Now, we forwarded incoming requests to istio ingress gateway. To distribute each request to related services we need to define VirtualServices. Let create a virtual service for our BookService .
This configuration implies that, if our ingress gateway receive a request that has host bookservice.example.com then the request will be routed to book-service-svc in the book namespace.
Our k8s sevices may have multiple subsets. Subsets can be considered as deployments or deamons sets. On the virtual service config all request forwarded to v01 subset.
When we create a deployment we can add labels to this deployments to distinguish from themselves. In istio we can distributed requests to specific deployment. To do this we need to create DestinationRules. For our example, lets create a destination rule for v01.
This configuration implies that, in book namespace, there exists a book-service-svc and it has a selector label to which deployments will be used.
This destination rule points the deployment which has a version as label and v01 as value. If you use v01 as subset in the VirtualService configuration then, the requests will be forwarded to Deployment which has version: v01 label. To clarify look at the graph.
Book service select v01 as workload. It is defined in VirtualService configuration. To select other subset as workload you need to define destination rules for each version and add them as subsets. A virtual service can have multiple subsets.
Request Header Controlling
For now, our book service works but any header parameters hasn’t been passed.
When we request books, it return nothing for now.
We will control requests with Istio and pass required headers to our application with the keys that is understandable by the application.
First, lets assume that front-end application knows the current user’s id and authentication state. While requesting for books front-end add header fields like customer-id=somecustomerid and logged-in=true . These fields unfamiliar to our microservices (Look at our .net code. We expect different header names). To align with front-end application, we need to update our code. Instead of updating our applications we will create an EnvoyFilter. You can read more about Istio’s EnvoyFilter here.
This configuration implies that, when book-service-svc on book namespace will receive a request, the headers of request will be modified. The lua script in the yaml, maps current customer-id and logged-in fields to user-id and is-authenticated fields. EnvoyFilter modifies the request header before request reaches to our services. With this way our applications will not effect from header parameter changes. Now if we call book service again, we get the result.
Our application doesn’t know customer-id and logged-in fields. Istio EnvoyFilter will change customer-id and logged-in fields in a way that our application can understand.
Now, lets try something more complicated. This time we will try parsing JWT with EnvoyFilter. In microservices JWT is generally generated by the user service and passed to all other services with cookies or Authorization header. Sometimes you need to parse JWT to get some payload information to use on your applications like user name, id etc. When a claim’s name in JWT payload changes what happens? You need to update all of your applications. To avoid this situation, we may parse it with an EnvoyFilter and pass required payload fields to our applications.
This configuration may looks very complicated, but is a lua script that decodes JWT and then puts decoded claims into request as header parameters. This is useful when you need to know payload claims. I found this config from github. You can check the git repository. In the last part of lua script, I get the customer-id and logged-in fields from the decoded jwt and put them into request as header parameters.
Our services do not need to parse JWT. Istio envoy filter can decode it for all of our services and serve the desired claims as header parameters. We don’t need a JWT parser library. If any of the claim changes, I only need the update Istio Envoy Filter configuration.
In conclusion, Istio can help us controlling our request headers. Instead of controlling request headers on your applications, you can centralize this control mechanism with Istio. This makes your architecture more flexible. Your services are not affected by the changes made by the other services. You just update your envoy filter.
Notes
The first approach is a good solution for request header controlling for your services. JWT approach isn’t an advanced solution. It only finds with simple claims, nested object or arrays won’t work. You may need to write better script for this. The developer of this lua script mentions more about on github.
Envoy filters may cause some performance problems on your service mesh. Custom lua scripts should be tested before production.
