Istio with Authentik: securing your cluster and providing authentication and authorization
The past week I had fun with setting up Istio on my Kubernetes cluster, and adding an authentication/authorization solution to the ingress. Mission accomplished: now I can manage users and groups and whatnot at a fine-grained level on all ingress to any app in my cluster, while keeping Istio’s allow-nothing base rule in place. All traffic in the cluster (ingress and inter-service) is forbidden unless explicitly whitelisted. Authentication and authorization happen at the ingress, before it even gets to any service hiding deep inside the Kubernetes cluster.
I had some challenges to overcome, and I had a hard time collecting all the necessary information in one place, to get a full picture of what I had to do. Hence I am writing it down, and why not share it with the world.
First challenge: choosing the tools
For the service mesh, I looked at Istio, Linkerd, and Open Service Mesh (OSM). For user management and authentication/authorization services (because well, they are indeed two separate things), I looked at Keycloak and Authentik (user management and authentication/authorization in one), and at Authelia, Pomerium and Apache APISIX (each for authentication/authorization, but in need of a separate service for the user management).
I got things working with Istio and Authentik. You may get lucky with any other combination, I am just telling what worked for me and what other options I decided not to go for. I am not going to claim that the other options are bad or do not work, but I can tell you that they did not satisfy me.
Istio
Istio is a service mesh. It implements mTLS for you, and it takes very little effort to setup and enable. You quickly get running with a cluster where all traffic between nodes is nicely encrypted. Not bad!
It also comes with its own ingress controller. I do not know if there is any reason to choose Istio’s ingress over another solution, but I figured it would be a wise decision to mix as few packages as I can.
Istio adds a sidecar to every pod in your cluster, which acts as a gateway for all network communication for that pod. This allows istio to enable mutual TLS (mTLS) for all pods, and it also means that you have the possibility to write rules for what traffic is allowed or not: AuthorizationPolicies.
The default mTLS configuration works fine, so that took no effort, but the AuthorizationPolicies are unavoidably customized to your cluster, so that took some reading.
Authentik
Authentik is awesome. Because it just works, and for a project like mine, it covers all I need. Have a look at this comparison chart, which might convince you to pick Authentik over Keycloak. It lets you manage users and groups, as well as applications (clients). Once you have clients defined, each user gets a landing page with links to all clients they have access to. It integrates very well with Istio’s ingress controller (if you make no typos in the configuration).
Second challenge: understanding how these tools work
Istio AuthorizationPolicy
It took me a while to figure out how Istio’s AuthorizationPolicies work, and most importantly, what the rules are when multiple policies exist. It is actually very well explained in this diagram. It took me days to actually find it in Istio’s documentation. Let me show a copy here:
One misunderstanding I initially had, was that a CUSTOM policy would play the roles of both DENY and ALLOW. But, as you can see in the tree, it really is a DENY rule with a custom decision, where custom means that the decision is taken by another service. If your CUSTOM policy decides not to deny someone entry (that is, you allow entry), the request can still be denied by the DENY and ALLOW policies further down the tree! That is important. So:
💡 If your CUSTOM policy decides not to deny someone entry (that is, you allow entry), the request can still be denied by the DENY and ALLOW policies further down the tree!
Authentik configuration
This documentation page of Authentik got me started with the configuration of istio. But I did not really understand how to add a client (service) to Authentik until I found this blogpost, which states:
Most newcomers usually get lost juggling these three.
“These three” refers to Application, Provider and Outpost. You must configure all three in Authentik's UI (or with its API, but the point is that Authentik needs to be live), before you can secure your application. The Application is your service that you want to secure, you give it a name and an icon and so one. The Provider tells Authentik how the public and private URLs of your application are connected, and finally the Outpost is the actual service which Authentik exposes to your service which needs to know if someone is authorized or not. I suggest you read that blogpost I referred to above.
The answer
Here’s the configuration that works for me.
Istio
I use the Helm installation of Istio, and this is my values.yaml for istiod:
global:
logging:
level: "default:debug"
proxy:
logLevel: debugmeshConfig:
defaultConfig:
discoveryAddress: istiod.istio-system.svc:15012
tracing:
zipkin:
address: zipkin.istio-system:9411
enablePrometheusMerge: true
rootNamespace: null
trustDomain: cluster.local
extensionProviders:
- envoyExtAuthzHttp:
headersToDownstreamOnAllow:
- cookie
headersToUpstreamOnAllow:
- set-cookie
- x-authentik-*
includeRequestHeadersInCheck:
- cookie
pathPrefix: /outpost.goauthentik.io/auth/envoy
port: 80
service: authentik.authentik.svc.cluster.local
name: authentik
Note that there is one small but crucial difference with the documentation page of Authentik: the port to which Istio’s ingress queries for the custom decision, must be the port which is exposed in your cluster. As I am using Kubernetes, this corresponds to the targetPort of the authentik service. This is opposed to the port 9000 in the documentation, which is internally the port which authentik listens on. Check which port you need for your use case.
Note too, that I had made a copy/paste typo, which took me a day to uncover. My mistake was in the includeRequestHeadersInCheck field: all request were going very well and were directed to authentik, I could login, but then.. I got a redirection loop. Because of the wrong value in the header name, my authentication header was not correctly communicated to Authentik.
The next challenge, was to get all the firewall rules (AuthorationPolicy) to work correctly.
AuthorizationPolicy
# FILE: allow-nothing.yaml
# this disallows everything:
# https://istio.io/latest/docs/concepts/security/#allow-nothing-deny-all-and-allow-all-policy
apiVersion: security.istio.io/v1beta1
kind: AuthorizationPolicy
metadata:
name: allow-nothing
namespace: istio-system
spec:
action: ALLOW
---
# FILE: peer-authentication.yaml
apiVersion: security.istio.io/v1beta1
kind: PeerAuthentication
metadata:
name: "default"
namespace: istio-system
spec:
mtls:
mode: STRICT
---
# FILE: authentik-ingress-istio.yaml
apiVersion: security.istio.io/v1beta1
kind: AuthorizationPolicy
metadata:
name: authentik-ingress-istio
namespace: istio-ingress
spec:
action: ALLOW
rules:
- to:
- operation:
hosts:
- auth.localhost
- auth.localhost:443
- auth.localhost:80
selector:
matchLabels:
app: istio-ingressgateway
---
# FILE: authentik-ingress-allow.yaml
apiVersion: security.istio.io/v1beta1
kind: AuthorizationPolicy
metadata:
name: authentik-ingress-allow
namespace: authentik
spec:
action: ALLOW
rules:
- from:
- source:
principals:
- cluster.local/ns/istio-ingress/sa/istio-ingressgateway
to:
- operation:
hosts:
- auth.localhost
- auth.localhost:*where auth.localhost is the public fully-qualified domain name (FQDN) to reach my instance of Authentik.
# FILE: authentik-allow-databases.yaml
apiVersion: security.istio.io/v1beta1
kind: AuthorizationPolicy
metadata:
name: authentik-allow-databases
namespace: authentik
spec:
action: ALLOW
rules:
- from:
- source:
principals:
- cluster.local/ns/authentik/sa/authentik
to:
- operation:
ports:
- "5432"
- "6379"
- "9100"
- "80"
- from:
- source:
principals:
- '*'
---
# FILE: httpbin-ingess-istio.yaml
apiVersion: security.istio.io/v1beta1
kind: AuthorizationPolicy
metadata:
name: httpbin-ingress-istio
namespace: istio-ingress
spec:
action: ALLOW
rules:
- to:
- operation:
hosts:
- httpbin.localhost
- httpbin.localhost:443
- httpbin.localhost:80
selector:
matchLabels:
app: istio-ingressgateway
---
# FILE: httpbin-ingress-custom.yaml
apiVersion: security.istio.io/v1beta1
kind: AuthorizationPolicy
metadata:
name: httpbin-ingress-custom
namespace: httpbin
spec:
action: CUSTOM
provider:
name: authentik
rules:
- to:
- operation:
hosts:
- httpbin.localhost
- httpbin.localhost:*where httpbin.localhost is the FQDN at which httpbin can be reached.
Gateway
# FILE: authentik-ingress.yaml
apiVersion: networking.istio.io/v1beta1
kind: Gateway
metadata:
name: authentik-ingress
namespace: istio-ingress
spec:
selector:
istio: ingressgateway
servers:
- hosts:
- auth.localhost
port:
name: http
number: 80
protocol: HTTP
tls:
httpsRedirect: true
- hosts:
- auth.localhost
port:
name: https
number: 443
protocol: HTTPS
tls:
credentialName: authentik-ssl-certificate-secret
mode: SIMPLE
---
apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
name: authentik-ingress
namespace: istio-ingress
spec:
gateways:
- authentik-ingress
hosts:
- auth.localhost
http:
- match:
- uri:
regex: ^\/[^\.]+.*
route:
- destination:
host: authentik.authentik.svc.cluster.local
port:
number: 80
- match:
- uri:
exact: /
redirect:
uri: /if/user
---
# FILE: httpbin-ingress.yaml
apiVersion: networking.istio.io/v1beta1
kind: Gateway
metadata:
name: httpbin-ingress
namespace: istio-ingress
spec:
selector:
istio: ingressgateway
servers:
- hosts:
- httpbin.localhost
port:
name: http
number: 80
protocol: HTTP
tls:
httpsRedirect: true
- hosts:
- httpbin.localhost
port:
name: https
number: 443
protocol: HTTPS
tls:
credentialName: httpbin-ssl-certificate-secret
mode: SIMPLE
---
apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
name: httpbin-ingress
namespace: istio-ingress
spec:
gateways:
- httpbin-ingress
hosts:
- httpbin.localhost
http:
- match:
- uri:
prefix: /outpost.goauthentik.io
route:
- destination:
host: authentik.authentik.svc.cluster.local
port:
number: 80
- match:
- uri:
regex: ^\/[^\.]+.*
- uri:
exact: /
route:
- destination:
host: httpbin.httpbin.svc.cluster.local
port:
number: 14001As you can tell from these two ingress configurations, I am using cert-manager with HTTP01 challenges: I do not match any path starting with a dot in both VirtualServices, but I do match anything else including the exact / path.
Configuring a client for httpbin in Authentik
Before configuring Authentik, if you go to the url of your service, you should get this result if everything was setup correctly:
{
"Message": "no app for hostname",
"Host": "httpbin.localhost",
"Detail": "Check the outpost settings and make sure 'httpbin.localhost' is included."
}I still need to build a solution where this is scripted, but for now I do it in the UI of Authentik. This mostly follows the aforementioned blogpost. In the admin interface, create a new application. There is not much you can do wrong here. While creating the application, you can create (and then select!) a Provider. For our setup with Istio ingress, it is paramount that you create a Proxy Provider ! Nothing else. And in configuring the proxy provider, it is paramount that you select Forward auth (single application) for our setup, nothing else. At the time of writing, your correct provider configuration looks like this:
Finally, as per the manual, we go to Outposts in the admin interface, and edit the authentik Embedded Outpost by selecting the application we just created, and save the Outpost. If your application is not there, maybe you forget to connect the Application to its Provider.
Now, if you navigate to your service, you should see it just fine. If you open a private window and go to the service, you should be redirected to Authentik’s login panel, after which you should be redirected back to the service ready for you to use.
That’s it.
Now, if you navigate to your service, you should see it just fine. If you open a private window and go to the service, you should be redirected to Authentik’s login panel, after which you should be redirected back to the service ready for you to use.
Please leave a comment if you think this setup can be improved, or if something is not clear yet!
