How to protect a Kubernetes Ingress behind Okta, with Nginx

In this blog post, we’ll see how easy it is to “protect” a web app behind Okta, using Nginx as a reverse-proxy in front of it. In a Kubernetes environment.

Vincent Behar
Mar 3 · 5 min read

Our use-case

We’re using both Kubernetes to deploy our applications and Okta as a company SSO. We’re also big fans of Jenkins X. Jenkins X comes with a few UI, which unfortunately don’t have native authentication/authorization implementation yet:

Jenkins X relies on Nginx for its ingress controller, and it uses the basic auth feature to protect its UI by default. The issue with this solution is that you either need to manually manage all your users (and passwords), or give them a shared set of credentials.

As we’re already using Okta at work, we wanted to integrate it into our Jenkins X setup.

Nginx, Vouch, and Okta

We found a blog post written by the Okta team on how to Use Nginx to Add Authentication to Any Application, which is already quite good. I won’t repeat everything they wrote, but instead, I’ll focus on the steps you’ll have to do if you’re working in a Kubernetes environment.

There are 3 parts in this setup:

  • create new applications in your Okta account
  • install and configure Vouch Proxy
  • configure the ingresses you want to protect

Okta Application(s)

First, you’ll need to create a new OpenID (Web) Application in Okta, with the “login redirect URL” set to something such as https://vouch.jx.example.com/auth — which is the Vouch Proxy Auth endpoint. Make sure to set the “grant type allowed” to Authorization Code. And keep the application’s client_id and client_secret, we’ll need them later.

All the web-apps you want to protect will redirect the users to the Vouch Proxy Login endpoint, which will then redirect to Okta using the OpenID Application you just created. So you can have as many web-apps as you want behind this single Okta Application. But the issue is that your users won’t see their web-apps in their Okta Dashboard.

The “solution” we found is to create a fake SWA Application for each web-app, with empty credentials. The SWA Application is only here to get a link to the real web-app from the Okta Dashboard.

Vouch Proxy

The vouch-proxy’s README is well written, and it contains a small section about how to run it in a Kubernetes cluster — with a link to a Helm chart.

We’re big fans of Helmfile — a tool used to manage your Helm releases in Gitops style. Installing Vouch in a Helmfile-managed cluster is as simple as adding to your helmfile.yaml:

repositories:
- name: vouch
url: https://halkeye.github.io/helm-charts
releases:
- name: vouch
chart: vouch/vouch
version: "0.1.4"
namespace: jx
values:
- vouch/values.yaml
secrets:
- vouch/secrets.yaml

and then you can customize Vouch’s configuration in a vouch/values.yaml file:

replicaCount: 1ingress:
enabled: true
annotations:
kubernetes.io/ingress.class: nginx
hosts:
- vouch.jx.example.com
paths: ["/"]
tls:
- secretName: tls-jx-example.com-p
hosts:
- vouch.jx.example.com
config:
vouch:
allowAllUsers: true
cookie:
name: jxVouchCookie
domain: jx.example.com
secure: true
httpOnly: true
maxAge: 14400
jwt:
issuer: Vouch
maxAge: 240
compress: true
session:
name: jxVouchSession
oauth:
provider: oidc
auth_url: https://yourdomain.okta.com/oauth2/v1/authorize
token_url: https://yourdomain.okta.com/oauth2/v1/token
user_info_url: https://yourdomain.okta.com/oauth2/v1/userinfo
scopes:
- openid
- email
- profile
callback_url: https://vouch.jx.example.com/auth

Note that Okta’s blog post uses URLs such as https://yourdomain.okta.com/oauth2/default/v1/... — but we found that it was not working in our case, because we don’t use Okta’s API Access Management product. The fix is to use https://yourdomain.okta.com/oauth2/v1/... URLs instead, as documented in Okta’s Help Center — or in Okta’s OpenID Connect Reference Documentation about URLs.

The client_id / client_secret of your Okta application are stored in the vouch/secrets.yaml file, which is a sops-encrypted file. Helmfile has native support for the helm-secrets plugin, which uses sops to encrypt/decrypt YAML files. As our Kubernetes clusters are hosted on GCP, we’ve configured sops to use GCP KMS API to get encryption/decryption keys. This way we can control who has permission to encrypt/decrypt our secrets, and we’re writing them in our git repositories, along with the other settings — Gitops style.

In our case, we configured Vouch to run at vouch.jx.example.com and to store the cookie on the jx.example.com domain. Which means that this instance will be able to protect ingresses exposed under the same domain, such as jxui.jx.example.com or deck.jx.example.com.

Ingress configuration

Protecting an existing web-app that is already exposed through an ingress requires just a few annotations — as long as this ingress is exposed using the ingress-nginx controller. Here is an example:

kind: Ingress
apiVersion: extensions/v1beta1
metadata:
annotations:
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/auth-response-headers: X-Vouch-User
nginx.ingress.kubernetes.io/auth-signin: https://vouch.jx.example.com/login?url=$scheme://$http_host$request_uri&vouch-failcount=$auth_resp_failcount&X-Vouch-Token=$auth_resp_jwt&error=$auth_resp_err
nginx.ingress.kubernetes.io/auth-snippet: |
auth_request_set $auth_resp_jwt $upstream_http_x_vouch_jwt;
auth_request_set $auth_resp_err $upstream_http_x_vouch_err;
auth_request_set $auth_resp_failcount $upstream_http_x_vouch_failcount;
nginx.ingress.kubernetes.io/auth-url: https://vouch.jx.example.com/validate

You can see all the supported annotations in the documentation. The Nginx Ingress Controller will do the hard work on generating the right Nginx configuration to delegate the authentication to Vouch, which itself will talk to Okta.

Expose Controller

Jenkins X doesn’t provide Ingresses directly, instead, it relies on a tool named Expose Controller to automatically create ingresses from services, based on annotations on the services.

In this case, you’ll need to wrap the Nginx-related annotations in the fabric8.io/ingress.annotations annotation:

apiVersion: v1
kind: Service
metadata:
annotations:
fabric8.io/expose: "true"
fabric8.io/ingress.annotations: |-
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/auth-signin: "https://vouch.jx.example.com/login?url=$scheme://$http_host$request_uri&vouch-failcount=$auth_resp_failcount&X-Vouch-Token=$auth_resp_jwt&error=$auth_resp_err"
nginx.ingress.kubernetes.io/auth-url: https://vouch.jx.example.com/validate
nginx.ingress.kubernetes.io/auth-response-headers: X-Vouch-User
nginx.ingress.kubernetes.io/auth-snippet: auth_request_set $auth_resp_jwt $upstream_http_x_vouch_jwt; auth_request_set $auth_resp_err $upstream_http_x_vouch_err; auth_request_set $auth_resp_failcount $upstream_http_x_vouch_failcount;

An alternative is to configure Nginx’s Global External Authentication and then disable it for the ingresses you don’t want to protect.


When should you use it?

  • when you’re already using Nginx as a reverse-proxy / ingress controller
  • when you’re already using Okta or another OAuth2 / OpenID Connect provider — supported by Vouch Proxy
  • when the web app you want to protect has no native authentication/authorization support
  • when you don’t care about who is connected — unless the web app supports reading that information from an HTTP Header
  • when you don’t care about logout — because most web app without native authentication/authorization support have no “logout” button

In our case — quickly protect the Jenkins X UIs — it’s a good solution. Mainly because these apps are read-only, and once we know people have been authorized to access the app, we don’t really care about who they are or what they do. Of course with a read-write application, it would be different. This is why our long-term solution is still to wait for Jenkins X to natively implement a real solution ;-)

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch

Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore

Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store