Authenticating Knative Services with oauth2-proxy

In this post we will walk through how to use oauth2-proxy to authenticate access to a Knative Service (this assumes basic familiarity with knative/serving). We will use Github as our OAuth2 provider, but oauth2-proxy supports a decent variety of options.

However, before we jump in I want to start by thanking Savita for driving the multi-container work these last few milestones. I think this is one of the biggest functional changes to the Knative Serving API since its inception, and I have been very eager to take it for a spin. Thank you! 🙏

Prerequisites

To complete this exercise, you will need to have Knative Serving 0.16+ installed with the multi-container feature enabled, and TLS configured. The simplest way to do this is to install mink which is a lightweight distribution of Knative:

kubectl apply -f https://github.com/mattmoor/mink/releases/download/v0.16.0/release.yaml

mink has the multi-container and auto-TLS features enabled by default, but you must follow these instructions to configure a domain for auto-TLS to work.

With a vanilla installation of Serving you can enable auto-TLS by following this guide, and the multi-container feature with:

kubectl patch configmap/config-features \
  --namespace knative-serving \
  --type merge \
  --patch '{"data":{"multi-container": "enabled"}}'

We will show how to add this to an existing application (below), so for this tutorial let’s deploy a simple service that we will add authentication to:

kubectl apply -f - <<EOF
apiVersion: serving.knative.dev/v1
kind: Service
metadata:
  name: autoscale-go
spec:
  template:
    spec:
      containers:
      - image: gcr.io/knative-samples/autoscale-go:0.1
EOF

You can watch for the service to become ready and get an https URL with:

watch kubectl get ksvc autoscale-go

If you open the URL with ?sleep=100 in your browser you should see:

Slept for 100.13 milliseconds.

Setting up the Github Application

Create a new Github application under your GitHub account here (there is a “New GitHub App” button). The pertinent fields for this exercise are:

1. Pick a name for the app, which will be displayed during the oauth2 flow.
2. Specify the “Homepage URL” (required, but not interesting for this exercise).
3. Specify the “User authorization callback URL”, which will be the hostname of the Knative service we plan to register (e.g. https://autoscale-go.default.mink.dev)
4. Under “Webhook” uncheck “Active” (we won’t use this here)
5. Under “User permissions” select “Read-only” from the “Email addresses” drop-down (this is to identify users).
6. Under “Organization permissions” select “Read-only” from the “Members” drop-down (this is to authorize users against a github team).
7. For “Where can this GitHub App be installed?” select “any account”.

Now hit “Create GitHub App”! You should see a page like below after submitting.

The GitHub App details, after hitting create.

Make note of the “Client ID” and “Client secret” fields (redacted above), we will need them in the next section.

To support authorization, we must install the application into the Github org containing a team we will use for authorization. To do this, select “Install App” from the left-nav on the screen above. Then hit install next to the appropriate Github org:

The App has been installed into “knobots” GitHub org.

Setting up the oauth2-proxy configuration

Next we are going to create our oauth2-proxy configuration file:

provider = "github"
# Fill these in from GitHub App page above. 
client_id       = "REPLACE ME"
client_secret   = "REPLACE ME"
# Fill in the name of the org we installed our app into,
# and the name of the team that users must be a member of
# to access the application.
github_org      = "REPLACE ME"
github_team     = "REPLACE ME"
email_domains   = ["*"]
# Configure the session cookie used by the proxy.
# You can generate a cookie_secret by running:
# `python -c 'import os,base64; print base64.urlsafe_b64encode(os.urandom(32))'`
cookie_secret   = "REPLACE ME"
cookie_name     = "login"
cookie_expire   = "6h"
cookie_refresh  = "10m"
cookie_secure   = true
cookie_httponly = true
cookie_samesite = "lax"
skip_provider_button = false
# Listen on :8081 and proxy to :8080
http_address = "0.0.0.0:8081"
upstreams    = ["http://127.0.0.1:8080"]
# Send ID token to the upstream app, but not the other headers
# (we don't need them)
pass_authorization_header = true
pass_basic_auth           = false
pass_user_headers         = false
# Allow certain probe paths through without auth like this:
skip_auth_regex       = ["^/healthz$"]
exclude_logging_paths = "/healthz,/ping"

Once you have filled out all of the “REPLACE ME” values as described in comments, save the file as oauth2_proxy.cfg and run the following command to turn it into a secret.

kubectl create secret generic oauth2-proxy-config \
   --from-file=./oauth2_proxy.cfg

Setting up your Knative Service

Now that we have our oauth2-proxy configuration set up, add the following sidecar and volume mount to the Knative Service spec. If you are adding this to a pre-existing service, be sure to remove any containerPort, readinessProbe , and livenessProbe fields from the existing container.

kubectl apply -f - <<EOF
apiVersion: serving.knative.dev/v1
kind: Service
metadata:
  name: autoscale-go
spec:
  template:
    spec:
      containers:
      # The application container.
      - name: user-container
        image: gcr.io/knative-samples/autoscale-go:0.1

        # Specify PORT to direct the app to listen on 8080.
        # For a different port, be sure to change oauth2_proxy.cfg
        # where it referenced 8080.
        env:
        - name: "PORT"
          value: "8080"

      # Everything below here is oauth2-proxy configuration.
      - name: oauth2-proxy
        image: quay.io/oauth2-proxy/oauth2-proxy
        command: ["/bin/oauth2-proxy"]
        args: ["--config=/etc/proxy-config/oauth2_proxy.cfg"]
        volumeMounts:
        - name: proxy-config
          mountPath: /etc/proxy-config/
        # multi-container: enabled requires exactly one container
        # to specify a port, so that we know where to route traffic
        # We also currently limit where readiness and liveness 
        # probes may occur to this container.
        ports:
        - containerPort: 8081

      volumes:
      - name: proxy-config
        secret:
          secretName: oauth2-proxy-config
EOF

Deploy the update, and try the URL (with ?sleep=100 again) and you should see the following landing page:

The oauth2 login page for signing in with GitHub.

Clicking the above will send you through a 3LO flow, and if you are a member of the appropriate GitHub team you will be redirected back to your application and see the same result as above, but now it is authenticated!

Voila, you have an authenticated Knative Service! 🎉