Integrating Keycloak IAM with APISIX to secure your Kubernetes Services

In my previous article, I showed you how to install Keycloak within a Kubernetes cluster that has an APISIX API gateway. In this follow up article we will use Keycloak and APISIX to secure our test service.

In the first part of this article, we installed Keycloak into our Kubernetes cluster and configured it so that it was accessible from the Internet. Now we need to put it to use.

Keycloak solution

Enforcing Authentication

In this article we will use the test service we used in a previous article as our microservice. Currently access to that service should be available, via the APISIX API gateway at apisix.<CUSTOM DOMAIN>/worlds as this is how we have configured our NGINX ingress gateway and APISIX.

What we will do in this article is secure this service so that you must log in and authenticate yourself before you can access it.

This enforcement is done within the APISIX data layer and requires us to configure APISIX to check for an authenticated user.

Flow

This will be the flow:

1. User attempts to access the service without a valid token and receives a 302 redirection to log in
2. User goes to Keycloak and logs in and receives a token in the form of an encrypted cookie
3. User now accesses the service again, this time with the token
4. APISIX verifies the token is valid
5. APISIX provides access to the service

Returning a 401 from an API Gateway

One thing I realised quite early on in my development of APIs against an API Gateway was the reliance of the 302 HTTP redirect status. Many API Gateways were designed to redirect the user to the login page by returning a 302 if they had not been logged in.

This highlights a widespread assumption that the user is accessing the API using a browser and that the browser is able to understand the 302 and redirect to the login.

But what if our client is not a browser but a system? What if the designer of the user interface wants to do something before the redirect happens?

Whilst the 302 response is easy for a browser to understand, it can be inappropriate for API use. In these cases, if there is no authentication, a 401 needs to be returned to show the request has no authentication attached to it and 403 if the user is authenticated but not authorised.

It is great to see that APISIX actually has a configuration option to switch between the two. For this article we will use the 302 response as we are using a browser but you can toggle the flag to get a 401 response instead.

Am I authenticated?

To protect the service we need to check if the user is logged in or not. We will do this using the Open ID Connect (OIDC) protocol. You can read about OIDC in another of my articles.

Using this protocol (specifically the Implicit Fow), we will only allow users to access the service who have a valid JSON Web Token (JWT — pronounced ‘jot’). We will be using Keycloak to create a valid JWT that APISIX will validate.

To do this we will need to set up Keycloak with our user information but more on that later.

This is a very simple use case but provides the foundation on which to build Single Sign On and Role Based Access Control (RBAC).

APISIX to check authentication

We will first configure APISIX to check to see if the user is authenticated and redirect to a login screen when the user is not.

APISIX supports the concept of plugins that can process all relevant requests that come in. In this case we will use the openid-connect plugin.

This plugin requires four pieces of information that we do not yet have:

1. A realm in which this service exists
2. A client ID registered with Keycloak
3. A secret key associated to that client ID
4. A URL at which Kong can load the configuration it needs from Keycloak

Configuring Keycloak

APISIX needs secure access to Keycloak as a Keycloak client. This is done by creating a client configuration within Keycloak.

Using the Keycloak Admin UI using the admin user name and password from the first part of this article, log in to the admin console.

Multi-tenancy

Multi-tenancy is where a single instance (or cluster) of an application supports multiple customers. Each customer is called a tenant.

In a multi-tenanted application, each tenant believes they are the only customer of the application. They can neither see nor access the other tenants, their accounts or their data.

Multi-tenancy application

In some cases, the tenant can create multiple users in their tenancy, like this:

Multiple users in a tenancy

There are more complicated structures whereby an existing user may be invited into another tenancy and may, therefore, exist in more than one tenancy. We will not be looking at this case in this article.

Creating a Realm

Keycloak supports multi-tenancy. This means that a single instance of Keycloak can support multiple tenants and, like shown above, each tenant can have multiple users.

Keycloak uses the term Realm when referring to a tenancy.

Before we can start configuring anything, we need to create a Realm for our service.

Log in to the admin UI and click on the Master realm drop down and then click on Create realm.

Create realm

In the screen that appears, you only need give the Realm a name. Lets call this Our World. Ensure the Realm is enabled and then click Create.

You should now be greeted with:

Keycloak welcome page to new Realm

Now we have access to our new Realm, it is time to start configuring it. We will not be using any of the OAuth specific features for now and will be focussing on OIDC.

To those who are observant, yes, OIDC is built on top of OAuth but Keycloak configures authentication only application as OIDC applications and those requiring authorisation as OAuth applications.

Creating a Client

Now we have a Realm, we need to create a Client in this Realm. A Keycloak Client is an application or service that will be using Keycloak as its OIDC Provider (or IdP). In our case, both APISIX and our microservice are potential clients of Keycloak but in this solution we will only be connecting APISIX to Keycloak and so only require a single client.

On the main menu, ensure Our World Realm is selected and then click on Clients.

For brevity, I am not going to show you all the screenshots of how to do this but will talk you through the process. This will also allow me to help you understand what we are doing rather than focussing on the how we are doing it.

You will see that Keycloak automatically registers a set of clients for its own, internal purposes. We can ignore these.

We will create our own Client by clicking on Create client. Now enter this information:

• Client type — OpenID Connect … this allows APISIX to use Keycloak as part of the user authentication (OIDC) flow based on OAuth.
• Client ID — apisix … this tells Keycloak how to identify this Client. It can be anything. In this case we are simply naming it the same as our APISIX API Gateway
• Name — apisix … a nice, human readable name for our client.

The description is optional.

Click Next.

There are now three options:

• Client authentication — On … by switching this on we are saying our Client (APISIX) is able to securely hold its authentication secret.
• Authorization — Off .. this means we will not be authorising the Client using fine-grained access control (ie: we will only be using OIDC features).
• Authentication flow — Standard flow and Implicit flow only … we will be using the Implicit flow.

Click Next.

On this page we configure the constraints about how URLs are used:

• Root URL — https://apisix.<CUSTOM DOMAIN> … it is useful to set this as it means you do not have to keep typing out the root URL and can leave everything else as relative paths.
• Home URL — /worlds … this is the ‘when in doubt, go back home’ URL. It is used whenever the OAuth flow fails, the user ends up somewhere they shouldn’t be or after logging out.
• Valid redirect URIs — /* … this list tells Keycloak where it can redirect to when asked by the application. It is designed to prevent malicious actors from inserting their own redirection without you knowing. If Keycloak gets a redirect to somewhere else, it will reject the request.
• Valid post logout redirect URIs — /* … like the redirect URIs, this list tells Keycloak where it can redirect to after the user has logged out.
• Web origins — + … this tells the Cross Origin Resource Sharing (CORS) where clients may come from. By entering + it means from any redirect URI location.

Click Save.

Once this is saved, you are now given far more configuration options!

As this is not a Keycloak tutorial, we will do just enough to get us going.

Click on the Credentials tab. Find the Client Secret and then click on the eye to reveal the secret. You will need this secret later.

Well-known API

There is one more piece of information we need from Keycloak, the well-know API URL.

The are a number of bits of information a client needs from the IdP, including API endpoints, certificates used to sign JWTs etc. These are made available through a standard endpoint known as the .well-known endpoint.

To find the address of this endpoint, go to the Realm settings option on the main menu. On the General tab, scroll to the bottom to find the Endpoints section.

Click on OpenID Endpoint Configuration. The URL of the page that appears is the URL you need.

You can look through the page to see the type of information available to the Client.

So now you should have:

• The client ID (apisix)
• The client secret
• The .well-known URL

Configure APISIX

With the required information, we can now install the OIDC plugin for APISIX. We do this on each route we want to protect. If you followed my APISIX installation article, you will have the following file, which we will extend with the lines marked in bold (replace the < > fields with your own values):

apisix-hw-route.yml

apiVersion: apisix.apache.org/v2
kind: ApisixRoute
metadata:
  name: hellow-world-route
  namespace: default
spec:
  http:
  - name: hello-world
    match:
      hosts:
      - apisix.requillion-solutions.cloud
      paths:
      - /*     
    backends:
    - serviceName: hello-world-1-svc
      servicePort: 80
      weight: 70
    - serviceName: hello-world-2-svc
      servicePort: 80
      weight: 30
    plugins:
    - name: proxy-rewrite
      enable: true
      config:
        uri: "/"
    - name: openid-connect
      enable: true
      config:
        client_id: apisix
        client_secret: <CLIENT SECRET>
        discovery: http://my-kc-service.kc:8080/realms/Our%20World/.well-known/openid-configuration
        scope: openid profile
        bearer_only: false
        realm: Our World
        introspection_endpoint_auth_method: client_secret_post
        redirect_uri: /redirect_url 

You can see that we have only added the openid-connect plugin to the bottom of the file. We then enable this and configure it as follows:

• client_id … this is set to the client ID we used in setting up the Keycloak Client, apisix
• client_secret … this is the client secret Keycloak generated for us
• discovery … this is the well-known endpoint that we found earler but you will notice we are using the internal DNS name for the host rather than going externally
• scope … this is defining the scopes we are asking for, in this case we must include openid to use OIDC and have added profile to get more information about the user (ie: the user’s name)
• bearer_only … this is the flag that, when false, returns a 302 redirect to login (true returns a 401) when the user is unauthenticated
• realm … the Realm we are using (Our World)
• introspection_endpoint_auth_method … defines how APISIX will retrieve the access token from Keycloak (using a POST of the client secret defined earlier)
• redirect_url … the place the user will be returned to after being authenticated (ie: after logging in) — must be allowed by the Keycloak Client configuration

When you hit problems with your OIDC or OAuth configuration, it is most likely the definition of the redirect_url. In our case we redirect to https://apisix.<CUSTOM DOMAIN>/redirect_url, which, because of the way we have defined our route (/*), will simply serve our hello word test service. As we are now authenticated, it will be served as expected.

We now apply the changes to the existing route (or create a new one if it is new) with:

kubectl apply -f apisix-hw-route.yml

A final configuration

There is one more configuration change that has to be made. When the encrypted token is returned, the NGINX proxy module runs out of space to hold the data before passing it on. This can result in a 500 status being returned by the NGINX gateway.

To prevent this we need to modify the proxy settings. Log in to the gw server as root and then modify the following file by adding the lines in bold:

/etc/nginx/proxy_params

proxy_set_header Host $http_host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_busy_buffers_size   32k;
proxy_buffers   4 32k;
proxy_buffer_size  16k;

Test the configuration and reload the new settings with:

nginx -t
systemctl restart nginx

Testing the configuration

Now we have configured Keycloak and APISIX, we can now test the set up.

Enter the following in the browser:

apisix.<CUSTOM DOMAIN>/worlds

Instead of seeing the Hello World 1!! response, you will now be direct to the Keycloak login screen. You cannot get passed this point as you do not have a log in. The only login you have is for the Admin user but they belong to the Master Realm and cannot log in to the Our World Realm.

To fix this, log in to the Keycloak Admin UI and go to the Our World Realm. Select Users from the main menu on the left.

• Click Add User
• Add a username (eg: Bob)
• Click Email verified so it is on (ie: no need to verify it as we have not set up outgoing email)
• Click Create

At this point you will see more options. Select the Credentials tab.

• Click Set password
• Enter a password and confirm it
• Unclick Temporary ( unless you want the user to be forced to change it after logging in)
• Click Save

Now, refresh your https://apisix.<CUSTOM DOMAIN>/worlds page and log in using the user you just created.

After logging in you will now see Hello World 1!! (if you are using my test service). If you refresh, you will not be asked to log in. If you refresh enough times you will see the Hello World 2!! response 3 times out of 10.

Congratulations, you have secured access to your Service using APISIX and Keycloak. If you protect any other Services, you will now realise you have Single Sign On (SSO) across all of them, if they are in the same Realm.

So what is happening

With this solution we are using the Implicit Flow. This flow is used when the application is incapable of storing a secret. Browsers are such an application, which is why we are using the Implicit Flow.

The flow is as follows:

Request: /worlds
Response: redirect (302) to /auth ? scope=xxx & state=xxx & client_id=xxx & response_type=xxx & redirect_uri

Request: .../auth ? (params)
Response: login page

Request: .../authenticate ? (params)
Response: redirect to (redirect_url) ? (params)

Request: (redirect_url) ? (params)
Response: redirect to /worlds setting session cookie

Request: /worlds with session cookie
Response: Hellow world response

The first request results in a redirect as there is no cookie.

The redirect to Keycloak /auth contains parameters that define what is required to authenticate to access the service, including the redirect URL to follow after authentication.

The /auth endpoint returns a login page at /authenticate. The result of logging in is to redirect back to the redirect URL. That redirect is intercepted by APISIX and leads to APISIX fetching a token from Keycloak based on the parameters in the redirect.

The token is then encrypted and added back in the response as a cookie for the original domain. The response is a redirect to the original /world URL.

In following the redirect, the browser now sends the session cookie. This is validated by APISIX and then passes the request straight to the Service.

If you then call /worlds again, the session cookie is passed and you receive the response straight away.

Should you go to /logout, APISIX intercepts the request and redirects you to /logout on Keycloak, with the session cookie. On confirming you want to log out, it then clears the session cookie and sends you back to /worlds.

As this URL is protected, you are immediately taken back to the login page!

Summary

In this article we…

• Looked at a way to protect our service using OIDC
• Discussed the difference between returning 302 or 401 from an API Gateway
• Created a Kecloak RealmCreated and configured APISIX as a Keycloak Client
• Obtained the information we needed from Keycloak
• Modified the APISIX route to secure our test service
• Updated the NGINX configuration
• Tested the configuration
• Looked at what was happening

If you found this article of interest, please give me a clap as that helps me identify what people find useful and what future articles I should write. If you have any suggestions, please add them in the comments section.