Keycloak: The Crown Jewel of Open Source Identity Management

In my 13 years of software experience, I’ve evaluated numerous identity providers, likely too many to count. The majority are cloud-based, catering to server-side client applications or Platform-as-a-Service providers like Firebase, Cognito, or Cloudflare access for managing client/user access. Each presents its own steep learning curve and a high risk of vendor lock-ins. Additionally, they tend not to be cost-effective. Supporting some large-scale apps in the industry has given me firsthand experience of how user access management costs can quickly escalate, particularly with multi-tenant solutions serving hundreds of thousands of users.

Integration also entails development costs, as each provider typically offers its own set of libraries and SDKs, further locking you into a specific service provider. Hence, it’s not always easy to build a secure, unified identity and access management control plane for all your heterogeneous and distributed applications spanning multiple providers and including diverse application stacks.

Needs no Introduction

Enter Keycloak! It stands as the crown jewel in the realm of open source identity and access management solutions. As an incubation project at Cloud Native Computing Foundation (CNCF) maintained by RedHat, it offers robust features such as single sign-on, social login, and centralized user management, all the while providing a seamless and secure experience for developers and enterprises alike. Designed to integrate effortlessly with a wide array of applications and services, Keycloak empowers you to manage digital identities with unmatched efficiency and ease.

This blog is designed to give a reader an introductory guide to setting up Keycloak and integrating it within your applications. We will cover:

1. Deploying Keycloak without High-Availability on Kubernetes (K8s) for evaluation purposes.
2. Using Cloudflare Zero Trust Tunnels to securely expose the Keycloak console to public internet without needing external load balancers or Ingress controllers.
3. Create a new testing realm and on-board a test user manually and try out some basic user authentication flows like resetting password with Keycloak’s hosted UI.
4. Integrate user authentication flows programmatically using a programming language (Dart — Not Flutter) that is not supported by Keycloak official libraries/adapters.

The grounds we will cover in this blog by no means represents a production deployment scenario and should not be used as one. But can be easily extended to a High availability architecture — since we are deploying this in Kubernetes.

I understand there are other Open-Source products but I don’t believe any of them are as stable, robust and scalable as Keycloak with it’s HA capabilities.

Step 1: Deploying Keycloak on Kubernetes

Keycloak offers various deployment options, including OpenJDK for bare-metal, Docker, Podman, OpenShift, and Kubernetes. I won’t delve deeply into the reasons for choosing Kubernetes as my deployment option, as I’ve covered it extensively in other Medium articles, demonstrating my fondness for the K8s community, perhaps at the expense of coming across rambling. Suffice it to say, it greatly simplifies your life. If you find the evaluation appealing and decide to move to production, Kubernetes with High-Availability makes it effortless, regardless of your chosen provider or hardware. You can self-host, deploy to any service provider, and migrate between clouds without significant re-architecting. This is the power K8s grants you — to develop and deploy applications that are truly cloud-agnostic.

However, as the saying goes, with great power comes great responsibility.

Let’s get started.

• Deploy a K8s cluster. This could be anywhere — from any Virtual machine on various cloud providers to something bare-metal sitting at your home.
  For example since this is just for evaluation purposes I am deploying this in a self-hosted Arm64 SBC at my home. It’s an inexpensive low power-usage SBC board OrangePi 5B that costs less than 150 US Dollars with a cooling unit mounted case and a 256 GB eMMC memory.

You can chose any K8s distribution or installer you like. How you deploy the Kubernetes cluster and provision the storage layer is not something we are going to cover in this article. Please refer to my article about more information and references about Kubernetes deployment and persistent storage.

# Create a K8s Cluster on a Linux machine using snap store.
# Refer: https://snapcraft.io/docs/installing-snapd
# Chosing 1 stable version below latest current since it has the widest support
# Below steps should take less than 1 min. 
sudo snap install microk8s --classic --channel=1.29
sudo microk8s status --wait-ready

# Join the group
sudo usermod -a -G microk8s $USER
mkdir -p ~/.kube
chmod 0700 ~/.kube
su - $USER

# Install or alias packaged kubectl and helm
alias kubectl="microk8s kubectl"
alias helm="microk8s helm"

# Provision a storage provider
# Refer: https://kubernetes-csi.github.io/docs/drivers.html
# You can chose any storage provider you like - but to keep things 
# purely open-source we are going to use Ceph. 
# Refer Step - 3: https://medium.com/@arko.basu09/self-host-a-gen-ai-powered-wordpress-site-using-kubernetes-on-any-commodity-hardware-10b29c53fb2d
# If you want to deploy a local ceph cluster. Please note that Ceph Mon 
# a good deal resources so use at least a machine which has 4cores and 8gb RAM.
# Make sure your system has RBD kernel available without which you won't be 
# able to use Ceph
sudo modprobe rbd
sudo microk8s enable rook-ceph

# Connect to an external ceph cluster. If you followed step3 from above
# article to provision ceph on the same cluster as your K8s, then 
# you don't need any of the additional arguments. Instead run 
# sudo microk8s connect-external-ceph 
# I am connecting to an external Ceph Cluster that I have at my home
# to back other applications. 
sudo microk8s connect-external-ceph --ceph-conf /path/to/minimal-ceph.conf \
--keyring /path/to/ceph.client.admin.keyring \
--rbd-pool k8s-test-rbd

# Check storage provisioning complete
kubectl --namespace rook-ceph-external get cephcluster

# Patch the default storage class so any new deployments use your provisioned
# persistent storage
kubectl patch storageclass ceph-rbd -p '{"metadata": {"annotations":{"storageclass.kubernetes.io/is-default-class":"true"}}}'

# Let's create a directory that will have the helm overrides for our Keycloak deployment
mkdir -p keycloak-testing && cd keycloak-testing 

# let's create our helm deployment overrides
cat <<EOF > values.yaml
image:
  tag: 24.0.5-debian-12-r0 ## Please use immutable tags

auth:
  adminUser: admin
  existingSecret: "keycloak-admin-user"
  passwordSecretKey: "password"

proxy: edge
EOF

# Let's create the referenced kubernetes secret for the above 
# deployment to use. Please keep it secure
k8s create secret generic keycloak-admin-user \
    --from-literal=password='S!B\*d$zDsb='

# Now let's set some environment variables and then deploy our keycloak application
export REPOSITORY_NAME=bitnamicharts
export REGISTRY_NAME=registry-1.docker.io
helm install keyclock-test-release \
  -f values.yaml oci://$REGISTRY_NAME/$REPOSITORY_NAME/keycloak

# Now give it a couple minutes to spin up.. And check it's status
kubectl get pods,svc,pvc -l app.kubernetes.io/instance=keyclock-test-release

If everything went well you should be able to see something like this:

Successful deployment of keycloak in your cluster than is not externally exposed using LB or Ingress

At this point we are ready to move on to our next step of exposing it to public internet without having to use Load Balancers or Ingress Controllers.

Step 2: Expose Keycloak admin console using Cloudflare Tunnels

I am not going to go into much detail about Cloudflare tunnels in this article. Please refer to Step 4 on aforementioned article for a product review and detailed step by step guide.

# Steps to be done from: Personal Computer and not the cluster
# Create a cloudflare tunnel and route it to a DNS record that is managed on 
# Cloudflare
cloudflared tunnel create keycloak-testing
cloudflared tunnel route dns keycloak-testing keycloak-testing.arkobasu.space

# Ship the tunnel crendentials JSON file to the machine where your Kubernetes 
# Cluster is running
scp ~/.cloudflared/<generated-tunnel-id>.json <username>@<ip-address>:/home/<username>/.tunnel_credentials.json

# Now let's save the tunnels credentials as a secret in the cluster
k8s create secret generic tunnel-credentials \
  --from-file=credentials.json=<full-path-of-file>

# Now let's copy the contents of 1 file in the folder you created earlier 
# from Cloudflare docs: https://github.com/cloudflare/argo-tunnel-examples/blob/master/named-tunnel-k8s/cloudflared.yaml
cd keycloak-testing
vim cloudflared.yaml

# Follow the isntrcutions in Step 4 of: 
# https://medium.com/@arko.basu09/self-host-a-gen-ai-powered-wordpress-site-using-kubernetes-on-any-commodity-hardware-10b29c53fb2d
# To check what all you need to update on this file
# You can find the service name for your keycloak service with the followin command
k8s get service keyclock-test-release-keycloak -o jsonpath='{.metadata.name}'

# Deploy the updated cloudflared with the service name for your Keycloak deployment
kubectl apply -f cloudflared.yaml

# Check for cloudflare tunnel status
kubectl logs $(kubectl get pod -l app=cloudflared -o jsonpath="{.items[0].metadata.name}")

At this point you should see that cloudflared has registered your tunnel and you are good to visit your app:

Successful deployment of Cloudflare tunnel exposing our cluster internal Keycloak application to the world

Keycloak admin console exposed to internet using cloudflare tunnels. Protected by SSL all the way to our cluster

Use the credentials you earlier used for deployment and login.

Admin console

Voila. You now have an enterprise grade Identity and access management system live that you can start using.

Step 3: On-board users with Keycloak as IDP

Now that we have Keycloak running, time to on-board some users and give take the application out for a spin. For this we will:

• Create a realm: A realm is a logical space within the keycloak deployment where you (an administrator) manage various entities such as users, applications, roles, and groups. Users are associated with a specific realm and log in to it. A single Keycloak deployment can define, store, and manage multiple realms, limited only by the database capacity. And since we are using Kubernetes and setup a distributed storage infrastructure for it — we don’t really need to worry about scaling up our application should the time and need come for it.
  Follow the instructions here to create a new realm after logging in.

Do not use the master realm for creating users or applications that use Keycloak for identity management.

New realm called testrealm

• Configure Email for Realm: Keycloak sends emails to users for purposes such as email verification, password recovery, or notifying administrators about server events. To enable this functionality, you need to configure Keycloak with your SMTP server settings.

Using application specific passwords from Zoho mail server to configure real email settings

Please do not use your actual passwords. Most Mail Services have a SMTP configuration that allows authentication using application specific password. I for example created an application specific password in Zoho and using their standard SMTP settings over SSL.

• Create a new User: In Keycloak, a user is linked to a set of attributes that help describe and identify them within the system and provide additional information to applications. A user profile establishes a clear schema for representing these attributes and managing them within a realm. By offering a consistent view of user information, it enables administrators to control various aspects of user attribute management. Follow the instructions to create a new user. The username field is the only required field — but feel free to add an email so we can validate reseting password functionality.

Creating a test user for myself in the testrealm

This article is just to give you an introductory evaluation experience with the Keycloak application. We are not going to delve into setting up Identity Brokering with Social Sign-in or SSO using SAML/OpenID Connect. Though it’s fairly straight forward as shown in their documentation to add an identity provider to an existing client within the deployment. Might do a follow up article later to review Social Sign-on and Active Directory integrations.

• Create a client [1]: Clients are entities that can request user authentication. There are two types of clients: applications seeking single sign-on functionality, relying on Keycloak for security, and clients requesting an access token to invoke other services on behalf of an authenticated user.
  In this article we are only going to cover the first one to get started with single sign-on functionality.

Create a minimal client that we will use to validate access for user

Please be extremely careful about using correct Web Origins and Redirect URLs. Exclusive redirect URI patterns are typically more secure.

“Make your registered redirect URIs as specific as feasible. Registering vague redirect URIs for Authorization Code Flows can allow malicious clients to impersonate another client with broader access.” — Keycloak documentation.

We will make some changes later to this client so we can test a local application during development as well. So as an administrator you can likely give users a test realm that is open to localhost or 127.0.0.1 hosts as redirect URLs for OAUTH flows. But for now lets test some access validation for the user.

Visit the following site:

https://<cloudflare-DNS>/realms/testrealm/account. In my case:

Login page for the test user on the test client we created earlier.

Post login hosted UI for for Keycloak to do self service user management

To confirm the client was created successfully, you can use the SPA testing application on the Keycloak website.

1. Open https://www.keycloak.org/app/
2. Change Keycloak URL to the URL of your Keycloak instance.
3. Click Save
4. Click Sign in to authenticate to this application using the Keycloak server you started earlier.

Testing Oauth flow using keycloak SPA testing application

Voila! You have a OpenID Connect client with which you can facilitate user authentication and protecting your websites.

Let’s try reseting the password. Click on Forgot Password and enter your email address:

Click on the link and reset your password and try logging back in with the updated password you updated

You can at this point seamlessly allow users to Register, integrate Social Sign-on providers like: Facebook, Google, Microsoft, Github and so many others. Have a look at he provider specific setup documentation if you would like to test out identity brokering. I have tested both Gmail and Facebook IDPs and they work without any issues. In the interest brevity I will not cover these topics in this article. If there are questions please don’t hesistate to reach out.

Great at this point, we have a OpenID Connect client that we can use to manage users. In the next section we will look into how do we integrate this Open ID Connect client into our applications.

Step 4: Integrate Keycloak OpenID Connect client into your application

In this section we are only going to cover the minimal steps to give you a rough idea on how to integrate OpenID connect clients, one like we just deployed. As a OAuth2, OpenID Connect, and SAML compliant server, Keycloak can secure any application or service that supports these protocols.

“Most of the support for some of these protocols is already available from the programming language, framework, or reverse proxy they are using. Leveraging the support already available from the application ecosystem is a key aspect to make your application fully compliant with security standards and best practices, so that you avoid vendor lock-in.” — Keycloak Documentation.

Keycloak has many adapters for widely used programming languages like: Javascript (client-side), Node-JS (server-side), C#, Python, Java, etc. But the idea is to show you how to use a sample Keycloak OpenID Connect client with any programming language that has a library with OpenID connect framework implemented. I for example love Flutter, Dart and Kotlin. And wanted to test out an authorization flow without too much complexity.

We are going to make slight modifications to our client earlier to allow Implicit Code Grants flow. This authorization flow should not be used in production scenarios. It’s just to show you the capability of quickly integrating with custom libraries that implement OpenID Protocols. Most of these libraries will implement the full protocol so you should easily be able to move to a more secure authorization flow.

Login to your testrealm console using your administrator account and make the following changes:

• Allow Implicit Flow for the testwebapp client.

Allowing Implicit flow

• Add the following to valid redirect URL and Web Origins. Please remember to remove it after testing.

So as to allow local testing

Without this change you will see error in the following steps since the Keycloak server will see this as your post-login local redirect URL for authentication as invalid.

• We are going to use a openid_client library from dart’s packages: https://pub.dev/packages/openid_client:

# Install flutter/dart. We are only going to use Dart to serve a 
# simple web app
# Clone the repository
https://github.com/appsup-dart/openid_client.git
cd example/browser_example

# Install necessary dependencies
dart pub dev
dart pub add build_runner build_web_compilers --dev
dart pub global activate webdev

# Update web/main.dart to point to your server
# https://github.com/appsup-dart/openid_client/blob/master/example/browser_example/web/main.dart#L5

# Serve the app locally. The port number should match what you allowed on 
# Redirect URL earlier.
webdev serve web:8888

At this point you should be able to open up a browser in your local system and visit 127.0.0.1:8888 . You should see a simple minimalistic web page that allows you to login to your Keycloak test OpenID Connect client.

Single page web-app redirects to our Keycloak Authentication self-hosted UI and then redirects back with profile information.

With this we have seamlessly integrated a self-hosted enterprise grade User Access management system into a custom web app/application. With this client we can at anytime highly control and extend the functionality with things like:

1. Single Sign-on for multiple applications using the same OpenID connect client.
2. Easy integration Identity brokering and Social Login providers.
3. Tightly control user attributes and authorization flows for various parts of any application.
4. Enable advanced security postures with highly customizable and hybrid authorization flows. Keycloak allows you to implement different access control policies like: ABAC, RBAC, UBAC, CBAC, etc. It also enables administrators for provisioning service accounts for server side applications that can seamlessly integrate with Kubernetes fullfilling Role based access control as well.
5. In addition it provides the ability to build an in-house Financial Grade API without having to worry about trust with pretty advanced security postures that is all plug and play.

In short, Keycloak is your one-stop show for all your identity and access management needs. And with the power of Kubernetes you can harness a cloud agnostic Identity and Access management system that seamlessly works across any and all your applications in modern day distributed environments.

I hope you enjoyed reading this article. Please don’t hesitate to reach out. Any feedback or suggestions are much appreciated.

References

[1] https://medium.com/keycloak/keycloak-realm-client-configuration-dfd7c8583489

[2] https://medium.com/@mihirrajdixit/getting-started-with-service-accounts-in-keycloak-c8f6798a0675