Open in app

Sign in

Write

Sign in

SSO-session failover with Keycloak and AWS S3

Georgijs Radovs
7 min readFeb 11, 2018

Hello, everyone!

As you may or may not know, Keycloak server supports High Availability cluster configurations — Standalone Clustered mode and Domain clustered mode.

In this article, I’m going to describe Standalone Clustered mode setup.

Standalone Clustered mode is used in small SSO-environments with no more than 2–4 Keycloak servers running, because Standalone Clustered mode requires manual server configuration file management on each of the cluster nodes. For example, if you have at least 2 Keycloak servers running in your environment, then you’ll have to manually manage standalone-ha.xml configuration file on both servers.

If your environment, has no more than 2–4 Keycloak servers running, then Standalone Clustered mode will be easier to configure and deploy. If your environment is bigger, then I’d suggest, you configure Domain Clustered mode.

In Domain Clustered mode, you will have a Domain controller, which will distribute Keycloak server configuration to a Keycloak server group. I tried to configure Domain Clustered mode in my environment, but I failed =)
If you would like to try Domain Clustered mode, please refer to Keycloak and Wildfly10 documentation on this topic

So, let’s start with Standalone Clustered mode setup.

If your environment is running in AWS cloud or other environment, where multicast is not supported, default JGroups configuration will not work, Keycloak servers won’t be able to communicate with each other.

One of the possible ways to resolve this is to use S3_PING protocol and AWS S3 service.

A quote from jgroups.org about what is S3_PING:

S3_PING uses Amazon S3 to discover initial members. New joiners read all addresses from this bucket and ping each of the elements of the resulting set of members. When a member leaves, it deletes its corresponding file.

It’s designed specifically for members running on Amazon EC2, where multicast traffic is not allowed and thus MPING or PING will not work. When Amazon RDS is preferred over S3, or if a shared database is used, an alternative is to use JDBC_PING.

Each instance uploads a small file to an S3 bucket and each instance reads the files out of this bucket to determine the other members.

So, basically, you need to have a AWS S3 bucket where SSO-session files will be stored, and your Keycloak servers configured to use S3_PING pointed at the bucket to read the SSO-session file from.

AWS S3 Configuration

To create and configure AWS S3 bucket, please, refer to AWS S3 documentation

There are multiple ways, how you can configure AWS S3 bucket access.
Again, a quote from jgroups.org:

Private buckets, Amazon AWS credentials given to each instance

Public readable and writable buckets, no credentials given to each instance

Public readable but private writable buckets, pre-signed URLs given to each instance

In this article, I’m going to describe how to configure private buckets with AWS credentials on each instance.

When you create a new S3 bucket, it should be private by default:

s3-ping-keycloak-example-com
S3 bucket public permissions disabled

With S3 bucket created, you will need to create a IAM user, which will have Read/Write access to the previously created S3 bucket.

To create a IAM user, do the following:

User creation for Keycloak to use S3_PING
  • Create Access keys for keycloak_s3_ping_user, so Keycloak server can access AWS using this user

With S3 bucket and user created, you will need to create and attach a IAM policy to this user, so he can access the S3 bucket:

keycloak_s3_ping_user permissions

And here is the S3 bucket access policy:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "s3:GetObjectVersion",
                "s3:DeleteObject",
                "s3:DeleteObjectVersion",
                "s3:GetObject",
                "s3:PutObject"
            ],
            "Resource": "arn:aws:s3:::s3-ping-keycloak-example-com/*"
        },
        {
            "Effect": "Allow",
            "Action": [
                "s3:ListBucketVersions",
                "s3:ListBucket"
            ],
            "Resource": "arn:aws:s3:::s3-ping-keycloak-example-com"
        }
    ]
}

After the IAM user and S3 bucket have been created, S3 policy attached, and access keys configured — it is time to configure S3_PING on the Keycloak server!

Keycloak configuration

  • SSH-login to your Keycloak server and change directory to where your Keycloak server is installed, to configuration directory which contains standalone-ha.xml file:
ssh user@keycloak.example.com
cd [keycloak_install_dir]/standalone/configuration
vim standalone-ha.xml

This file contains the usual Keycloak server configuration with the addition of WildFly10 High Availability extensions like Infinispan HA cache and JGroups HA communication channels and their configuration settings.

  • Open standalone-ha.xml file in a text editor, and find the line which starts with <subsystem xmlns="urn:jboss:domain:jgroups:4.0">

This subsystem block contains configuration for Wildfly10 JGroups functionality, in which we are going to enable S3_PING.

By default it should look something like this:

<subsystem xmlns="urn:jboss:domain:jgroups:4.0">
            <channels default="ee">
                <channel name="ee" stack="tcp"/>
            </channels>
            <stacks>
                <stack name="udp">
                    <transport type="UDP" socket-binding="jgroups-udp" site="example.com" machine="keycloak.example.com"/>
                    <protocol type="PING"/>
                    <protocol type="MERGE3"/>
                    <protocol type="FD_SOCK" socket-binding="jgroups-udp-fd"/>
                    <protocol type="FD_ALL"/>
                    <protocol type="VERIFY_SUSPECT"/>
                    <protocol type="pbcast.NAKACK2"/>
                    <protocol type="UNICAST3"/>
                    <protocol type="pbcast.STABLE"/>
                    <protocol type="pbcast.GMS"/>
                    <protocol type="UFC"/>
                    <protocol type="MFC"/>
                    <protocol type="FRAG2"/>
                </stack>
                <stack name="tcp">
                    <transport type="TCP" socket-binding="jgroups-tcp" site="example.com" machine="keycloak.example.com"/>
                   <protocol type="MPING" socket-binding="jgroups-mping"/>
                   <protocol type="MERGE3"/>
                    <protocol type="FD_SOCK" socket-binding="jgroups-tcp-fd"/>
                    <protocol type="FD"/>
                    <protocol type="VERIFY_SUSPECT"/>
                    <protocol type="pbcast.NAKACK2"/>
                    <protocol type="UNICAST3"/>
                    <protocol type="pbcast.STABLE"/>
                    <protocol type="pbcast.GMS"/>
                    <protocol type="MFC"/>
                    <protocol type="FRAG2"/>
                </stack>
            </stacks>
        </subsystem>

To enable S3_PING, we need to edit tcp stack:

<subsystem xmlns="urn:jboss:domain:jgroups:4.0">
            <channels default="ee">
                <channel name="ee" stack="tcp"/>
            </channels>
            <stacks>
                <stack name="tcp">
                    <transport type="TCP" socket-binding="jgroups-tcp" site="example.com" machine="keycloak.example.com"/>
                   <protocol type="MPING" socket-binding="jgroups-mping"/>
                   <protocol type="MERGE3"/>
                    <protocol type="FD_SOCK" socket-binding="jgroups-tcp-fd"/>
                    <protocol type="FD"/>
                    <protocol type="VERIFY_SUSPECT"/>
                    <protocol type="pbcast.NAKACK2"/>
                    <protocol type="UNICAST3"/>
                    <protocol type="pbcast.STABLE"/>
                    <protocol type="pbcast.GMS"/>
                    <protocol type="MFC"/>
                    <protocol type="FRAG2"/>
                </stack>
            </stacks>
        </subsystem>
  • All you have to do is add new protocol block:
<protocol type=”S3_PING”>
      <property name=”access_key”>
          keycloak_s3_ping_user AWS IAM access key
      </property>
      <property name=”secret_access_key”>
          keycloak_s3_ping_user AWS IAM secret key
      </property>
      <property name=”location”>
          s3-ping-keycloak-example-com <-- AWS S3 bucket name
      </property>
</protocol>
  • So now, tcp stack with S3_PING added should look like this:
<stack name="tcp">
    <transport type="TCP" socket-binding="jgroups-tcp" site="example.com" machine="keycloak.example.com"/>
    <protocol type="MPING" socket-binding="jgroups-mping"/>
       <protocol type=”S3_PING”>
           <property name=”access_key”>
               keycloak_s3_ping_user AWS IAM access key
           </property>
           <property name=”secret_access_key”>
               keycloak_s3_ping_user AWS IAM secret key
           </property>
           <property name=”location”>
               s3-ping-keycloak-example-com
           </property>
       </protocol>
       <protocol type="MERGE3"/>
       <protocol type="FD_SOCK" socket-binding="jgroups-tcp-fd"/>
       <protocol type="FD"/>
       <protocol type="VERIFY_SUSPECT"/>
       <protocol type="pbcast.NAKACK2"/>
       <protocol type="UNICAST3"/>
       <protocol type="pbcast.STABLE"/>
       <protocol type="pbcast.GMS"/>
       <protocol type="MFC"/>
       <protocol type="FRAG2"/>
</stack>
  • Save and close standalone-ha.xml file.

To test new configuration, you can run Keycloak server in foreground:

  • cd to standalone/bin directory:

user@keycloak_server_fqdn:~ cd /opt/keycloak-3.0.0.Final/bin

  • Execute standalone.sh script with standalone-ha.xml file:

./standalone.sh -c standalone-ha.xml

When Keycloak server starts, you should see something like this in the output:

Keycloak running in foreground with JGroups enabled

Notice, there is only one Keycloak server detected - keycloak.example.com

Also, s3-ping-keycloak-example-com S3 bucket should contain something like this:

Inside the s3-ping-keycloak-example-com bucket, there should be ee directory which is used by ee stack configuration from standalone-ha.xml:

<channels default="ee">
    <channel name="ee" stack="tcp"/>
</channels>

Inside ee directory, there should be a .list file or files, containing information about Keycloak HA-cluster members:

These text files, are used by Keycloak to keep track of the Keycloak HA-cluster members, which will pick up SSO-sessions from Keycloak servers that went offline.

So, S3 bucket and the first Keycloak server is configured for SSO-session failover. Now, it is time to add a second Keycloak server!

Edit standalone-ha.xml file on the second Keycloak server the same way as the first Keycloak server. Now, assuming your first Keycloak server is still running in foreground — launch the second Keycloak server and watch the output of the first Keycloak server:

Had to truncate the output because it would not fit the screen

As you can see, keycloak2.example.com has joined the party =)

And here is what happens, when keycloak.example.com left the party =( :

Now, you can create a scenario where one of the Keycloak servers goes offline, and test if the SSO-session failover works as expected.

Thank you for reading this lenghty article, I hope it will be of some use to you. If you notice any mistakes or incorrect information — please, write in the comments and I will update it.

May the Gods give you strenght and provide wisdom.
Live long and prosper! =)

AWS
Keycloak
Sso
Failover
Amazon S3

--

--

Georgijs Radovs

Written by Georgijs Radovs

49 Followers

Sysadmin, powerlifter, dude with a beard.

Help

Status

About

Careers

Press

Blog

Privacy

Terms

Text to speech

Teams