Securely Authenticate the GKE/Anthos cluster with Okta

What is Okta?

Okta is a secure solution to add authentication and authorization to the applications. It will secure our application without any security risk and deployment overhead.

Objective

To secure the Anthos cluster with Okta, so that unauthenticated users cannot access your cluster.

For this first, Deploy the Anthos cluster on AWS, and for authentication and authorization via Okta, please follow the below steps.

Creating Role and Policy for SSM

SSM is a session manager which helps to manage or login the EC2 instance with an interactive one-click browser. Creating a key and using the key to login is not a best practice, so that’s why use SSM, below are the permissions that are required for SSM.

1. Navigate to the AWS console.
2. Create a policy with the following permissions.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "ssmmessages:CreateDataChannel",
                "ssmmessages:OpenControlChannel",
                "ssmmessages:CreateControlChannel",
                "ssmmessages:OpenDataChannel",
                "ssm:UpdateInstanceInformation",
                "ssm:StartSession",
                "ssm:TerminateSession",
                "ssm:ResumeSession",
                "ssm:DescribeSessions",
                "ssm:GetConnectionStatus"
            ],
            "Resource": "*",
            "Condition": {
                "StringEquals": {
                    "aws:RequestedRegion": [
                        "us-east-1",
                        "us-east-2"
                    ],
                    "aws:PrincipalAccount": "xxxxx"
                }
            }
        }
    ]
}

3. Create a Role and add the above-created policy to it.

Creating EC2 Instance

As a best practice, to access the AWS Anthos cluster via Okta we need to create an EC2 instance, Below are the steps to create an instance

1. Navigate to the AWS console.
2. Create an Amazon Linux EC2 instance in the same VPC with the public subnet in which the Anthos cluster is created.
3. Attach the Role created above for SSM access.
4. Create an inbound rule for a port [PORT] (use [PORT] greater than 1024) with your local IP for authenticating the cluster via Okta.
5. Install gcloud and kubectl inside the instance.

Creating redirect URLs

Google cloud redirect URL

Cloud SDK is installed on each developer’s local machine and includes the gcloud CLI. A port number should be greater than 1024 to use for the redirect URL:

http://instance_publicIP:[PORT]/authorization-code/callback

Replace [PORT] with your port number greater than 1024.

When you configure the Google provider, specify http://instance_publicIP:[PORT]/authorization-code/callback as one of your redirect URLs.

Cloud Console redirect URL

The redirect URL for Cloud Console is:

https://console.cloud.google.com/kubernetes/oidc

When you configure your OIDC provider, specify https://console.cloud.google.com/kubernetes/oidc as one of your redirect URLs.

Creating Okta Application

Create an account on Okta using the business email id https://www.okta.com/free-trial/ and logged in as an Admin User

1. In the Admin Console, go to Applications, and then select Applications.
2. Click on Create App Integration.
3. To create an OIDC app integration, select OIDC — OpenID Connect and Web Application then hit next.

Configure initial settings

In General Settings:

1. App name: Specify a name for your application.
2. App logo : This is an optional field where you can add a logo for your app.
3. In Configure OpenID Connect:
4. Sign-in redirect URIs: Add your two redirect URLs, Recall that you created above a redirect URL for the gcloud CLI and a redirect URL for Cloud Console.
   The Sign-in redirect URIs is where Okta sends the authentication response and ID token for the user's sign-in request.
5. Sign-out redirect URIs (Optional): After your application contacts Okta to close the user session, Okta then redirects the user to one of these URIs.
6. Click Save. This creates the integration and opens the settings page to configure additional options.
7. Save the Client ID and the Client secret will be given to you, which will be used later.

Enable Refresh token

1. In the Admin Console, go to Applications, and then select Applications.
2. Select the OpenID Connect client application that you created above.
3. Navigate to the General Tab and click Edit in the General Settings section.
4. Locate to Grant type.
5. Select Refresh Token.
6. Click Save.

Add Group Claims

1. In the Admin Console, go to Applications, and then select Applications.
2. Select the OpenID Connect client application that you created above.
3. Navigate to the Sign On tab and click Edit in the OpenID Connect ID Token section.
4. In the Groups claim type section, select Filter.
5. In the Group claims filter section, leave the default name groups (or add it if the box is empty), select Matches regex, and enter .* to return all of the user's Groups.

Adding People

Launch the Wizard

1. In the Admin Console, go to Directory, then select People.
2. Click Add People.
3. Enter a First name, Last name, and Username as email.
4. Click Save or Save and Add Another.

Creating Okta group

Launch the Wizard

1. In the Admin Console, go to Directory, then select Groups.
2. Click Add Group.
3. Enter a name in the Name field for the group.
4. Description. This is an optional field where we can enter a description for the group.
5. Click Save.

Add Users to Group

1. In the Admin Console, go to Directory, then select Groups.
2. Select the Group you created above.
3. Click Assign People.
4. In the Search field, Enter the name of the person.
5. Click + to assign the person to the group.
6. Click Save.

Add Applications to Group

1. In the Admin Console, go to Directory, then select Groups
2. Select the Group you created above.
3. Click Applications, and then select Assign Applications.
4. Please enter the name of the application that we created before in the Search field.
5. Click Assign to assign the application to the group.
6. Click Done.

Configuring Authorisation Server

Add groups scope

1. In the Admin console, go to Security, then select API.
2. Select the Authorisation Server that you want to configure. example: default
3. Navigate to the Scopes tab.
4. Click Add Scope.
5. Enter the Name as groups.
6. Enter the Display phrase as groups.
7. Optional. Enter a description for the scope in the Description field.
8. Select Include in public metadata.
9. Click Create.

Add groups claim

1. In the Admin console, go to Security, then select API.
2. Select the Authorisation Server that you want to configure. example: default
3. Navigate to the Claims tab.
4. Click Add Claim.
5. Enter the Name as groups.
6. Configure the claim settings:
7. Include in token type: Select ID Token as Always.
8. Value type: Select Groups.
9. Filter: Select Matches regex and enter .*.
10. Select Any scope for Include in the field.
11. Click Create.

Connect to the Anthos cluster from the EC2 instance

gcloud container hub memberships get-credentials cluster_name

Edit the config file

kubectl --kubeconfig=KUBECONFIG_PATH edit ClientConfigs default -n kube-public

Replace KUBECONFIG_PATH with the path to your cluster’s kubeconfig file — for example, $HOME/.kube/config

A text editor loads your cluster’s ClientConfig resource. Add the part in the file as shown below. Do not modify any default data that has already been written.

Configuring your cluster

spec:
  authentication:
    - aws:
        region: XXXX
      name: oidc
      oidc:
        clientID: CLIENT_ID
        clientSecret: CLIENT_SECRET
        cloudConsoleRedirectURI: XXXX
        groupsClaim: groups
        issuerURI: ISSUER_URI
        kubectlRedirectURI: http://instance_publicIP:[PORT]/authorization-code/callback
        scopes: offline_access,email,profile,groups
        userClaim: email

An updated version of the gcloud tool with the anthos-auth components

$ gcloud components update

$ gcloud components install anthos-auth

Create the configuration with gcloud anthos

$ gcloud anthos create-login-config --kubeconfig $HOME/.kube/config

On Linux and macOS, by default, this file is at ~/.kube/config

This command generates a file (kubectl-anthos-config.yaml) containing the configuration information.

Copy the config file to a location

$ cp kubectl-anthos-config.yaml $HOME/.config/google/anthos/kubectl-anthos-config.yaml

Login config default directories

The default locations for storing the configuration file for each OS are as follows:

Linux

$HOME/.config/google/anthos/kubectl-anthos-config.yaml, where $HOME is the user's home directory.

macOS

$HOME/Library/Preferences/google/anthos/kubectl-anthos-config.yaml, where $HOME is the user's home directory.

Windows

%APPDATA%/google/anthos/kubectl-anthos-config.yaml, where %APPDATA% is the user's application data directory.

Creating an RBAC policy for your user cluster

Define a Role. Copy the following YAML into a file named role.yaml

apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  namespace: role-test
  name: pod-reader
rules:
- apiGroups: [""]
  resources: ["pods", "services", "secrets", "nodes"]
  verbs: ["get", "watch", "list"]

Define a RoleBinding. Copy the following YAML into a file named role-binding.yaml

apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: read-pods
  namespace: role-test
subjects:
- kind: Group
  name: XXXX    # "name" is case sensitive, name of the group which is created in okta console
  apiGroup: rbac.authorization.k8s.io
roleRef:
  kind: Role 
  name: pod-reader 
  apiGroup: rbac.authorization.k8s.io

Apply role.yaml and role-binding.yaml to your cluster with kubectl

Authenticating

Use gcloud Anthos auth login to authenticate to your cluster with your login config.

Default

$ gcloud anthos auth login --cluster CLUSTER_NAME

Replace CLUSTER_NAME with a fully qualified cluster name. For example, projects/my-gcp-project/locations/global/awsClusters/cluster_name.

Optional parameters

gcloud Anthos auth login support the following optional parameters:

gcloud anthos auth login --cluster CLUSTER_NAME \
--user USERNAME --login-config ANTHOS_CONFIG_YAML \
--login-config-cert LOGIN_CONFIG_CERT_PEM \
--kubeconfig=KUBECONFIG --dry-run

this command will redirect to an Okta login page to authenticate the user and once the authentication is successful, that particular user will be able to access the services. Hence the user will have permission to the Anthos cluster via Okta.

Example login config

apiVersion: authentication.gke.io/v2alpha1
kind: ClientConfig
metadata:
  creationTimestamp: null
spec:
  authentication:
  - aws:
      region: XXXX
    name: oidc
    oidc:
      clientID: XXXX
      clientSecret: XXXX
      cloudConsoleRedirectURI: XXXX
      groupsClaim: groups
      issuerURI: XXXX
      kubectlRedirectURI: http://instance_publicIP:[PORT]/authorization-code/callback
      scopes: offline_access,email,profile,groups
      userClaim: email
  certificateAuthorityData: XXXX
  internalServer: ""
  name: projects/XXXX/locations/XXXX/awsClusters/XXXX
  server: https://gke-p5jkh55ausbq-cp-6047815259562bb2.elb.us-east-1.amazonaws.com
status: {}

Whitelist IPs Via OKTA on a specific Application level to restrict access to the Anthos cluster

FYI: The Admin user of Okta will have successful authentication, even if his IP is not whitelisted.

1. In the Admin Console, go to Security, then select Networks.
2. Click the LegacyIpZone Edit button.
3. Add allowlist IP in Gateway IPs.

1. In the Admin Console, go to Security, then select Authentication policies.
2. FYI: Before proceeding to the next step, ensure that the Application is attached to the Default policy.

3. Click the Default policy.

4. Click on Add Rule.

5. Enter your Rule name.

6. Select “In any of the following zones:” and select ZONE in Gateway IPs.

7. Select the “Allowed after successful authentication” radio button in Access is and SAVE

8. Or we can Deny the IP and create a rule in the same default policy.

If IP is whitelisted of an okta user, then it will show below successful response

If IP is not whitelisted of an okta user, then it will show below unsuccessful response

Thank you for reading :)

Questions?
If you have any questions, I’ll be happy to read them in the comments.

Follow me on LinkedIn.

Happy Learning !!