Kconnect: A Better Way to Connect to Kubernetes Clusters

Authors include Sean Kelly and Rob Casale

Dmitry Kovalchuk /Shutterstock.com

Kubernetes comes in many different shapes and forms across different cloud providers, and while kubectl/helm acts as a uniform way to interact with the clusters, connecting to them is another matter entirely. Depending on your company’s IDP and cloud provider, there are a multitude of ways of connecting to clusters, such as obtaining a whole KUBECONFIG file from a webpage (Rancher) to running a CLI with very limited features (EKS). Sometimes the authentication flow will involve having to go to a browser to obtain an access token (AKS), certainly not ideal for use in pipeline. Sometimes there is no easy way.

At Fidelity Investments, we had a challenge. We needed to connect to our EKS clusters, but to authenticate we had to use Ping NTLM and SAML. To solve this issue, we created an in-house tool, eksconnect. This CLI leveraged the brilliant saml2aws library (https://github.com/Versent/saml2aws), and we added some nice-to-have features with it. These features include showing previous connection history and being able to interactively select which IAM role to use. Later, some teams required Kubernetes clusters on Azure (AKS) and we had similar problems. Therefore, we also created aksconnect to deal with these issues.

We saw an opportunity to combine both tools and provide a common interface, as well as a chance to contribute back to the open-source community. The goal of kconnect was to provide an easy-to-use interface to connect to any K8s cluster, across a multitude of CSPs (AWS, AKS and RKS are currently supported), and to support a wide array of different identity providers and protocols. If we could achieve this, it would hugely improve the developer experience, as well as substantially speed up other companies setting up their Kubernetes environments.

Prerequisites

Before we start, it is important to mention that if you are going to follow along, or plan on using kconnect in the future, then you will need some additional binaries depending on which CSP you are attempting to connect to:

• If connecting to AWS aws-iam-authenticator is required.
• If connecting to AKS kubelogin is required.

How to use Kconnect (the hard way)

Let’s take kconnect for a spin, without any additional configuration setup. For these first demos we will be using EKS but will quickly demonstrate AKS and Rancher later.

~ $ kconnect use eks --idp-protocol saml
? Select your identity provider
  ADFS
  ADFS2
  Akamai
  AzureAD
  F5APM
  GoogleApps
  JumpCloud
  KeyCloak
  NetIQ
  Okta
  OneLogin
  PSU
  Ping
> PingNTLM
  Shibboleth
  ShibbolethECP
? Enter the endpoint for the IdP https://endpoint.yourcompany.com
? Select an AWS region
...
> us-east-1
  us-east-2
  us-west-1
  us-west-2
? Username: seank
? Password: ****************
? Select AWS role Account:
  Account: myaccount1 (111222333444) / EKS_Role1
  Account: myaccount2 (555666777888) / EKS_Role2
> Account: myaccount3 (999000111222) / EKS_Role3
info    requesting AWS credentials using SAML   {"provider": "saml", "sp": "aws"}
info    discovering clusters    {"app": "kconnect", "provider": "eks"}
info    discovering EKS clusters        {"app": "kconnect", "provider": "eks"}
? Do you want to set an alias? (y/N)N
info    setting current context {"context": "kconnect-2714388176@mycluster3"}
info    kubeconfig updated      {"path": "/home/seank/.kube/config"}
~ $ kubectl get pods
NAME      READY   STATUS    AGE
mypod1    1/1     Running   53d
mypod2    1/1     Running   53d
mypod3    1/1     Running   19d

Voila! There is kconnect in action! We are now connected to our cluster, and we can now run our kubectl commands against it. However, it is a bit bulky and awkward to use in this state. We need to remember our IDP protocols and config and enter them every time. Let’s make things a bit easier to use.

How to use Kconnect (the easy way)

Config

The first step in making kconnect easier to use is by setting up our default configuration. This is done by creating a config file:

Let’s see an example config file now, and how it will make life easier:

~ $ cat myconfig.yaml
apiVersion: kconnect.fidelity.github.com/v1alpha1
kind: Configuration
spec:
  global:
    username: seank
    providers:
  eks:
    idp-endpoint: https://myendpoint.company.com
    idp-protocol: saml
    idp-provider: PingNTLM
    region: us-east-1
    role-filter: EKS

First, we have our global config section. This will apply to every command we use, regardless of provider. Here we will specify our username, so we no longer need to provide this in the CLI.

Next is the EKS-specific section. First, we are defining our IDP configuration. Now we no longer need to provide the — idp-protocol flag, or the provider and endpoint. Additionally, we are going to always use the us-east-1 region and only show IAM roles that contain “EKS.” There are a lot more options available, for example if we only want the US regions to be displayed, we can specify that by adding the config: “region-filter: ^us-”.

Now we need to import this configuration:

~ $ kconnect config -f myconfig.yaml
info importing configuration {"file": "config.yaml"}
info successfully imported configuration

This configuration import can also be done through a URL, so that companies can allow developers to easily obtain the correct details:

~ $ kconnect config -f http://kconnectconfig.yourcompany.com
info importing configuration {"file": "http://kconnectconfig.yourcompany.com"}
info successfully imported configuration

Now that we have our configuration done, let’s run kconnect again:

~ $ kconnect use eks
? Password: ***************
? Select AWS role Account:
  Account: myaccount1 (111222333444) / EKS_Role1
  Account: myaccount2 (555666777888) / EKS_Role2
> Account: myaccount3 (999000111222) / EKS_Role3
? Do you want to set an alias? Y
? Enter the alias name mycluster1
info Command to reconnect using this alias: kconnect to mycluster1 {"app": "kconnect"}
info kubeconfig updated {"path": "/home/seank/.kube/config"}

And it’s as easy as that! This time, all we had to input was our password and AWS IAM role. If we had multiple clusters in the same account and region, we would have to select which cluster to connect.

Aliases

The next step to making kconnect easier to use is the concept of aliases. You may have noticed in the above command that when prompted to set an alias, we said yes and input a name of mycluster1. If we wanted to reconnect to this cluster, due to either credentials expiring or we were working on a different cluster, all we would have to run is:

~ $ kconnect to mycluster1
? Password: ***************
info kubeconfig updated {"path": "/home/seank/.kube/config"}

History

If we want to view a list of historic connections we have made, we can utilize the history command:

~ $ kconnect ls
CUR ID        ALIAS      PROVIDER PROVIDERID         USER  TIME LEFT
>   01f123123 mycluster1 eks      arn:...mycluster3  seank 56m40s
    01f12asa9 mycluster2 eks      arn:...mycluster2  seank 0s
    01f12asa9 mycluster3 eks      arn:...mycluster1  seank 0s

If we wanted to reconnect to “mycluster2”, we could run “kconnect to mycluster2”. If we had forgotten to set the alias, we also could have run “kconnect to 01f12asa9.”

We can also select a cluster to reconnect to interactively from the above history list.

~ $ kconnect to
? Select a history entry [Use arrows to move, type to filter]
CUR ID        ALIAS      PROVIDER PROVIDERID         USER  TIME LEFT
>   01f123123 mycluster1 eks      arn:...mycluster3  seank 56m40s
    01f12asa9 mycluster2 eks      arn:...mycluster2  seank 0s
    01f12asa9 mycluster3 eks      arn:...mycluster1  seank 0s
? Password: ***************
info kubeconfig updated {"path": "/home/seank/.kube/config"}

Additional Tips

If you want to quickly reconnect to your last used cluster:

~ $ kconnect to -
? Password: ************
info kubeconfig updated {"path": "/home/seank/.kube/config"}

You can also set kconnect arguments as environmental variables e.g. password to avoid having to type it every time

~ $ export KCONNECT_PASSWORD="mypassword"
~ $ kconnect to -
info kubeconfig updated {"path": "/home/seank/.kube/config"}

Specify namespace to use

~ $ kconnect use eks --namespace mynamespace
…
info kubeconfig updated {"path": "/home/seank/.kube/config"}
~ $ kubectl get pods
No resources found in mynamespace namespace.

Kconnect & AKS

Kconnect also works with AKS. Let us look at an example

~ $ kconnect use aks
? Username: seank
? Password: ***************
? Enter the Azure tenant ID 1234-b54s-asda-56756-c123laksjd
? Choose the Azure subscription [Use arrows to move, type to filter]
  azure-sub1-dev
> azure-sub2-dev
  azure-sub3-dev
? Select a cluster [Use arrows to move, type to filter]
> aks-cluster-1
  aks-cluster-2
  aks-cluster-3
? Do you want to set an alias? N
info kubeconfig updated {"path": "/home/seank/.kube/config"}

Rancher

~ $ kconnect use rancher --idp-protocol rancher-ad
? Username: seank
? Password: ***************
? Enter the Rancher API endpoint https://rancher-dev.yourcompany.com/v3
? Do you want to set an alias? N
info kubeconfig updated {"path": "/home/seank/.kube/config"}

Conclusion

Kconnect has greatly simplified the cloud development experience in Fidelity by allowing our Kubernetes users to not have to worry about how to connect to their clusters, in both their local development environments and in their deployment pipelines. This has saved on countless hours and effort, allowing our developers, DevOps engineers, and SREs to focus more on meaningful work.

For anyone interested in seeing the source code, or getting more information from the documentation, you can find the project open-sourced on GitHub:

• https://github.com/fidelity/kconnect
• https://fidelity.github.io/kconnect/