Switching Between Kubernetes Clusters in GKE

with the help of direnv

Recently I’ve the opportunity to play with Kubernetes in multiple projects that live in Google Cloud Platform (GCP). This is an exciting experience but also a scary one as I might forget to switch context and apply the changes to the wrong project/cluster unintentionally. Therefore, I am looking for a way that would switch to the proper gcloud config and Kubernetes context when I am in the directory of a specific project. The obvious tool that can help me to do so is direnv and with it, I’ve arrived at a solution that works for me using KUBECONFIG and CLOUDSDK_ACTIVE_CONFIG_NAME.

Setup gcloud config

To get started, first we need to create a new gcloud config using the init sub-command

$ gcloud init
Welcome! This command will take you through the configuration of gcloud.
Settings from your current configuration [...] are:
core:
  disable_usage_reporting: 'True'
Pick configuration to use:
 [1] Re-initialize this configuration [...] with new settings
 [2] Create a new configuration
 [3] Switch to and re-initialize existing configuration: [...]
Please enter your numeric choice:  2

and choose “Create a new configuration”, then

1. enter a configuration name,
2. choose an account for this new config,
3. choose a project to be associated to this new config,
4. setup the default Compute Region and Zone.

Once the config is setup, we can check its status with

$ gcloud config configurations list

Setup Kubernetes config

The idea for this to work is to keep a different Kubernetes config file for each project. Set these two environment variables in the root of our project directory

$ export CLOUDSDK_ACTIVE_CONFIG_NAME=<gcloud-config-name>
$ export KUBECONFIG=`pwd`/kube-config

• CLOUDSDK_ACTIVE_CONFIG_NAME is to make sure that the gcloud config that we setup in the previous section is activated (see gcloud topic configurations).
• KUBECONFIG is to tell kubectl to use the specific config-file named kube-config in our project directory (see kubectl config --help). Note that, at this point, the config file kube-config does not exist at the specified location.

We can then run the following command to initialize the proper configuration for kubectl

$ gcloud container clusters get-credentials <cluster-name>

As the result, the kube-config is created for the <cluster-name> in the GCP project described in CLOUDSDK_ACTIVE_CONFIG_NAME.

Automate the Switching

The two environment variables mentioned above can be set in the .envrc (we can create a new one by running direnv edit . in the root directory of the project) file of direnv as

export CLOUDSDK_ACTIVE_CONFIG_NAME=<gcloud-config-name>
export KUBECONFIG=`pwd`/kube-config

This will ensure that the kubeclt, helm, … commands that are executed within the project directory will use the correct context as specified in KUBECONFIG. The downside to this setup is that we can only run those commands inside the project directory.