This has been documented elsewhere but I think it’s worthwhile to summarize
gcloud use of configs, their interaction, and how to switch between Kubernetes configs when you have multiple Kubernetes clusters.
I’m using Kubernetes Engine which makes configuration management much easier but — if you’re like me — this can result in complacency in letting the tools do their job without spending time to grok what’s going on.
Let’s assume you’ve created 2x Kubernetes Engine clusters and obtained credentials for both of them. You’ll be configured to use the cluster that was most recently authenticated:
gcloud container clusters create black ...
gcloud container clusters create white ...gcloud container clusters get-credentials black ...
gcloud container clusters get-credentials white ...kubectl get nodesNAME STATUS ROLES AGE VERSION
white-2378271e-qpd6 Ready <none> 1h v1.10.2-gke.3
white-5d71e034-njph Ready <none> 1h v1.10.2-gke.3
white-f4567e1d-4rwd Ready <none> 1h v1.10.2-gke.3
NB I’ve renamed
NAMEin the above to demonstrate the point.
We’re authenticated (currently) for our
Let’s have a look at Kubernetes (Engine) configuration:
kubectl config view
The underlying configuration file is probably (!) located in:
Your config will resemble this file. I’ve redacted and renamed some elements for clarity.
There are 2 clusters, 2 contexts and 2 users here. There’s also a property
current-context that points to the current context. One of the two names. In this case
One thing that may not be obvious is that, since I created both clusters (
black) and I authenticated against both of them, I don’t need to have 2 different but identical users in my configuration. They both reflect the same user. I propose that you don’t this but for clarity, this configuration is identical to the above:
NB I’ve renamed one of the 2
userswith my user
name, deleted the duplicate and renamed the references in
contextswith my user
Let’s drill into this
Grab the access-token from your
kubectl config view. It should be of the form
ya29.... Let’s call it
NB The expiry in the results. If you try this command after that time, your token will have expired. If it has, make an arbitrary e.g.
kubectl get nodescall to force a refresh of the token, rerun the
kubectl config viewand try again.
Then you may browse or curl:
"scope": "https://www.googleapis.com/auth/userinfo.email https://www.googleapis.com/auth/cloud-platform https://www.googleapis.com/auth/appengine.admin https://www.googleapis.com/auth/compute https://www.googleapis.com/auth/plus.me",
NB email reflect the account that you are using with
gcloud. If you review your
gcloud auth listor
gcloud config listor, even more precisely,
gcloud config get-value account), this should be your
cmd-args and 2 properties
cmd-path should match:
cmd-args isn’t well-documented by Google (filed a bug :-)) but it is explained by one of Google’s Engineers (link):
gcloud config config-helper
or you can tack on
kubectl, it uses the
auth-provider configuration to use
gcloud config config-helper to grab an
access_token for your account so that it may access a cluster’s resources.
Let’s briefly detour through
configurations to explain this.
gcloud config configurations
You’ll note in the above that
configurations too. I’ve used this facility less frequently but, for completeness…
gcloud config list
account = [[YOUR-EMAIL]]
disable_usage_reporting = FalseYour active configuration is: [default]
gcloud store its configuration? Probably:
ls -la ~/.config/gcloud
NB Edited to demonstrate the point
Which is a key to an entry in configurations prefix
account = [[YOUR-EMAIL]][container][compute]
NB Your configuration may have more properties.
This should reflect what the result of
gcloud config list.
You may (!) create multiple configurations to reflect different GCP configurations that you wish to use. For example:
gcloud config configurations list
NAME IS_ACTIVE ACCOUNT
default True [[YOUR-EMAIL]]gcloud config configurations create henry
Activated [henry].gcloud config configurations list
NAME IS_ACTIVE ACCOUNT
default True [[YOUR-EMAIL]]
Then, you can verify:
henryls -l ~/.config/gcloud/configurations
And, tidy up with:
gcloud config configurations activate defaultgcloud config configurations delete henry
The following configurations will be deleted:
Do you want to continue (Y/n)? YDeleted [henry].
I find little use of gcloud configurations and instead tend to have multiple logins against the
gcloud auth list
[[EMAIL-#3]]To set the active account, run:
$ gcloud config set account `ACCOUNT`
And, as shown above, switch between these using, e.g.
gcloud config set account [[EMAIL-#3]].
kubectl config use-context
OK, we now understand how
gcloud to auth against GCP. But, we’ve not yet addressed how to manage multiple
This is documented (link) but here’s my spin on it. Let’s return to our
black clusters. As I mentioned, Kubernetes Engine will create unique names for your
users. I’ve explained how you may (you don’t need to and be careful doing it) rename|consolidate your
users. You may do the same thing for
contexts too. Just be careful that you correctly rename all references to these in your config file.
If you break something, you may rerun
gcloud container clusters get-credentials to re-authenticate against your cluster(s).
So, how do we switch between clusters? It’s actually straightforward. Assuming:
kubectl config get-contexts
CURRENT NAME CLUSTER AUTHINFO NAMESPACE
* white white dazwilkin
black black dazwilkin
NB You’ll need to use whatever values of
NAME are defined in your config.
kubectl config use-context black
Switched to context "black"kubectl config use-context white
Switched to context "white"
And, if you want to make life simplier for yourself:
alias black="kubectl config use-context black"
Switch to context "black"
kubectl with Kubernetes Engine leverages Google Cloud Platform’s
gcloud CLI to facilitate auth against clusters. This post summarizes how to switch between auth’d clusters without using
gcloud repeatedly to get credentials.
Along the way, the post provides some handle-with-care instructions for simplifying a
kubectl configuration, and a summary of how
gcloud manage configuration and where the data is stored.
Feedback always welcome!
by the author.