Safer `gcloud` and `kubectl`

How to protect yourself from yourself

OK, this get its own story!

I’ve long sought a safety-net with kubectl that I use all-the-time with gcloud. Today, I realized that it’s been there and I just didn’t realize it.

Now I can:

kubectl get nodes
The connection to the server localhost:8080 was refused - did you specify the right host or port?
kubectl get pods
The connection to the server localhost:8080 was refused - did you specify the right host or port?

… and this makes me happy!

Let me back-up… because I think this is a decent practice and I’ve long-used it with gcloud.

gcloud

Google Cloud Platform’s CLI (informally called gcloud) like kubectl wants to save time by persisting|assuming context. You’ll often see:

gcloud config set project ${PROJECT}
gcloud [[ create marvels]]
gcloud [[ delete marvels]]
gcloud project delete

All good.

The problem is — if like me — you’re fallible, sometimes you get command-happy and the gcloud [[ delete ]] isn’t directed at the thing you intended.

Then you iz unhappy.

So, I use a minimal gcloud config. It’s the Scandinavian Design of config files:

gcloud config list
[core]
account = dazwilkin@google.com

This though requires me to be very explicit with my gcloud commands:

gcloud container clusters create ... --project=${PROJECT} ...

And — perhaps more importantly — very explicit with the bad guys:

gcloud container clusters delete ... --project=${PROJECT} ...

Yes, I can still get click-happy with --project=${PROJECT} but, one thing this approach further enables is that I maintain a local session state and so, as long as I don’t get too carried away with multiple shells, I’m generally able to know from which resources I’m operating on:

PROJECT=[[TODAY's PROJECT]]
REGION=[[ALWAYS-US-WEST1-COS-I'M-WEIRD-LIKE-THAT]]
ZONE=${REGION}-c

And, with thanks to Preston who first showed me this trick, you can bung that kind of stuff in e.g. env.sh and then, when you get to the project directory, you can source ./env.sh with impunity.

So, yes, more typing but more control and confidence too.

If you have a murky gcloud config list, you can gcloud config unset [section]/[property] or, if you’re confident, edit the file directly. It’s likely ~/.config/gcloud/configurations/config_default so perhaps:

CONFIG="${HOME}/.config/gcloud/configurations/config_default"
cp ${CONFIG} ${CONFIG}.$(date '+%y%m%d') && \
nano ${CONFIG}

kubectl

TIL that you can generally (!) achieve a similar result with kubectl.

A caveat is that gcloud container clusters mutates kubectl config when you create and delete clusters. This will reset your default cluster. You may unset your default cluster with:

kubectl config unset current-context

And then everything will break goodly ;-)

kubectl get nodes
The connection to the server localhost:8080 was refused - did you specify the right host or port?

This is functionally equivalent to unsetting projects and regions etc. in gcloud. Now, we can explicitly reference a context (and thus a cluster) each time we use kubectl. For example:

kubectl get nodes --context=[[YOUR-CONTEXT]]

But how to know which contexts are available?

kubectl config get-contexts
kubectl config get-contexts
CURRENT   NAME          CLUSTER        AUTHINFO    NAMESPACE
          black         black-180612   dazwilkin   
          white         white-180612   dazwilkin

NB Your context and clusters names — particularly if they’re uniquely generated by Kubernetes Engine — will not be this pretty.

Have no fear, because we can:

kubectl config rename-context white dev
kubectl config get-context
kubectl config get-contexts
CURRENT   NAME          CLUSTER        AUTHINFO    NAMESPACE
          black         black-180612   dazwilkin   
          dev           white-180612   dazwilkin

and then, of course, you get to:

kubectl delete namespace/next-game --context=dev

That’s more reassuring, isn’t it?

Yes, it’s more typing but — once again — we get to use our shell environment to help us:

CONTEXT=dev
kubectl get pods --context=${CONTEXT}

If I save one person one time that horrible, sinking feeling of deleting the wrong thing, I’ll be happy!

Update 180614: rename|delete

I’ve been pedantically (often just deleting entirely but usually just) editing ~/.kube/config to keep it tidy. There’s a simpler way (that’s better yet using kubectl autocompletion, see below).

Suppose you have a bunch of Kubernetes Engine generated unique albeit lengthy contexts:

kubectl config get-contexts
CURRENT   NAME                                     CLUSTER ...
          gke_[[PROJECT]]_[[REGION]]_[[NAME]]      ...
          gke_[[PROJECT]]_[[REGION]]_[[NAME]]      ...
          gke_[[PROJECT]]_[[REGION]]_[[NAME]]      ...

You can simply:

kubectl config rename-context gke_... [[ALIAS]]

And, if you’ve enabled autocompletion, you can simply start hitting TAB after ku to get kubectl, after conf to get kubectl config and after gke_ to start enumerating the gke-prefixed contexts. Once you have selected the desired context, you can rename it to a more useful alias.

NB Kubernetes Engine uses the same name for the context, the cluster and for the authinfo (often duplicative) section.

Similarly, if enumerating your contexts, lists orphans, you can tidy them up with kubectl delete-context. Pay attention doing this, of course, and don’t delete important contexts. In many cases, you can recover Kubernetes Engine contexts by rerunning gcloud container clusters get-credentials ....

Enabling Autocompletion

This is so good:

https://kubernetes.io/docs/tasks/tools/install-kubectl/#enabling-shell-autocompletion