Kubernetes goodies for your local workspace

I’d like to share my local setup for Kubernetes that helps me in being a bit more productive every day. It’s NOT a pro guide for infra engineers, but it should be enough for software/data engineers who are trying to get the most out of Kubernetes. We use this setup at kiwi.com.

If you have the same/similar setup as me, you can use the exact commands I’ll include in this guide. I guess a lot of you have some slight differences in computer setup (e.g. bash, running on Linux). I believe it should be almost the same in those cases (just use apt-get instead of brew). I’ll try to include bash setup when possible, and include links for the software used — so that you can follow the official guide for Linux. For Windows, I’m sorry.

This my local configuration:

• MacBook Pro, macOS High Sierra (with security updates)
• brew
• iTerm2
• zsh, oh-my-zsh, agnoster theme 😎

gcloud

brew cask install google-cloud-sdk

Autocompletion

bash

# ~/.bashrc
source '/usr/local/Caskroom/google-cloud-sdk/latest/google-cloud-sdk/path.bash.inc'
source '/usr/local/Caskroom/google-cloud-sdk/latest/google-cloud-sdk/completion.bash.inc'

zsh

# ~/.zshrc
source '/usr/local/Caskroom/google-cloud-sdk/latest/google-cloud-sdk/path.zsh.inc'
source '/usr/local/Caskroom/google-cloud-sdk/latest/google-cloud-sdk/completion.zsh.inc'

Tip: I have an “if” clause around it:

if [ -f '/Users/maroshmka/google-cloud-sdk/path.zsh.inc' ]; then . '/Users/maroshmka/google-cloud-sdk/path.zsh.inc'; fi
if [ -f '/Users/maroshmka/google-cloud-sdk/completion.zsh.inc' ]; then . '/Users/maroshmka/google-cloud-sdk/completion.zsh.inc'; fi

Then run:

gcloud init

You’ll need to do auth with Google account in the browser. Then, configure the project and set default zone. You can add the next projects later.

Official guide (also Linux, Windows) here.

NOTE: There is some initiative on interactive shell with gcloud, it looks cool but it's in beta now. I don’t use it, but if you feel experimental give it a try and let us know :) You can find it here.

kubectl

brew install kubectl

Autocompletion

# ~/.zshrc
source <(kubectl completion zsh)

Kube official guide for autocompletion

Tip: One of the most important things. Set an alias, be a pro.

alias k="kubectl"

Connect to a cluster in your GCP project

Which means, you need to be part of a GCP project and have a cluster created. It’s likely that you have — that if your company uses Kubernetes. If not, you can create your own for testing purposes, it should be pretty easy.

# check projects available
gcloud projects list
# set the project
gcloud config set project my-cool-gcp-project
# validate
gcloud info
# list clusters in projecs
gcloud container clusters list
# get credentials that are added to you kube config
gcloud container clusters get-credentials my-cool-cluster --zone europe-west1-c

validate that it works 😎

k get pods
k get namespace

kube-ps1

This is a nice thingy that adds info about what’s your current context.

You’ll see cluster:namespace at the start in your prompt

brew install kube-ps1
# add to zshrc or bashrc
source “/usr/local/opt/kube-ps1/share/kube-ps1.sh”
PS1=’$(kube_ps1)’$PS1

Pro Tip: Use kubeoff and kubeon. Add kubeoff to the end of the zshrc file. Then, by default, you’ll see nothing. When you need to work with kube, just gokubeon and info will spawn.

https://github.com/jonmosco/kube-ps1

kubectx & kubens

Switching clusters and namespaces is kind of a pain in kube cli, so this package is a must-have.

brew install kubectx

You can then easily switch between clusters and namespaces with kubens my-namespace or kubctx some-long-name-from-gcloud

Tip: Create an alias for cluster.

kubectx # press enter & choose the long name for the cluster
kubectx my-name=.
# then you can use. btw it has autocomplete :)
kubectx my-name

https://github.com/ahmetb/kubectx

kube-secrets

It’s kinda cumbersome to create secret in Kubernetes from a local computer. You can’t upload json of secrets, at least kubectl doesn’t support that or at least I don’t know about it 🤔 If you know how to do it let me know.

You can use the --from-file option, but you need to create small manifest and encode the secrets to base64, then push it. It's not much, but I just hated it, so I wrote this little package.

At Kiwi.com, we use Terraform and Vault. So, most of the time we sync our secrets in GCP from vault using Terraform. But, there are times when you just need to quickly upload some secrets that you are using for any dev/test/sandbox purposes.

pip3 install kube-secrets

Then it allows you to upload json only with secrets.

echo '{"a": "top-secret"}' > s.json
kube-secrets create --name my-secrets -n default --data-file=s.json

Pro Tip: Pull the json directly from vault.

vault read secret/my-app -format=json | jq '.data' | \ 
    kube-secrets create --name my-project-secrets -n my-namespace -

https://github.com/maroshmka/kube-secrets

Test everything

Let's deploy an example pod to validate if everything works as expected.

1. Create our secrets. (Optional, if you’ve installed kube-secrets)

echo '{"A": "top-secret"}' > s.json
kube-secrets create --name top-secret -n default --data-file=s.json

2. Create a deployment file.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-deployment-test
  labels:
    app: nginx
spec:
  replicas: 1
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: nginx
        # omit this if you skipped 1. step
        envFrom:
        - secretRef:
            name: top-secret

3. Apply.

kubeon  # switch cluster/namespace
kubectx my-cluster-name
kubens default
k apply -f deployment.yml

4. Validate and cleanup.

k get pods  # pod should be visible and running
k exec -it nginx<TAB><TAB> bash
echo $A  # you should see top-secrets
k delete deploy nginx<TAB><TAB>

Hope it helps!

I hope this quick guide helps you to start using Kubernetes from your computer more effectively.

Let's deploy all the stuff!

Note: if you wanna have cool terminal, here’s a quick — old, but still valid —guide from my colleague m1ňo.