Running Octant as a Container on vSphere With Kubernetes

David Adams
Jul 14, 2020 · 7 min read

Disclaimer: Everything in this post is taken from my initial investigation and proof of concept work in my lab. I think it may be helpful for others but know that this is not intended to be a production-ready architecture.

I’ve seen some cool Tweets about Octant for visualizing Kubernetes workloads recently and had to check it out for myself for the Kubernetes clusters I have on my Dell Technologies Cloud Platform (DTCP) lab which is comprised of VMware Cloud Foundation (VCF) 4.0 on VxRail HCI. Thanks to Vineeth and Ben for getting my attention!

While I’m probably more at home with kubectl, there are a lot of cool benefits to visualizing your Kubernetes objects, especially when it comes to debugging. So I jumped in to see what Octant was was all about. As I looked at the installation steps, which are all client side, my first reaction was that it seemed kind of painful to access it from anything other than localhost. After I thought about it a bit though, as a developer tool driven by your current config context, it made sense to run it locally most of the time. Still, I wanted to come up with an easier way to make it available outside of a single machine since I often times do demos for people or need to provide them easy access.

I started by trying to just run the binary on my Ubuntu machine with OCTANT_LISTENER_ADDR specified to allow connecting outside of localhost:

$ OCTANT_LISTENER_ADDR=0.0.0.0:80 ./octant --kubeconfig ~/.kube/config --disable-open-browser

But I kept getting this permission error:

unable to start runner: use OCTANT_LISTENER_ADDR to set host:port: failed to create net listener: listen tcp 0.0.0.0:80: bind: permission denied

I tried running it as root but got other errors and quickly got frustrated with the usual “my system is slightly different than what they have in the documentation or any tutorials I can find” problem. A more savvy Linux user could probably figure all that out, but this was when I decided to just build a container and run it that way because it always seems to “just work” for me.

Build an Octant Container

The first step was to build an Octant container because I couldn’t find one available anywhere. I grabbed the latest pre-built Linux binary, octant_0.13.1_Linux-64bit.tar.gz, and stuck it in the same directory where my Dockerfile would reside. The Dockerfile can be very short since I’m just sticking the binary in there and running it.

FROM alpine:3.12
COPY . /.
WORKDIR /.
EXPOSE 80
CMD OCTANT_LISTENER_ADDR=0.0.0.0:80 ./octant --kubeconfig ./.kube/config --disable-open-browser

It’s important to note that I use the —-kubeconfig flag to tell Octant where to look for the kubeconfig file. To test it out I built the config file (which I got from ~/.kube/config on my local machine) into the container image. This config file is populated with the connection information for all the Kubernetes clusters you have access to, in my case the vSphere Supervisor cluster which I connected to with the downloaded vSphere Kubectl binary (more on this later):

$ kubectl vsphere login --server=100.80.26.1 --insecure-skip-tls-verify

Later in the post I’ll show you how I externalized it with a ConfigMap in Kubernetes so I don’t have to rebuild the image every time I changed my kubectl configuration (i.e. connecting to a new cluster). Here’s what my directory tree looks like right now:

$ tree -a
.
├── Dockerfile
├── .kube
│ └── config
└── octant
1 directory, 3 files

Next, I build my container locally and test it out to make sure I can run it as a single container from my local machine.

$ sudo docker build -t davidadams3/octant:0.1 .
$ sudo docker run --name octant -d -p 80:80 davidadams3/octant:0.1

Now, putting in the IP or hostname of my local machine (not localhost) allows me to access Octant from anywhere that has a route to this machine! This is a viable solution as is if I just want an easy way to access Octant from somewhere other than localhost so feel free to stop here if that makes your life easier. Otherwise, you can keep reading to see how I ran my new container on Kubernetes and externalized the Kubeconfig file.

Run it on Kubernetes

As previously mentioned, I’m doing this on my VCF 4.0 setup so I am able to leverage all the goodness it offers for building and operating your private or hybrid cloud. If you’re not familiar with VCF and VxRail you can check it out here. For an introduction to all the new Kubernetes capabilities VCF 4.0 brings I’d recommend Cormac Hogan’s blog. It’s worth the time to get familiar with it all the new capabilities, but I won’t rehash that here since Cormac does a great job with that. I’m picking up here with vSphere for Kubernetes already enabled, a namespace created, a TKG cluster created and the Kubectl + vSphere plugin CLI.

Previously I talked about moving the kubeconfig file out of the image because I knew I could use a Kubernetes ConfigMap to mount it to the container and make it much more flexible. So the first step was to remove the file from the directory tree where I built my container image from and rebuild the container then push it up to my preferred Docker registry.

After that’s done we need to create our ConfigMap object now. This file is automatically created in a hidden file in your home directory when kubectl is connected to your cluster(s) and that’s how Octant is able to get your cluster info. So if you haven’t already you need to login to your Supervisor cluster and/or your TKG cluster depending on which one you want visualize in Octant. You’ll be prompted for credentials to connect. For the rest of the demo I’m going to stay logged into the Supervisor cluster since that’s where I plan to deploy Octant as a vSphere pod.

$ kubectl vsphere login --server=100.80.26.1 --insecure-skip-tls-verify
$ kubectl vsphere login --server=100.80.26.1 --insecure-skip-tls-verify --tanzu-kubernetes-cluster-namespace=test-01 --tanzu-kubernetes-cluster-name=tkg-cl-01

Now your ~/.kube/config file has the connection information required and can be the source for your ConfigMap

$ kubectl create configmap octant-config --from-file ~/.kube/config

Now we are able to create the Kubernetes deployment and service to expose it externally. The service is type LoadBalancer which is possible because vSphere with Kubernetes configures everything required with NSX-T during the deployment including the pool of ingress IPs I provided. This is a really nice feature which delivers the same type of experience you’d get from a public cloud managed Kubernetes experience (GKE, AKS, EKS, etc.).

apiVersion: v1
kind: Service
metadata:
name: octant-service
labels:
run: octant
spec:
type: LoadBalancer
ports:
- port: 80
targetPort: 80
protocol: TCP
name: http
selector:
run: octant

The deployment object is also quite straight forward but note that we have to create a volume for the ConfigMap and mount it into the directory the Octant container is expecting it to be (from the Dockerfile above it’s /.kube/config).

apiVersion: apps/v1
kind: Deployment
metadata:
name: octant-deployment
spec:
selector:
matchLabels:
run: octant
replicas: 1
template:
metadata:
labels:
run : octant
spec:
containers:
- name: octant
image: docker.io/davidadams3/octant:0.1
imagePullPolicy: Always
volumeMounts:
- name: config
mountPath: /.kube
ports:
- containerPort: 80
volumes:
- name: config
configMap:
name: octant-config

I put this all together in a single octant-deployment.yaml file:

apiVersion: v1
kind: Service
metadata:
name: octant-service
labels:
run: octant
spec:
type: LoadBalancer
ports:
- port: 80
targetPort: 80
protocol: TCP
name: http
selector:
run: octant
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: octant-deployment
spec:
selector:
matchLabels:
run: octant
replicas: 1
template:
metadata:
labels:
run : octant
spec:
containers:
- name: octant
image: docker.io/davidadams3/octant:0.1
imagePullPolicy: Always
volumeMounts:
- name: config
mountPath: /.kube
ports:
- containerPort: 80
volumes:
- name: config
configMap:
name: octant-config

Apply the yaml with Kubectl.

$ kubectl apply -f octant-deployment.yaml
service/octant-service created
deployment.apps/octant-deployment created

Checking the deployment I can see the Octant pod is ready and available.

$ kubectl get deployments
NAME READY UP-TO-DATE AVAILABLE AGE
kuard 1/1 1 1 19h
octant-deployment 1/1 1 1 33m

Now I just need to check for the automatically assigned EXTERNAL-IP for the service, in this case 100.80.26.3 and it can be accessed on port 80.

$ kubectl get services
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kuard-service LoadBalancer 10.96.0.29 100.80.26.2 80:32681/TCP 18h
octant-service LoadBalancer 10.96.0.203 100.80.26.3 80:31177/TCP 40s
tkg-cl-01-control-plane-service LoadBalancer 10.96.1.96 100.80.26.4 6443:31498/TCP 140m

We now have Octant running in vSphere Pod on the Supervisor cluster!

Thanks for reading this far and I’d like to re-iterate what I mentioned in the disclaimer at the beginning: I’m not convinced (yet) that this is the best way to deploy Octant but it was a fun exercise to work through so I thought I would share it. The obvious concern is around security and sharing access to your Kubernetes clusters without any sort of login screen (since it leverages the kubeconfig file for access to your clusters). I’ll do a little more tinkering to see if there are more features in Octant I’m not taking advantage of in this regard.

The Startup

Get smarter at building your thing. Join The Startup’s +785K followers.

Sign up for Top 10 Stories

By The Startup

Get smarter at building your thing. Subscribe to receive The Startup's top 10 most read stories — delivered straight into your inbox, once a week. Take a look.

By signing up, you will create a Medium account if you don’t already have one. Review our Privacy Policy for more information about our privacy practices.

Check your inbox
Medium sent you an email at to complete your subscription.

David Adams

Written by

Christ follower, husband, father, @NUTrackandField All-American, sub-4 miler, Tech Marketing for @DellTechCloud. Views are my own. #iwork4Dell

The Startup

Get smarter at building your thing. Follow to join The Startup’s +8 million monthly readers & +785K followers.

David Adams

Written by

Christ follower, husband, father, @NUTrackandField All-American, sub-4 miler, Tech Marketing for @DellTechCloud. Views are my own. #iwork4Dell

The Startup

Get smarter at building your thing. Follow to join The Startup’s +8 million monthly readers & +785K followers.

Medium is an open platform where 170 million readers come to find insightful and dynamic thinking. Here, expert and undiscovered voices alike dive into the heart of any topic and bring new ideas to the surface. Learn more

Follow the writers, publications, and topics that matter to you, and you’ll see them on your homepage and in your inbox. Explore

If you have a story to tell, knowledge to share, or a perspective to offer — welcome home. It’s easy and free to post your thinking on any topic. Write on Medium

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store