Ziv Rechnitser
Nov 13, 2018 · 2 min read

We’re using Helm to deploy services to K8s for several clients we’re supporting. Helm gives you the option to install and upgrade releases in your K8s clusters. As part of this process, the ability to know if the release has ended successfully or not is very important (whether to rollback or not). Helm does this by utilizing an ability to read the deployment status of k8s resources. K8s provides Helm several “metrics” about its health, for example: liveness of the Pod, service’s lack of an endpoint, Pod restart and more. Helm provides the option to add a “wait” and “timeout” flags to install and upgrade commands.

From Helm formal documentation:

https://docs.helm.sh/using_helm/#helpful-options-for-install-upgrade-rollback

Wait: if set, will wait until all Pods, PVCs, Services, and minimum number of Pods of a Deployment are in a ready state before marking the release as successful. It will wait for as long as — timeout.

Timeout: time in seconds to wait for any individual Kubernetes operation (like Jobs for hooks) (default 300).

Helm is a great tool with many pros BUT unfortunately, we found a major issue with the Wait flag; The Wait flag is used to let Helm server (Tiller) know that we want to wait for the command to finish to understand the status, the issue is the Wait flag supports only K8s resources under the V1 API. So if you use K8s resources in other API namespaces the Wait flag will be usable.

Here you can find the official issue in Helm Github repository:
https://github.com/helm/helm/issues/3173

“wait flag does not wait for deployment pod readiness properly”

And another comment from Helm tests:
https://github.com/helm/chart-testing/blob/4960e30445ead42157850a75245a6c543460ddee/lib/chartlib.sh#L326

“For deployments — wait may not be sufficient because it looks at ‘maxUnavailable’ which is 0 by default”

We use a workaround to solve this issue, for now, the Kubectl command:

kubectl rollout status — watch deployment/SERVICE_NAME — namespace=NAMESPACE

This command uses Kubectl to get the health of the deployment and then we can check the status of the deployment act depending on the result.

My name is Ziv, and I am an engineer at ProdOps — a global consultancy that delivers software in a Reliable, Secure and Simple way by adopting the Devops culture. Let me know your thoughts in the comments below, or connect with me directly on Twitter @zivtalal. Clap if you liked it, it helps me focus my future writings.


Originally published at www.prodops.io.

ProdOpsIO

Fusion of Engineering and Operations

Ziv Rechnitser

Written by

ProdOpsIO

ProdOpsIO

Fusion of Engineering and Operations

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade