Diving Deeper into Helmfile

Exploring advanced concepts with this tool

Photo by Annie Spratt on Unsplash

Intro

When we have to manage Kubernetes, handling different environments and maintaining consistency throughout the software lifecycle can become very complex.

The most primitive approach is to use raw YAMLs and deploy our resources directly. Helm, through the Go templating engine, greatly enhances deployment capabilities, making it the standard for deploying to Kubernetes today. It offers excellent features such as version management and maintaining the state of deployed resources.

But what happens if we need to deploy the same charts many times, in different environments, on different clusters, and in different namespaces? We end up repeating a lot of code, and our changes become tedious.

We want to maintain balance in these trade-offs, using DRY without making it difficult to understand.

Helmfile is a powerful tool for addressing these challenges, allowing for an organized and scalable approach to managing Helm charts.

The idea is to delve into some advanced concepts of Helmfile, focusing on environment management, repository management, deployment strategies, and the use of state values and layers to optimize your deployment workflows in Kubernetes.

Setup

Install Helm

Install helmfile

Install this plugins:

helm plugin install https://github.com/aslafy-z/helm-git --version 1.3.0

helm plugin install https://github.com/databus23/helm-diff

Alternatively you can run helmfile init to install dependencies.

Environment binding

This feature allows for customizations tailored for different environments. We can change configurations seamlessly, such as using different Kubernetes contexts for development, testing, and production.

environments:
  dev:
    kubeContext: arn:aws:eks:us-east-1:111111111111:cluster/dev
  test:
    kubeContext: arn:aws:eks:us-east-1:222222222222:cluster/test
  prod:
    kubeContext: arn:aws:eks:us-east-1:333333333333:cluster/prod

This is a nice safeguard that avoid to making damage in the wrong context.

State Values

Values that are not sensitive, which are environment specific can be defined in a very straightforward way. These are Helmfile values and slightly differ from Helm values as they are rendered during Helmfile rendering, prior to the chart rendering.

Let’s start with a simple example:

# state-values/prod-state-values.yaml
METRICS_SERVER_CHART_VERSION: 3.11.0
---
# state-values/dev-state-values.yaml
METRICS_SERVER_CHART_VERSION: 3.12.0
---
# state-values/test-state-values.yaml
METRICS_SERVER_CHART_VERSION: 3.12.0
---
# helmfile.d/helmfile.yaml.gotmpl
environments:
  dev:
    kubeContext: arn:aws:eks:us-east-1:111111111111:cluster/dev
    values:
      - ../state-values/dev-state-values.yaml
  test:
    kubeContext: arn:aws:eks:us-east-1:222222222222:cluster/test
    values:  
      - ../state-values/test-state-values.yaml
  prod:
    kubeContext: arn:aws:eks:us-east-1:333333333333:cluster/prod
    values:  
      - ../state-values/prod-state-values.yaml
---
repositories:
  - name: metrics-server
    url: https://kubernetes-sigs.github.io/metrics-server/ 
releases:
    name: metrics-server
    namespace: kube-system
    version: {{ .StateValues.METRICS_SERVER_CHART_VERSION }}
    chart: metrics-server/metrics-server

Also you can use the environment name to dynamically reference helm values files:

# values/prod/metrics-server.yaml
podDisruptionBudget:
  enabled: true 
  maxUnavailable: 50%
---
# values/test/metrics-server.yaml
podDisruptionBudget:
  enabled: false
---
# values/dev/metrics-server.yaml
podDisruptionBudget:
  enabled: false
---
# helmfile.d/helmfile.yaml.gotmpl
environments:
  dev:
    kubeContext: arn:aws:eks:us-east-1:111111111111:cluster/dev
    values:
      - ../state-values/dev-state-values.yaml
  test:
    kubeContext: arn:aws:eks:us-east-1:222222222222:cluster/test
    values:  
      - ../state-values/test-state-values.yaml
  prod:
    kubeContext: arn:aws:eks:us-east-1:333333333333:cluster/prod
    values:  
      - ../state-values/prod-state-values.yaml
---
repositories:
  - name: metrics-server
    url: https://kubernetes-sigs.github.io/metrics-server/ 
releases:
    name: metrics-server
    namespace: kube-system
    version: {{ .StateValues.METRICS_SERVER_CHART_VERSION }}
    chart: metrics-server/metrics-server
    values:
      - ../values/{{ .Environment.Name }}/metrics-server.yaml

To run this, you will need to add — environment flag to helmfile command:

helmfile diff --environment dev

Layering

It is a technique that allows you to separate and organize your configurations, adhering to the DRY (Don’t Repeat Yourself) principle, which makes your Helmfile definitions more manageable and easier to handle.

# helmfile.d/environments.yaml
environments:
  dev:
    kubeContext: arn:aws:eks:us-east-1:111111111111:cluster/dev
    values:
      - ../state-values/dev-state-values.yaml
  test:
    kubeContext: arn:aws:eks:us-east-1:222222222222:cluster/test
    values:  
      - ../state-values/test-state-values.yaml
  prod:
    kubeContext: arn:aws:eks:us-east-1:333333333333:cluster/prod
    values:  
      - ../state-values/prod-state-values.yaml
---
# helmfile.d/kube-system-resources.yaml
bases:
  - environments.yaml
repositories:
  - name: metrics-server
    url: https://kubernetes-sigs.github.io/metrics-server/ 
releases:
    name: metrics-server
    namespace: kube-system
    version: {{ .StateValues.METRICS_SERVER_CHART_VERSION }}
    chart: metrics-server/metrics-server
    values:
      - ../values/{{ .Environment.Name }}/metrics-server.yaml

# helmfile.d/monitoring-resources.yaml
bases:
  - environments.yaml
repositories:
 - name: prometheus-community
   url: https://prometheus-community.github.io/helm-charts
releases:
 - name: kube-prometheus-stack
   namespace: monitoring
   chart: prometheus-community/kube-prometheus-stack
   version: {{ .StateValues.KUBE_PROM_STACK_CHART_VERSION }}

Nested State

This feature allows to create module-like recipes for helmfile, this way you can easily deploy releases referencing a git repo and a tag:

# helmfile.d/monitoring-resources.yaml
bases:
  - environments.yaml
---
helmfiles:
  - path: git::https://github.com/myorg/devops-tools.git@centralized-helmfile/helmfile.d/prometheus.yaml.gotmpl?ref=helmfile-0.0.1
    values:
      - ../state-values/{{ .Environment.Name }}-state-values.yaml
---
# https://github.com/myorg/devops-tools.git@centralized-helmfile/helmfile.d/prometheus.yaml.gotmpl?ref=helmfile-0.0.1
environments:
  dev:
    kubeContext: arn:aws:eks:us-east-1:111111111111:cluster/dev
  test:
    kubeContext: arn:aws:eks:us-east-1:222222222222:cluster/test
  prod:
    kubeContext: arn:aws:eks:us-east-1:333333333333:cluster/prod
---
repositories:
 - name: prometheus-community
   url: https://prometheus-community.github.io/helm-charts
releases:
 - name: kube-prometheus-stack
   namespace: monitoring
   chart: prometheus-community/kube-prometheus-stack
   version: {{ .StateValues.KUBE_PROM_STACK_CHART_VERSION }}
# other releases here....

It is important to note that the kube context and state values are relative to the scope of the file being rendered. So, in this case, if you want the context to be explicitly selected in the nested state, it must be explicitly declared in the remote location. Also, note that the state values are passed as an argument to be merged.

State-Values and Helm Values

Don’t expect the same behavior for both of these values. When you pass a state value as described above, it is a general value that won’t be directly linked to any of your release values. If you want to bind them, you should do this explicitly:

# helmfile.yaml
helmfiles:
- path: git::https://github.com/myorg/helmfiles.git@subhelmfile.yaml?ref=0.1.0
  values:
  - additional.values.yaml
---
# additional.values.yaml
port: 2
---
# git::https://github.com/myorg/helmfiles.git@subhelmfile.yaml?ref=0.1.0
releases:
- name: test
  namespace: prometheus
  chart: ./test 
  values:
    - subvalue.yaml.gotmpl 
---
# git::https://github.com/myorg/helmfiles.git@subvalue.yaml.gotmpl?ref=0.1.0
port: {{ .Values.port }}

It is not enough to have the same path .Values.port, because they are different things. To avoid ambiguity, I prefer using the alias StateValues.

# helmfile.yaml
helmfiles:
- path: git::https://github.com/myorg/helmfiles.git@subhelmfile.yaml?ref=0.1.0
  values:
  - additional.values.yaml
---
# additional.values.yaml
port: 2
---
# git::https://github.com/myorg/helmfiles.git@subhelmfile.yaml?ref=0.1.0
releases:
- name: test
  namespace: prometheus
  chart: ./test 
  values:
    - subvalue.yaml.gotmpl 
---
# git::https://github.com/myorg/helmfiles.git@subvalue.yaml.gotmpl?ref=0.1.0
port: {{ .StateValues.port }}

Bash commands

This is powerfull, you can run a command inside a template block:

The exec function allows you to run a command, returning the stdout of the command. When the command fails, the template rendering will fail with an error message.

I have used this to login to the OCI AWS registry:

repositories:
  - name: awskarpenter
    url:  public.ecr.aws
    oci: true
    username: AWS
    password: {{ exec "aws" (list "ecr-public" "get-login-password") }}

This execute aws cli to retrieve the docker token. This syntax is very powerful, the only limit is your imagination.

Conclusion

In conclusion, Helmfile provides an effective way to manage deployments in Kubernetes, facilitating the configuration and maintenance of multiple environments. The features we reviewed simplify configuration management and enable more flexible and organized deployments, fully aligned with modern DevOps practices.

Resources & References

• Readme https://helmfile.readthedocs.io/en/latest/
• Best Practices: https://helmfile.readthedocs.io/en/latest/writing-helmfile/
• Tool Repo: https://github.com/helmfile/helmfile