TIQETS.ENGINEERING

Migrating to Helm 3

How Tiqets migrated from Helm 2 to Helm 3

What is Helm?

Helm is a tool to manage Kubernetes applications, like a package manager. Helm uses a packaging format called charts. Charts help define, install, and upgrade your Kubernetes resources.

What’s new?

• The tiller client is removed
• helm init and helm home are removed
• The -n flag no longer exists when using helm install
• Helm3 will create the namespace if you use the --create-namespace flag
• We can have the same release name on different namespaces.
• helm delete will always purge releases: the —-purge flag is removed. If we want to keep release history, we need to use --keep-history.
• The —-timeout flag now requires a unit of duration, for example, 300ms or 5m
• helm status no longer shows the status of the resources Helm creates.
• 3-way Strategic Merge Patches. Helm considers the old manifest, its live state, and the new manifest when generating a patch.
• requirements.yaml and requirements.lock is moved to Chart.yaml and Chart.lock

Read more here https://helm.sh/docs/topics/v2_v3_migration/

Installing Helm 3

To install the Helm 3 binary, fetch and execute the installation script.

$ curl -fsSL -o get_helm.sh https://raw.githubusercontent.com/helm/helm/master/scripts/get-helm-3
$ chmod 700 get_helm.sh
$ BINARY_NAME=helm3 ./get_helm.sh

Since we already have Helm 2 installed in our CI/CD pipeline, we wanted to download the latest Helm 3 binary and name it helm3. We did this by setting the BINARY_NAME before running the get_helm.sh script.

Migration tool

Once you have the Helm 3 binary installed, you can start using the Helm 2to3 plugin to migrate Helm 2 GitLab releases to Helm 3.

$ helm3 2to3
Migrate and Cleanup Helm v2 configuration and releases in-place to Helm v3

Usage:
  2to3 [command]

Available Commands:
  cleanup     cleanup Helm v2 configuration, release data and Tiller deployment
  convert     migrate Helm v2 release in-place to Helm v3
  help        Help about any command
  move        migrate Helm v2 configuration in-place to Helm v3

Flags:
  -h, --help   help for 2to3

Use "2to3 [command] --help" for more information about a command.

Basically, the plugin supports:

• Migration of Helm v2 configuration.
• Migration of Helm v2 releases.
• Clean up Helm v2 configuration, release data, and Tiller deployment.

Backup

Before migrating, it is essential to backup all your Helm 2 releases. Although the plugin should not delete your data, it’s better to be safe.

helm-backup

We used the helm plugin helm-backup to back up our releases. To install it, run:

$ helm plugin install https://github.com/maorfr/helm-backup

You can backup all your releases by running this command:

$ helm backup [flags] <NAMESPACE>

And to restore your releases:

$ helm backup [flags] <NAMESPACE> --restore

kubectl

You could also simply backup all your Secrets or ConfigMaps since Helm stores the release information there.

The easiest way to do it is to run the following command:

$ kubectl get <configmaps/secrets> \
    --namespace kube-system \
    --selector "OWNER=TILLER" \
    --output "yaml" > helm2-backup.yaml

Your choice between configmaps or secrets depends on your Helm Tiller storage backend.

Migration steps

Migrate the Helm configuration

We need to migrate the Helm 2 config and data folders. The command will create the Helm 3 config and data folders if they don’t exist. If you already have a repositories.yaml file, it will be overwritten.

Always use the --dry-run flag to verify what actions will be carried out.

$ helm3 2to3 move config --dry-run

List all releases

Then list all the Helm 2 releases that are available to be converted

$ helm list

Convert the releases from Helm 2 to Helm 3

On each release you want to convert, run the helm 2to3 convert command.

Always use the --dry-run flag to verify what actions will be carried out.

$ helm3 2to3 convert <RELEASE-NAME>

Note: Releases are namespace-scoped in Helm 3. To list the releases, you always need to specify the namespace.

Cleanup the Helm 2 releases and Tiller

After successfully converting your Helm releases, it is time to clean up the Helm 2 resources.

Always use the --dry-run command first to check the result.

$ helm3 2to3 cleanup

This will delete the following:

• Configuration (Helm home directory)
• v2 release data
• Tiller deployments

Issues

“UPGRADE FAILED: cannot patch.”

After the migration, some helm upgrades may fail with an error similar to the following:

Error: UPGRADE FAILED: cannot patch "..." with kind Deployment: Deployment.apps "..." is invalid: spec.selector:

or

Error: UPGRADE FAILED: cannot patch "..." with kind StatefulSet: StatefulSet.apps "..." is invalid:

These are known issues on Cert-Manager and Redis chart. This happens when the labels on some Deployments and StatefulSets are immutable and can not be changed from Tiller (set by Helm 2) to Helm (set by Helm 3). To solve these issues, force-replace the labels by hand. (See the issue links in this paragraph for more details.)

Author

Raditya Surya is an Infrastructure Engineer at Tiqets, and also tries to catch up with the latest technology available.