Can Helm and Operators coexist
[You take the blue pill and the story ends. You wake in your bed and believe whatever you want to believe and continue with Helm charts. You take the red pill and you stay in Wonderland and I show you how deep the operator goes. ]
No, we are not talking about humans and robots, but Helm and Operators
I have worked with operators since the beginning of its birth. Started with Golang operators because I like Golang and then started slowly to spread my operators to Ansible, but paused when getting into Helm. At first, I couldn’t understand why these two technology coexists
And then I started receiving questions about Helm charts and operators within my teams and colleagues and most of the questions seemed to imply that these two are engaged in a “Mortal Kombat”. Some suggested we have Helm-based Operators with limited functionality and some suggested building tools to migrate Helm templates to pure operators without having to depend on a Helm template or to need to maintain them.
Helm: the package manager for Kubernetes
What Is Helm?
In Short, Helm is a tool for Kubernetes packages called charts.
Going back in time, Great Scott! It was precisely on Nov. 2, 2015, the first version of Helm charts was released at the same time when Kubernetes was at version 1.1.0 and was gearing up to their very first KubeCon that was about to take place in 2015.
The mission statement was
Helm provides package management for Kubernetes
From the beginning, Helm was designed to solve one big problem: How do we share reusable Kubernetes resources for installing (and upgrading and uninstalling) things on Kubernetes? And they did achieve it, many k8s users quickly adapted Helm charts today, Helm has over a million downloads a month, and there are many publicly available charts in the repo.
Enter The Operators — There is no opponent
Operators: a design pattern for operational knowledge
Almost exactly one year after Helm 0.0.1, Nov 13, 2016, to be precious, Brandon Philips, then CTO of CoreOS, posted on a blog post: Introducing Operators: Putting Operational Knowledge into Software.
Philips pointed out that we often relied upon humans to manage the runtime needs of applications. But with a system like Kubernetes, much of the material once set down in run books and user manuals could now be transformed into code.
A Kubernetes operator is a method of packaging, deploying, and managing a Kubernetes application
An Operator is an application-specific controller that extends the Kubernetes API to create, configure, and manage instances of complex stateful applications on behalf of a Kubernetes user.
It builds upon the basic Kubernetes resource and controller concepts but includes domain or application-specific knowledge to automate common tasks.
As time passed Operators mature and quickly adapt to vanilla Kubernetes and RedHat Openshift. Mainly because of the completion of CRD and maturity of Kubernetes API. Also, thanks to Kube-builder and Operator-SDK these two tools made it possible to write a complex operator very easily.
The dev-Ops and Support engineers celebrated the birth of operators, it was the tool to solve day to day operations and beyond. It’s not just the package manager for Kubernetes any more. The operator design pattern is here to stay
But then I have Helm charts, Why do I need operators?
Didn’t I say, There is no opponent?
What happens in the Helm template stays in the Helm templates.
Helm charts cannot have fully operational knowledge besides just specifying the Kubernetes resource and how they can be configured via defaults override.
I want Helm to run as operators, I know they are package managers.
The answers lie in Helm-based operators that were made possible by the operator SDK team.
Helm Operator enables pairing a Helm Chart with the operational knowledge of installing, upgrading, and managing the application on Kubernetes clusters.
There is a current version of Helm-based operators and the newer hybrid version of Helm-Ansible Operators. Both of the implementations are tightly coupled with Helm charts. You have to maintain Helm templates.
All your Kubernetes resources stay in Helm templates and In the later version, the operators convert Helm templates into ansible templates on the fly and allow to pass defaults via ansible default variables, but the user still has to maintain their Helm charts to define their resources.
This brings to the limitation of not able to expand Helm charts into full-blown operators. Cannot make beyond install and upgrade.
It was said that you would destroy the Helm, not join them, bring balance to the force, not leave it in darkness.
Are Helm and Operators competitors? do they hate each other?. Or is it merely because there are too many Helm charts. Like I said millions and they refuse to let it go and join the force… Or the Darkside… Depends on which side you are.
Then why do I hear people discuss like they are competitors to each other?.
I think the answer is because Helm and operators use some same terminology, install, upgrades, lifecycle, configure.
People can use helm charts to do their job to some degree if not fully like operators do, but then we have helm-based operators which are a wrapper around helm charts and watches for changes and it makes it easier for some users to stay with helm and continue to adopt the operator SDK path of creating a helm-based operator.
Helm? Where We’re Going, We Don’t Need Helm.
Helm is a package manager for Kubernetes, period. They are what they are defined to be. But Operators are designed pattern-driven code that encapsulates business logic for running your application. It knows not only to deploy the application but also configure, scale, backup, Monitor, etc
As a Helm chart user “I want to write some template and be able to install my resources without having to edit them but still able to pass some arguments and able to build a deployment logic”.
When a Helm user mentions managing applications they are thinking of having simple templates able to deploy and able to upgrade/rollback or delete the application.
For operators, it will be like “I want an operator to write, because I want to install, upgrade, maintain the state, scale, and monitor”.
Here it’s often detailed, I want day two operations, monitoring my application, rolling back from failure, automatic backup, keeping my application in a healthy state.
So we have two kinds of users, with different needs. But then the helm user needs able to sufficient in the initial stage, as the application matures Helm user might think of moving to full-blown operators and they have two choices, either use Helm-based operators offered by Operator-SDK which will take you two seamless upgrades but still not an operator with full capabilities.
“I am superior, sir, in many ways, but I would gladly give up to be Helm”. Its clear Operators by itself are superior, but not as a Helm-based operator, they won’t function in their fullest capacity, Instead, you rather stay as Helm charts.
Helm is merely templated YAML, which is Go Templates, decorated with Sprig functions. Operators are a bunch of CRDs and Controllers to manage application life cycles. But that doesn’t mean CRD’s and Controllers solve all problems. They are two different technologies made to fulfill two different purposes.
Who is the winner?
In the short term probably both. Operators have a different focus than the Helm charts. Helm fulfills the need for packaging and managing your applications.
Helm has its place, maybe not as Operators.
And in the long term as more and more advances happen in the operator’s world, and there are more and more resources available to support and develop in operators. As the operator framework matures, helm users might find it hard to stay as helm-based operators and too complex to migrate to Operators.
So if you need to take advantage of operators, then now is the time to make the move.
But for now, Helm has now graduated from incubation at the Cloud Native Computing Foundation as a fully-fledged CNCF project.
CNCF Landscape — Showing HELM graduated
Helm’s project status was boosted further by an independent security audit, completed in November 2019, that concluded Helm was “highly mature” software built with good security practices and could be recommended for public deployment.
Helm is here to stay! But Operators are superior!
Wait! , What if I want to migrate from Helm to Operators to take advantage of the full capabilities of the operators?.
There is no easy way to migrate Helm charts to Ansible Templates or Go structs to build Ansible Operators Or Go Operators.
Below is the tool Helm Ansible Template Exporter I worked on along with a colleague, a few months ago to help Helm users to transition into Ansible Operators.
Why ansible? Ansible uses Jinja2 Templates which is equivalent to Helm Go Templates. But there is a caveat, Helm Classic’s template tool includes an array of built-in functions And there are 40+ functions available. These functions are supplied by the built-in Go text/template package and the Sprig template function library. So the tool I developed will help you migrate most of the Helm templates to Jinja2 Templates but then there are some workarounds required which you will see as you read the tool documentation below. Overall you can achieve 60% of porting from Helm to Ansible operator and the rest requires manual intervention.
Helm Ansible Template Exporter
Helm Ansible Template Exporter is a set of tools aimed at helping K8S application developers transition from Helm to Ansible Role-Based Operators. This project is separated into two main tools:
Helm To Ansible Exporter
This tool automates conversion from Helm Chart to Ansible Playbook Role. Helm Charts are defined utilizing Go Template Language, and Ansible Playbook Roles are defined using Yaml and Jinja2 Template Language. Because Go Template Language and Jinja2 Template Language are completely separate languages, some facets of Helm charts are difficult or impossible to convert automatically. This tool performs a best attempt conversion of a Helm chart into an Ansible Playbook Role, but some aspects of the conversion process must be performed manually after utilizing this tool. Additionally, when ambiguities in conversion arise, this tool makes a best attempt to emit warnings where a manual conversion or inspection is needed after running the tool.
The output of this tool is an Ansible Role which can be installed using Ansible Playbook, or built into an operator using the Ansible Operator Builder.
Helm To Ansible Exporter Current Capabilities
The current offering does the following:
- Creates a role in the workspace directory using ansible-galaxy.
- Raw copies templates into the generated Ansible Playbook Role templates directory, renaming each template with a “.j2” extension.
- Merges values.yml (or values.yaml) into the generated Ansible Playbook Role defaults/main.yml file.
- Searches the generated Ansible Playbook Role’s defaults/main.yml file for self-references (i.e., references to .Values.) and comments them out. Ansible Playbook is incapable of expressing self-references in defaults/main.yml, a clear technology gap between Ansible Playbook and Helm charts. A “WARN” message is output indicating that a manual change is required to defaults/main.yml on the appropriate lines.
- Convert Branch syntax for if, range <list> and range <map> in each template to utilize proper Jinja2 syntax. This includes a heuristic which attempts to determine if conditionals are checking for definition v.s. boolean evaluation.
- Convert boolean composition ordering. Go Templating utilizes “and “ format. On the other hand, Jinja2 utilizes “ and “ formatting. helmExport handles this conversion automatically.
- Template functions invocations are converted to Jinja2 Ansible Filter invocations. This requires converting direct function invocations to piped invocations, as well as inserting the appropriate parentheses and commas for argument lists.
- Removes references to “.Values.” in the generated Ansible Playbook’s Roles’ templates. Ansible Playbook can directly reference the values in defaults/main.yml, so this reference isn’t needed.
- If the “generateFilters” flag is set to true, some stub Ansible Filter implementations are installed for use in converting Go template functions to Ansible Playbook Filters.
Helm To Ansible Exporter Known Limitations
Go/Sprig Template Function Limitations
Ansible Playbook Filter(s) is one candidate to replace Go/Sprig Template Functions. However, they are not a direct 1:1 mapping, so a few extra steps must be taken after conversion.
Implement or Replace Template Function Invocations with Ansible Filters
There are several Ansible Playbook Filters built directly into Ansible. For instances where there does not seem to be a replacement, you will need to create your replacement. If you utilize the “generateFilters” command-line argument, some example filters will be installed into your generated Ansible Playbook Role.
“template” and “include” are not supported
Go Templates provide “template” and “include” to support dynamic template creation. Ansible has no direct replacement, and although there are some similar constructs, helmExport currently doesn’t support any conversion. Instead, consider using Ansible defaults as a replacement.
Where We Go From Here Is A Choice I Leave To You.
Thank you
