Deploy Kubernetes add-ons based on cluster run-time state

Gianluca Mardente
3 min readDec 2, 2022

Kubernetes itself is not a complete solution. To build a production cluster, you need various additional addons. If you are managing multiple clusters, that is not an easy task.

Sveltos wants to figure out the best way to install, manage and deliver cluster addons. It does so by providing declarative APIs allowing to deploy Kubernetes addons across multiple Kubernetes clusters.

The main idea of Sveltos is to allow users to programmatically list which addons (Helm charts or kubernetes resources) need to be deployed where (where is expressed using a ClusterSelector which selects all ClusterAPI powered clusters with labels matching the selector).

Sveltos Kubernetes add-on controller in action.

Helm values can be passed directly. Or Sveltos can be instructed to fetch those from the management cluster. For instance:

Having worked in the past in a different startup doing policy distribution in a distributed system, I know visibility is important. So sveltos provides user with DryRun mode as well where user can see the effect of a change before committing it

./bin/sveltosctl show dryrun

+-------------------------------------+--------------------------+-----------+----------------+-----------+--------------------------------+------------------+
| CLUSTER | RESOURCE TYPE | NAMESPACE | NAME | ACTION | MESSAGE | CLUSTER PROFILE |
+-------------------------------------+--------------------------+-----------+----------------+-----------+--------------------------------+------------------+
| default/sveltos-management-workload | helm release | kyverno | kyverno-latest | Install | | dryrun |
| default/sveltos-management-workload | helm release | nginx | nginx-latest | Install | | dryrun |
| default/sveltos-management-workload | :Pod | default | nginx | No Action | Object already deployed. | dryrun |
| | | | | | And policy referenced by | |
| | | | | | ClusterProfile has not changed | |
| | | | | | since last deployment. | |
| default/sveltos-management-workload | kyverno.io:ClusterPolicy | | no-gateway | Create | | dryrun |
+-------------------------------------+--------------------------+-----------+----------------+-----------+--------------------------------+------------------+

To know more about dry-run see this section of Sveltos documentation.

So far, users still had to go and manage cluster labels. I wanted clusters’ labels to change as cluster runtime state was changing. So I can express my intent and then forget about it.

So a second concept was introduced in Sveltos: Classifier CRD which allows users to classify clusters based on cluster runtime state (currently Kubernetes version and/or resources deployed).

Doing so I can easily now say:

- any cluster running Kubernetes version v1.25.2 needs this version of calico as CNI

- as clusters are upgraded to Kubernetes version v1.25.2 the right labels are added on cluster so that right version of calico is deployed.

Sveltos Kubernetes cluster classification

Sveltos will soon support onboarding existing clusters and managing those same way ClusterAPI powered clusters are managed.

Documentation can be found here.

Contributing to Sveltos

🎉 We welcome and appreciate all contributions to Sveltos! Whether you want to report a bug, suggest a new feature, or simply ask a question, we’d love to hear from you. Here are a few ways to connect with us:

  1. Open a bug/feature enhancement on github
  2. Chat with us on the Slack in the #projectsveltos channel
  3. Contact Us

👏 Support this project

If you like the project, please give us a if you haven’t done so yet. Your support means a lot to us. Thank you 🙏.

--

--