How we configured our manifests for k8s micro service pipelines in Harness CD 2.0
This blog gives an overview of how the Harness Ops team migrated pipelines in the context of managing k8s manifests from Harness CD 1.0 to CD 2.0. We will also discuss how we re-used (templatized) microservices for different environments( uat, production, etc.).
We will start with an overview of how the configuration of k8s microservices in Harness UI in CD 1.0 and how we manage in CD 2.0. The below diagram gives the sequence of applying the k8s manifests in both versions. In CD 1.0, the configurations are all managed through Harness UI (internally, this is backed in Git); in CD 2.0, we are maintaining the complete configuration in Git repo, and runtime parameters(like environment, etc.) are derived from Input sets(CD 2.0 feature).
We have organized k8s manifests of our services in git by two categories Service[name] and Service Group[name]. The rationale of this grouping was to avoid the repetition of a similar configuration.
Service[name]: A standalone/independent service that doesn’t share or use configurations with other services.
Service Group[name]: Collection of more than one microservice which can be clubbed together due to some of the common characteristics(e.g., passwords, URLs, data stores, etc.).
The below diagram gives an overview of the structure. This includes relevant details along with the sequence of application of values.yaml for both categories mentioned above.
Below is an example where values.yaml is used for the category service-group and covers the category service-name. Here we have the Harness variables, literals, envSpecific tags that point the keys in the environment folder under a specific envType(uat, prod) and serviceGlobal tag, which refers to values.yaml immediately under service-group.The purpose of using these tags rather than setting default values is to avoid unexpected consequences if one forgets to override any environment-specific variable.
Mapping to Service & Service group in context of Inputset from where we derive the runtime parameters in our case envType(uat, prod etc) and availabilityType(primary[Active] & failover[Passive]) in environments folder (in above Diagram 2: Organization of CD 2.0 manifests in git repo).
Based on the pattern of manifests, we were able to use/templatize a pipeline (containing one or more microservices) for different environments in conjunction with inputset, as shown below.
We can replicate locally application of values.yaml at various override levels on the templates of a microservice before we deploy( except the secrets) using the go-template executable mentioned in the article Harness Local Go-Templating. Below is the syntax in our use case.
Standalone service-name
go-template -t <path git-repo dir>/service-name/templates/<template-name> -f <path git-repo dir>/service-name/values.yaml -f <path git-repo dir>/environments/<envType>/all-services.yaml -f <path git-repo dir>/<envType>/service-name/values.yaml -f <path git-repo dir>/<envType>/service-name/<availabilityType>/values.yaml
Service-group
go-template -t <path git-repo dir>/service-group/service-name/templates/<template-name> -f <path git-repo dir>/service-group/service-name/values.yaml -f <path git-repo dir>/service-group/values.yaml -f <path git-repo dir>/environments/<envType>/all-services.yaml -f <path git-repo dir>/environments/<envType>/service-group/values.yaml -f <path git-repo dir>/environments/<envType>/service-group/service-name/values.yaml -f <path git-repo dir>/environments/<envType>/service-group/service-name/<availabilityType>/values.yaml
Here envType = uat, prod etc & availabilityType = primary & failover.
In conclusion, the arrangement of manifests may differ from customer to customer, and this blog was an attempt to present our internal use case in Harness.
Thanks for reading!
References:
