Using Concourse CI/CD to publish Helm charts to ChartMuseum and report results to Slack (part #1)
This guide will walk you through the process of creating Concourse CI/CD pipeline to build and publish Helm charts to ChartMuseum. And then extending it further to include things like Slack notifications.
The post assumes you already familiar with Kubernetes, Helm, ChartMuseum, Concourse and want to build some automation on top.
We will show the resulting Concourse CI/CD pipeline that will continuously build and publish Helm chart on changes on git, which:
- will be fully declarative and concise (under 40 lines of YAML)
- will integrate with Git, ChartMuseum and Slack
- will leverage Concourse resource abstractions and external tasks
- will have no complex bash scripts
Taking Helm Chart for Consul
As an example, let’s try to build a basic CI/CD pipeline which takes an official HashiCorp Helm chart for Consul (https://github.com/hashicorp/consul-helm) and publishes it into a local ChartMuseum repository on every commit.
Why did we pick this use case?
Basically, it’s one of the things we had to do internally since HashiCorp does not host their own chart repository. We needed a packaged Consul chart in ChartMuseum, so it could be installed from a Helm chart repo instead of from a chart source in git.
Building a basic CI/CD Pipeline
As a first step, let’s define Concourse resources for our pipeline:
- consul — for retrieving upstream consul-helm GitHub repository https://github.com/hashicorp/consul-helm
- chartmuseum — for publishing charts into ChartMuseum using https://github.com/cathive/concourse-chartmuseum-resource
After that, let’s define a single job publish-upstream-chart-into-chartmuseum, which uses these two resources to get the contents of a git repo with Consul chart, and then build/publish the assembled chart into ChartMuseum.
You can view the full pipeline at https://github.com/Aptomi/concourse-pipelines/blob/master/consul-chart-blog/01_simple.yaml
Running the pipeline
You can use code from our GitHub repository to load this CI/CD pipeline into Concourse:
$ cd concourse-pipelines/consul-chart-blog
$ fly -t target set-pipeline -c 01_simple.yaml -p publish-consul-chart -l params.yaml
$ fly -t target unpause-pipeline -p publish-consul-chart
When Concourse runs it, the UI will show:
Basically, it just grabbed chart source from git, built the Helm chart for Consul, and published it into ChartMuseum. From now on, the pipeline will ensure this happens on every git commit.
Pretty cool for such a short pipeline.
You can also verify that the chart is now in ChartMuseum:
$ curl https://your-chart-museum.company.com/index.yaml apiVersion: v1
- apiVersion: v1
description: Install and configure Consul on Kubernetes.
Adding Slack notifications
With Slack webhooks and Concourse Slack Resource (https://github.com/cloudfoundry-community/slack-notification-resource), it’s pretty straightforward to add notifications.
Configure Slack resource with your Slack Webhook URL:
Then use this resource to send basic notifications from the job:
You can also import it from our GitHub repo:
$ fly -t target set-pipeline -c 02_simple_with_slack_notifications.yaml -p publish-consul-chart -l params.yaml
And voila, you can run the pipeline again and see the result in Slack:
- The pipeline is just 40 lines, and it already can get data from git, publish charts, and send notifications to Slack
- It’s fully declarative, relies on Concourse resources and doesn’t contain any custom scripts
Making it generic
After replacing names and URLs with Concourse pipeline parameters, the pipeline can be applied to pretty much any Helm chart — just pass the right ((chart-name)) and ((chart-git-repo)) when you create it.
And don’t forget to store ((slack-webhook)) in Vault, or any other credential manager supported by Concourse. That’s how we run it in production.
The full pipeline can be viewed at:
Even though this example is pretty simple, you can see that Concourse gives you very powerful abstractions to build CI/CD pipelines. If you are switching to GitOps and declarative approach for continuous delivery of your apps and infrastructure, you should definitely take a closer look at Concourse.
Also, keep an eye for part #2, where we will take another iteration on the same pipeline and show how to:
- leverage external tasks in Concourse
- use task variables and perform local testing of tasks (new feature in Concourse 5.0)
If you have any questions, please don’t hesitate to comment below or reach out to me directly via firstname.lastname@example.org.