Testing Kubernetes Operators using GitHub Actions and Kind
Automated testing is an important feature of any well managed repository. It provides validation for everyone to ensure code quality and functionality.
In this article, we’ll cover linting, unit testing, integration testing, and docker build. We’ll also cover a good strategy for managing secrets in Github.
Setup
In the root of your directory, create the following directory structure and files.
.github/
|- workflows/
|- linting.yaml
|- testing.yaml
|- docker-build.yaml
MakefileGitHub Actions works by reading the contents of the .github directory.
Linting
Code linting is static code analysis used to flag programming errors, bugs, stylistic errors and suspicious constructs.
Add the following in linting.yaml. This will run the workflow whenever you create or update a pull request. If it finds any linting errors, it will leave code comments on your PR for you to resolve.
Unit Testing
We’re going to implement unit testing and integration testing jobs into our next workflow. First up, is unit testing.
In testing.yaml, add the following. This will run the workflow whenever you create or update a pull request.
First, we install a specific version of Golang because we can’t be certain what version the default runner, ubuntu-latest, will be using. Finally, we run our unit tests using a Makefile. Here is an example of how the Makefile might look.
# Run tests
unit-test: fmt vet
go test ./pkg/... -coverprofile cover.out integration-test: fmt vet
go test ./controllers/... -coverprofile cover.out# Run go fmt against code
fmt:
go fmt ./... # Run go vet against code
vet:
go vet ./...
You can adjust the path go unit-test will target. In this example, it is looking for *_test.go files in the pkg directory. For extra validation. go fmt and go vet are also run.
Integration Testing
Next up is integration testing. It is very likely integration testing will require interacting with systems that require authentication. We’re going to add secrets as environment variables to satisfy this requirement.
Modify testing.yamlwith a new job at the bottom. Here is what the complete file will look like. Note the addition of integration-tests under jobs.
First, we start up a Kind cluster to run our operator against. We are testing against Kubernetes version 1.20.7. When this spins up, it will automatically set the kube config in the GitHub Action Runner that Operator-SDK will use to run.
Kind is a tool for running local Kubernetes clusters using Docker container “nodes”. Kind was primarily designed for testing Kubernetes itself, but may be used for local development or CI.
Next, we securely grab any secrets from Azure Key Vault our operator requires. You can read more about Azure/get-keyvault-secrets Github Action here. Later on, we’ll discuss why we are getting secrets this way.
Next up, we install the version of Operator-SDK we are using.
Finally, we run the integration tests by calling make integration-test
Docker Build
The final workflow we will create is to test that we can successfully build our operator’s container image. Add the following to docker-build.yaml .
Managing GitHub Secrets Efficiently
You can add secrets to Github and call them in Actions using ${{ secrets.MY_SECRET_NAME }} . This approach is fine and simple, but it becomes a headache when managing multiple secrets. If you have a service principal that needs it’s secret rotated regularly, you have to remember to change it everywhere including GitHub. Furthermore, the more locations you store secrets, the greater your attack surface for hackers.
However, if you dynamically retrieve secret values are runtime from Azure Key Vault, you only need to worry about managing secret rotation and security in one place. This is why Azure/get-keyvault-secrets is so awesome.
Wrapping Up
After setting up these workflows, make sure to require they pass before a PR can be merged. You can do this in the repository settings by creating a branch protection rule for main and selecting “Require status checks to pass before merging”. Then list the names of the workflows we created above.
Now your Operator-SDK based application is ready for automated testing! Reach out if you have any questions or comments!
Cheers.
