The core logic of Kubernetes operator controller is reconciliation that adjust the actual state to the desired state of CR(Custom Resource). In this post, I’ll investigate typical reconciliation logic and how to unit test it.
What this post covers
- Reconciliation logic of operator controller that leverages Golang (not Helm or Ansible)
- How to unit test operator controller
- Example code(Operator-SDK official sample) with explanation
What this post doesn’t cover
- The basic concept of Kubernetes, Custom Resource, Operator
- How to install and setup Kubernetes cluster
- How to use Operator-SDK
- Golang syntax
I assume you understand above concepts, but you never have to be a master of them to understand this post. (I don’t know much neither)
Operator controller reconciliation logic
Operator controller should have reconciliation logic for the CR it manages. It adjust the actual state of the child resources of CR to the desired state declared in CR manifest.
Let’s see sample code from Opearor-SDK official sample code.
Reconciliation logic of sample code goes like this
- Retrieve CR object from cluster using name, namespace given via request parameter. Note that the controller already knows resource type, obviously CR, in this case Memchached.(Line 1:6)
- If there’s no CR with given name, it does nothing but return success. It means the CR object requested already deleted therefore no action is necessary. We cannot reconcile dead CR. (Line 7:14)
- For the successfully retrieved CR object, if there’s no child resource(deployment) it creates one using CR object to reconcile and requeue. (Line 20:33)
- It does reconciliation between CR object and its child resources. In above example CR has only one child resource, deployment. It updates Deployment.Spec.Replicas to match Memcached.Spec.Size, where the prior is child of CR object and the later is the CR object itself. Then requeue. (Line 39:50)
- Finally update status and return success (Line 54:75)
Reconciliation logic is pretty simple. Just fetch CR object and create or update child objects(e.g., deployment, service, secret, …) and update status accordingly.
How to unit test controller
Disclaimer about “good” unit testing
What is good unit testing is one of the most controversial topic among software engineers who matter software quality, refactoring, and agile manifesto. Now I carefully talk about my “opinionated” point of view.
- Unit test is all about assumption and result. It’s just same as A -> B proposition. “If A is given, then this module returns B”. We call A test fixture, and B test result.
- Unit test should mock(or stub) dependencies of the target module.
- If unit tests cover core business logic, then coverage(%) doesn’t matter. I don’t pursue 90~100% coverage dogmatically.
- Unit tests have to minimize assumption about dependencies of the target module. The dependencies of module should be loosely coupled with test codes.
Mock k8s client
Since controller heavily use k8s client, we need to mock this object efficiently. Thanks to k8s fake client library, we can easily inject CRD and CR object as constructor parameters of k8s client.
Unit test cases
We need test cases to write tests. So what’s core business logic of controller? Of course reconciliation. We can derive three test cases from the logic.
- If the CR object doesn’t exist, then this module should do nothing and return success
- If the CR object exists but child resource of it doesn’t exist, then this module should create one and requeue
- If the CR object exists and child resource also exists but two doesn’t match, then this module should update child resource to match CR object and requeue.
- … (We can add test cases about status update logic)
I want to add unit test codes for above sample controller but i have no time for it. If someone needs it i’ll update this post later. Just comment.