Crossplane is underrated
Although there are many different Infrastructure as Code (IaC) tools available, the people I know mostly use Terraform. As someone who prefers a more Kubernetes-native approach to IaC, I was motivated to write this short article about Crossplane. My aim is to spark interest among more users and encourage them to consider it as a viable alternative to their current IaC tool and move towards a declarative architecture.
Crossplane is an open-source Kubernetes extension that transforms your Kubernetes cluster into a “Universal Control Plane.” In other words it is a framework for provisioning infrastructure by extending Kubernetes APIs.
Primarily aimed at Platform Engineers and SRE’s Crossplane allows users to provision any resource (assuming the provider support exists, which it usually does). This can range from an RDS instance on the cloud to a more complex managed Kubernetes cluster, which requires a group of individual resources such as IAM roles, security policies, role policy attachments (yes, this is an object in Crossplane), etc. To keep things simple, I will provision an S3 bucket and bucket-public-access-block via Crossplane in the following Demo.
Repository Used in this Article: https://github.com/moonorb/crossplane-demo
Crossplane can be used for deploying and managing multiple Kubernetes clusters, and it accelerates deployment by leveraging Kubernetes-native APIs and ensuring consistent infrastructure across different environments.
What problem are we trying to solve, and why should someone switch to Crossplane? The answer is straightforward: the power of a “Control Plane.”
“Drift” occurs when the desired state of the infrastructure differs from the actual state. This is a common issue in the Terraform world because it’s tempting to make quick changes via the Console (ClickOps). As soon as this happens, the desired state in your state file has “Drifted” from your actual state. This issue becomes especially complicated when multiple engineers are working on the same infrastructure. Fortunately, if such a change was made on a resource managed by Crossplane, the manual change would have been reverted within a few minutes. Unlike Terraform, where commands run in a “fire and forget” mode, Crossplane uses the Kubernetes reconciliation loop mechanism to provision resources and continuously monitor the state in memory instead of a file. This ensures that the intended state of the resources always matches the actual state, effectively preventing drift once and for all.
This behaviour is similar to deleting a pod from a deployment with two replicas, where a new pod is created as soon as one is deleted. While you can override this by setting “Observe” on the resource’s management policy, the principle remains the same. This feature is, in my opinion, the most powerful aspect of Crossplane.
To use Crossplane, a “Management Kubernetes Cluster” is needed. For testing purposes, this can be achieved simply via Kind (for demo purposes) on a single host. This is the easiest method, which I have demonstrated in the following Demo. For actual production systems, a more robust Kubernetes cluster should be used.
A basic Crossplane installation deploys two pods on the Management Cluster in a dedicated namespace. Similar to Terraform, providers enable Kubernetes cluster to connect to the public cloud by translating manifests (for declaring resources) into API calls that cloud providers understand. Each provider also runs as a dedicated pod in the Management Cluster.
Since I am provisioning a simple bucket and a public-access-block resource, I only need to deploy the provider-aws-s3. Depending on the resource being deployed via Crossplane, the related provider must be deployed on the Management Cluster as a Custom Resource Definition (CRD). Besides cloud providers, there are also Kubernetes, Helm. OpenStack and various other providers. All providers can be found at Upbound Marketplace.
Below are the main Crossplane concepts:
Composite Resource Definitions(XRD): These are Custom APIs that use the OpenAPIv3 schema to further extend Kubernetes with custom API endpoints. XRDs define what fields should (or must) be in your compositions.
Composition: A collection or template of managed resources, such as an S3 bucket and a Bucket Policy.
Composite Resource(XR): Collections of objects created by users calling the custom API. A single Composite Resource can have multiple Managed Resources linked to it, making them higher-level abstractions than managed resources.
Claim: Namespaced Composite Resources. Each claim is linked to one Composite Resource.
Managed Resource: Individual resources created by Crossplane. When a Managed Resource is created inside your management cluster, a corresponding external resource will be created on your endpoint(i.e. AWS). To delete a resource (e.g., an S3 bucket), Managed Resource must be deleted from the management cluster.
Every component in Crossplane is declared via Custom Resource’s specific to Crossplane.
Below is a mapping of the CR’s that I used in this Demo.
Instead of installing kubectl, crossplane-cli and kind we will instead use Nix. While Nix is not a requirement for Crossplane, it is the only pre-requisite for this Demo.
# Clone the repo
git clone https://github.com/moonorb/crossplane-demo.git
# Install Nix Deamon
sh <(curl -L https://nixos.org/nix/install) --no-daemon
# Enter nix shell
cd crossplane-demo
nix-shell --run $SHELL
# Update aws-credentials
cat <<EOF > credentials/aws-credentials.txt
[default]
aws_access_key_id = <YOUR ACCESS KEY ID HERE>
aws_secret_access_key = <YOUR SECRET ACCESS KEY HERE>
EOF
# Run installation script
./scripts/install-crossplane.nix
# Deploy Crossplane CR's
kubectl apply -f apis/definition.yaml
kubectl apply -f apis/composition.yaml
kubectl apply -f examples/objectstorage-claim.yaml
# Check managed Objects
kubectl get managed
# After a few minutes, run watch command to make sure all Managed Resources are synced
watch "crossplane beta trace Objectstorage myobjectstorage -n test-ns"Managed Resources are created as Kubernetes objects in the Management Cluster. READY status is True which means they are successfully provisioned on AWS.
All these components are defined as Custom Kubernetes Resources(CR) in yaml format. There are also functions, packages and configurations but that’s for another day.
I tried to keep things super simple. Although things get a little more complicated when dealing with dependencies between different resources/compositions and there are some challenges with troubleshooting, so far I have been impressed with Crossplane.
My motivation to prefer Crossplane:
- It is a CNCF project with an Apache 2.0 license, donated to the community by Upbound. It has a very active Slack channel with great community support.
- There is a clear separation between Application users(Developers) and Platform teams in the Crossplane model. As a result, infrastructure complexity is abstracted from the end-user, making it the perfect tool for IDP.
- The Reconciliation Loop is just awesome.
- Resources are loosely coupled with an eventual consistency model(this term is from Victor Farcic). In other words, if you have 2 compositions with certain managed resources that are dependent on each other you can provision both Compositions and only the individual managed resources will wait for each other. As a result, overall provisioning time is improved when compared to TF.
- It leverages Kubernetes tooling by using CRDs as opposed to a proprietary language.
- You can use any programming language to write your functions or use the functions from Upbound marketplace, developed by the community.
- It has built-in support for packaging (via OCI containers) and versioning.
- It is more declarative than Terraform with the help of OpenAPIv3 schemas.
Written by Serhan Turkmenler
Reviewed by Anil Sonmez
