Using Crossplane to Provision a Kubernetes Cluster in Google Cloud
The New Kid in Town
Ever since a former co-worker pointed me in the direction of Crossplane, I’ve been both obsessed and intrigued. I just HAD to try it out for myself. In case you’re not familiar with it, Crossplane is an open-source a Kubernetes-native tool for provisioning Cloud infrastructure. It was initially released in December 2018.
Fair question! There are a few things that really intrigued me about Crossplane:
- It adheres to the principles of Infrastructure as Data. It’s Kubernetes-native, so you use Kubernetes Custom Resources (YAML — i.e. text) to declaratively provision Cloud infrastructure.
- No state file. Unlike Terraform and Pulumi, Crossplane doesn’t use a state file. That said, as a Kubernetes Operator, it has a controller to reconcile desired state against current state, making use of etcd to do so. Furthermore, unlike Terraform and Pulumi, if someone tries to do something sneaky like update a Cloud resource using the CLI or an admin console, Crossplane won’t let you get away with it. It is the source of truth.
Okay…enough of me yapping. Let’s dig in!
Creating a GKE Cluster Using Crossplane
This tutorial will guide you through the creation of a Kubernetes cluster on Google Cloud using Crossplane.
This tutorial assumes that:
- You have an existing Google Cloud project
- You’ve created a Service Account in Google Cloud
- You’ve created a Google Kubernetes Engine (GKE) cluster before
- You have an existing Kubernetes cluster up and running, since we’ll be installing Crossplane on that cluster.
- You have the
envsubstcommand installed on your machine. You can find instructions for Mac and RedHat/CentOS here. On Ubuntu, use
apt-get install gettext.
1- Clone the tutorial repo
Let’s begin by cloning the tutorial repo:
git clone firstname.lastname@example.org:d0-labs/crossplane-gke.git
2- Set up your environment variables
For your convenience, I’ve created a file called
env_vars.sh in the
Replace the values in
<...> with your own as follows:
<gcp_project_id>: This is the name of our Google Cloud project. If you’re wondering what your project name is, use this command:
gcloud projects list
PROJECT_ID NAME PROJECT_NUMBER
aardvark-project aardvark-project 112233445566
Use the value returned in the
<gcp_service_account_name>: The name of the service account for your Google Cloud project.
<gcp_service_account_keyfile>: This is the fully-qualified name of the JSON service account private key stored on your local machine. For example,
/home/myuser/my-sa.json, if your file is located in the
/home/myuser folder. Or
my-sa.json if your file is located in your current working directory.
Note: This JSON private key is generated upon creation of the Service Account, to be sure to store it somewhere safe (and not in version control). Per Google’s docs on Service Account Keys, “After you download the key file, you cannot download it again.”
<gke_cluster_name>: Name of the Kubernetes cluster on Google cloud where you’ll be installing Crossplane.
<gke_cluster_zone>: Name of the zone in which your Crossplane Kubernetes cluster resides.
Note: This tutorial assumes that the k8s cluster you’re using to install Crossplane is running on Google Cloud. Feel free to comment out/remove lines 10 and 11 above as needed.
Save your changes, and head on over to the next step. We’re not executing this file at this point.
2- Configure Google Cloud
Note: If the Kubernetes cluster on which you’re installing Crossplane is not on Google cloud, you can skip this step.
Let’s make sure that you’re all set up properly in Google Cloud. The
1-gcp_config.sh script activates your Service Account to authenticate against Google Cloud, set your GCP project to the one where your (soon-to-be) Crossplane Kubernets cluster is located, and connects to that cluster.
Run the script:
3- Install Crossplane on your Kubernetes cluster
Note: Upbound Cloud is nothing more than a Cloud-hosted Kubernetes cluster that already has Crossplane installed for you. This is definitely a very viable option, as it spares you the need to provision a Kubernetes cluster just to run Crossplane.
If you choose to use Upbound Cloud instead, feel free to skip this step.
For our example, we’ll be installing Crossplane on an existing Kubernetes cluster using the
2-install_crossplane.sh script below:
This script installs Crossplane v1.2.0 on your Kubernetes cluster using Helm (lines 6–9). Let’s go ahead and do that:
4- Install & Configure the Crossplane GCP Provider
This script installs and configures the Crossplane GCP provider. A Provider is code used by the Cloud infrastructure provisioning tool (in this case, Crossplane) that allows it to interact with the target Cloud service’s API (in this case, the GCP API). The Cloud service’s API is what’s used to provision the infrastructure. It’s the same concept in Terraform and Pulumi.
Line 6 uses
kubectl to install the GCP Provider. The
Provider is a custom resource. We could also use Crossplane’s CLI (it’s a
kubectl plugin) to do that, but I prefer this approach, since it’s one less thing to install, and it’s declarative, so you can keep it version-controlled.
Provider YAML looks like this:
Note: You can find this YAML in Crossplane’s own provider-gcp GitHub repo.
After installing the GCP Provider, we must configure the Crossplane provider, by telling it about our GCP project and how to authenticate, so that Crossplane can actually provision infrastructure there. This happens in Line 16 of the shell script.
Although I could’ve hard-coded my GCP Provider configs in a YAML file, I’ve chosen to do some templating instead, using the Linux
envsubst command (see prerequisites section on how to install). This command replaces environment variables in a specified file (in our case,
provider-config-gcp.template.yml), with their values. In our script, we set the environment variables in Line 3 of the shell script, when we call
. ./scripts/env_vars.sh. The appropriate values are then replaced in
provider-config-gcp.template.yml, and we end up with a new file called
provider-config-gcp.yml, with the replaced values from
provider-config-gcp.yml file will look something like this:
In the file above, we’re creating Kubernetes
Secret which contains our GCP Service Account key file. The
Secret is used by a
ProviderConfig custom resource to authenticate to our Google Cloud project,
my-gcp-project. This gives Crossplane permission to create resources in
Now that we understand what the shell script is doing, let’s go ahead run it, so that we can install and configure the GCP Provider:
4- Provision a GKE cluster
With Crossplane installed and our GCP Provider configured, we can finally provision our cluster! The script below does this for us:
gke-install.yml, we see that this is where the magic happens. Here, we’re provisioning a GKE cluster and nodepool:
If you’ve provisioned a GKE cluster with Terraform, Pulumi, or Ansible, the fields may look familiar. Be sure to check out the GKECluster API docs and the NodePool API docs for more info on these fields.
NodePoolcustom resources are not namespaced (i.e. don’t belong to a namespace).
To create the cluster, let’s run the script:
We can check the status of our cluster:
kubectl describe gkecluster gke-crossplane-cluster
If you scroll down to the end of your output to the
Events section, you’ll see something like this, which tells us that our cluster is being provisioned:
If you take a peek at your Google Cloud Console, under
Kubernetes Engine > Clusters, you’ll see something like this:
And if you click on the cluster, and then the Nodes tab, you’ll see that your node pool is being provisioned as well:
Don’t panic if you see the error and warning messages like the ones above. The node pool is still being created.
To check on the status of the node pool creation:
kubectl describe nodepool gke-crossplane-np
It takes a while to create the node pool, so keep running the above command over and over until you get a message like the one highlighted above.
The overall cluster creation process should take about 5–10 minutes.
5- Connect to the cluster
If all goes well, you should now have a brand-spaking-new Kubernetes cluster. Before we can check our cluster, we need to add it to our
gcloud container clusters get-credentials gke-crossplane-cluster --zone us-central1-a --project <your_project>
Be sure you replace
<your_project> with your actual GCP project name.
Now, let’s do a quick spot check. First, let’s make sure that the cluster is in our
kubectl config get-contexts
We should get an output that looks something like this:
Yup. There’s our cluster!
Let’s also run a quick command in our cluster to check the namespaces:
kubectl get nodes
Your output should look something like this:
We provisioned 2 nodes, and we see two nodes.
And let’s peek into our namespaces:
kubectl get ns
There you go! We’ve got ourselvs a GKE cluster!
6- Delete the cluster and nodepool
To delete the cluster, all you need to do is delete the
GKECluster resource from Kubernetes.
kubectl delete gkecluster gke-crossplane-cluster
The command prompt won’t return until the cluster has been deleted. You can also check deletion status in the Google Cloud Console.
For good measure, also delete the
kubectl delete nodepool gke-cluster-np
Thoughts on Crossplane for Cloud Provisioning
Crossplane hasn’t been around for very long, and it shows. While there’s a lot of support for AWS resources, support for Google Cloud and Azure is rather lacking in comparison. That said, I’m sure that as it picks up steam, the good folks at Crossplane will be adding more resources from those two Cloud Providers (and others) into the mix.
As far as open source software goes, I found it relatively straightforward to get going using their getting started guide. I did run into a bit of a pickle initially, as I found myself reading docs from both version 1.2.0 and 0.7, not realizing it at the time, and wondering why things weren’t working as they should. That was my my fault…but since it happened to me, it could happen to you, so just make sure you’re looking at the right version of the Crossplane docs when you’re trying stuff on your own.
My only complaint with the quickstart is that I really don’t understand why Crossplane suggests installing their CLI and then installing your desired Provider package, when you could easily do it declaratively by applying the Provider YAML (like for GCP) using
I like the fact that Crossplane has the Upbound Cloud managed Crossplane service, so you don’t have to figure out how to set up and configure your own k8s running Crossplane just to be able to provision Cloud infrastructure.
It’s a nice option to have, though I didn’t find it too bad installing it on an existing k8s cluster.
Note: My example had you installing Crossplane on an existing GKE cluster to provision another GKE cluster (very meta). I’m guessing that this is not a very common use case, and that the makers of Crossplane expect you to either use the Upbound Cloud managed Crossplane service or to install Crossplane on a local k8s cluster like KinD or MiniKube.
Crossplane is a Kubernetes-native tool for provisioning Cloud infrastructure.
Right now, I’m still leaning towards Ansible for provisioning Cloud resources in Google Cloud or Azure at the moment, since the number of resources supported by Crossplane for these providers is a fair bit lower compared to AWS. That said, don’t count Crossplane out of the mix. It’s definitely worth keeping an eye on it to see what the future brings!
And now, I will leave you with a picture of our sleepy rat, Susie, in my husband’s arms.
Peace, love, and code.
Be sure to check out other posts in my Cloud infrastructure series!
Shifting from Infrastructure as Code to Infrastructure as Data
A brief intro to Cloud infrastructure, and a dive into why you should take a data-driven approach to provision cloud…
Using Ansible’s GCP Library to Provision a Kubernetes Cluster in Google Cloud
Step aside, Terraform! Learn how to use Ansible’s Google Cloud modules to provision a GKE cluster in an easy and…