Open in app

Sign in

Write

Sign in

Creating a GKE cluster using Crossplane

Andre Almar
Quiqup Engineering
Published in
7 min readAug 5, 2022

What is Crossplane?

You can think of Crossplane as being a framework for building your own cloud native control plane or platform. You can do it declarative so you don’t have to write any code to make this happen.

But what is a Control Plane?

Google Cloud is a very good example of a control plane. They have been using control planes for years. If you ask for a cloud resource, in their backend services they have a control plane running to do all of the orchestration for the machines, storage, etc to provision and dynamically give you the services and resources you requested. Kubernetes is another example of a control plane. It orchestrates pods, applications, containers and so on.

Crossplane will help you build your own control plane and also allows you to put your own opinion into that control plane. Anything with an API could be managed by Crossplane. You can even order a Domino’s Pizza using Crossplane. Don’t believe it? Check it out 🍕

Concepts

Before we get our hands dirty we need to learn a concept that will help us understand how Crossplane works.

Managed Resource

A Managed Resource represents external objects in Kubernetes. On Google Cloud we have lots of things that we could use like Kubernetes Clusters (GKE), Databases (Cloud SQL), VPCs, PubSub and so on. We can represent those objects in Kubernetes YAML style. For example, if we want to represent a Google Cloud Storage Bucket using Crossplane we will do something like:

apiVersion: storage.gcp.crossplane.io/v1alpha3
kind: Bucket
metadata:
  name: example
  labels:
    example: "true"
  annotations:
    # Note that this will be the actual bucket name so it has to be globally unique/available.
    crossplane.io/external-name: crossplane-example-bucket
spec:
  location: US
  storageClass: MULTI_REGIONAL
  providerConfigRef:
    name: gcp-provider
  deletionPolicy: Delete

We are describing what infrastructure resource from Google Cloud we want to create (a Bucket) and delegating Kubernetes to manage the state of that resource for us.

This gives us many advantages over traditional IaaC tools like Terraform. Nic Cope told us about one of the advantages in his article Crossplane vs Terraform. He states:

Terraform relies on a monolithic state file to map desired configuration to actual, running infrastructure. A lock must be held on this state file while configuration is being applied, and applying a Terraform configuration is a blocking process that can take minutes to complete. During this time no other entity — no other engineer — can apply changes to the configuration.

Collaboration scales in Crossplane because the Crossplane Resource Model (XRM) promotes loose coupling and eventual consistency. In Crossplane every piece of infrastructure is an API endpoint that supports create, read, update, and delete operations. Crossplane does not need to calculate a graph of dependencies to make a change, so you can easily operate on a single database, even if you manage your entire production environment with Crossplane.

This is just one of many advantages we can have using Crossplane. Read Nic’s article to find more about it. There are more concepts about Crossplane but this is not the scope of this article. In the future we are going to cover other concepts so stay tuned!!!

Enough talking lets get to the action.

Installing Crossplane

I’m supposing you already have a local Kubernetes cluster in your machine with Helm installed. I’m using Minikube but you can user whatever you want (kind, k3d, etc).

Install Crossplane in your local cluster using Helm:

kubectl create namespace crossplane-systemhelm repo add crossplane-stable https://charts.crossplane.io/stablehelm repo updatehelm install crossplane --namespace crossplane-system crossplane-stable/crossplane

Check Crossplane Status to see if everything is ok:

helm list -n crossplane-system

Note that we are installing Crossplane in our local Minikube cluster. We need Crossplane running in an already created Kubernetes cluster in order to use Crossplane to create our GKE cluster. That’s a kind of a chicken and egg problem. How do we solve this? In a later post I will describe how to use Crossplane to manage the same cluster it is running on.

Creating a Service Account

You will need to create a Service Account in order to make Crossplane communicate with your GKE cluster (the GKE cluster where you will CREATE your Cloud Resources):

The Service Account must be created with the following Roles:

  • Computer Network Admin
  • Kubernetes Engine Admin
  • Service Account User

You will also need to create a new JSON key for this newly created Service Account:

After you created the JSON key, download it to your computer.

Creating a Provider Secret

Lets create a Provider Secret from the JSON Key you just downloaded from GCP:

cat > authentication.yaml <<EOF
apiVersion: v1
kind: Secret
metadata:
  name: gcp-account-creds
  namespace: crossplane-system
type: Opaque
data:
  credentials: $(base64 crossplane-credentials.json | tr -d "\\n")
EOF

P.S — I renamed the downloaded JSON key to crossplane-credentials.json

Apply the newly created authentication.yaml file:

kubectl apply -f authentication.yaml

Installing GCP Crossplane Provider

Install the Crossplane CLI to extend kubectl functionality to build, push and install Crossplane packages:

curl -sL https://raw.githubusercontent.com/crossplane/crossplane/master/install.sh | sh

Install the GCP Crossplane provider using the command below:

kubectl crossplane install provider crossplane/provider-gcp:v0.21.0

To see the latest GCP provider version go to: https://doc.crds.dev/github.com/crossplane/provider-gcp

Check to see if the provider is operational:

kubectl get providersNAME                      INSTALLED   HEALTHY   PACKAGE                           AGE
crossplane-provider-gcp   True        True      crossplane/provider-gcp:v0.21.0   41s

Note that INSTALLED and HEALTHY sections are True, so you are good to go.

Configuring ProviderConfig

Before creating any resources, we need to create and configure a GCP cloud ProviderConfig resource in Crossplane, which stores the cloud account information in it. All the requests from Crossplane to GCP will use the credentials attached to this ProviderConfig resource.

In this step we are configuring the Crossplane GCP provider to tell it where to get the authentication information we created on the step above.

cat > provider-config.yaml <<EOF
apiVersion: gcp.crossplane.io/v1beta1
kind: ProviderConfig
metadata:
  name: crossplane-provider-gcp
spec:
  projectID: YOUR_PROJECT_ID
  credentials:
    source: Secret
    secretRef:
      namespace: crossplane-system
      name: gcp-account-creds
      key: credentials
EOF

Apply:

kubectl apply -f provider-config.yaml

GCP Network

Before building our GKE cluster we first need to define which network and subnet the cluster will use. Lets create a network.yaml file:

apiVersion: compute.gcp.crossplane.io/v1beta1
kind: Network
metadata:
  name: network
spec:
  forProvider:
    autoCreateSubnetworks: false
    description: 'This is a network built by crossplane'
    routingConfig: 
      routingMode: 'REGIONAL'
  providerConfigRef:
    name: crossplane-provider-gcp
---
apiVersion: compute.gcp.crossplane.io/v1beta1
kind: Subnetwork
metadata:
  name: subnet
spec:
  forProvider:
    ipCidrRange: '10.144.0.0/20'
    networkRef: 
      # make sure this matches the network name you defined in Network
      name: network
    region: europe-west1
  providerConfigRef:
    name: crossplane-provider-gcp

Apply:

kubectl apply -f network.yaml

Check to see if the network and subnet were created:

$ kubectl get network
NAME       READY   SYNCED
network    True    True$ kubectl get subnetwork
NAME         READY   SYNCED
subnetwork   True    True

READY and SYNCED sections are True. You can check your Google Cloud console and see that the network and subnet were created.

GKE Cluster

Now we have all the elements to create our GKE cluster. Create a gke-cluster.yaml file with the following content:

---
# API Reference: https://doc.crds.dev/github.com/crossplane/provider-gcp/container.gcp.crossplane.io/Cluster/v1beta2@v0.21.0 
apiVersion: container.gcp.crossplane.io/v1beta2
kind: Cluster
metadata:
  name: gke-crossplane-cluster
spec:
  forProvider:
    initialClusterVersion: latest
    network: "projects/YOUR_PROJECT_ID/global/networks/network"
    subnetwork: "projects/YOUR_PROJECT_ID/regions/europe-west1/subnetworks/subnet"
    ipAllocationPolicy:
      useIpAliases: true
    defaultMaxPodsConstraint:
      maxPodsPerNode: 110 # By default, GKE allows up to 110 Pods per node on Standard clusters
    addonsConfig:
      cloudRunConfig:
        disabled: true
        loadBalancerType: LOAD_BALANCER_TYPE_UNSPECIFIED
      dnsCacheConfig:
        enabled: false
      gcePersistentDiskCsiDriverConfig:
        enabled: true
      horizontalPodAutoscaling:
        disabled: false
      httpLoadBalancing:
        disabled: false
      kubernetesDashboard:
        disabled: true
      networkPolicyConfig:
        disabled: false
    location: europe-west1
    binaryAuthorization: 
      enabled: false
    legacyAbac:
      enabled: false
    masterAuth:
      clientCertificateConfig:
        issueClientCertificate: false
    monitoringService: monitoring.googleapis.com/kubernetes
  providerConfigRef:
    name: crossplane-provider-gcp
  writeConnectionSecretToRef:
    name: gke-crossplane-cluster
    namespace: default
---# API Reference: https://doc.crds.dev/github.com/crossplane/provider-gcp/container.gcp.crossplane.io/NodePool/v1beta1@v0.21.0
apiVersion: container.gcp.crossplane.io/v1beta1
kind: NodePool
metadata:
  name: standard-pool
spec:
  forProvider:
    autoscaling:
      autoprovisioned: false
      enabled: true
      minNodeCount: 1
      maxNodeCount: 4
    cluster: projects/YOUR_PROJECT_ID/locations/europe-west1/clusters/gke-crossplane-cluster
    config:
      serviceAccount: YOUR_SERVICE_ACCOUNT@YOUR_PROJECT_ID.iam.gserviceaccount.com
      diskSizeGb: 100
      diskType: pd-ssd
      imageType: cos_containerd
      labels:
        team: platform
        cluster_name: gke-crossplane-cluster
        created_by: crossplane
      machineType: n1-standard-2
      oauthScopes:
        - "https://www.googleapis.com/auth/devstorage.read_only" # is required for communicating with gcr.io
        - "https://www.googleapis.com/auth/logging.write"
        - "https://www.googleapis.com/auth/monitoring"
        - "https://www.googleapis.com/auth/servicecontrol"
        - "https://www.googleapis.com/auth/service.management.readonly"
        - "https://www.googleapis.com/auth/trace.append"
        - "https://www.googleapis.com/auth/compute" # is required for mounting persistent storage on your nodes.
    initialNodeCount: 1
    locations:
      - europe-west1-c
      - europe-west1-d
    management:
      autoRepair: true
      autoUpgrade: true
    upgradeSettings:
      maxSurge: 1
      maxUnavailable: 0
  providerConfigRef:
    name: crossplane-provider-gcp

Apply:

kubectl apply -f gke-cluster.yaml

This will take a while to be in a ready state. It took me around 20 minutes to create the Cluster. The time varies a lot. It could be more, it could be less than that.

Check to see if the cluster is running:

$ kubectl get cluster
NAME                     READY   SYNCED   STATE     ENDPOINT       LOCATION       AGE
gke-crossplane-cluster   True    True     RUNNING   100.111.7.99   europe-west1   25m

STATE is RUNNING so the cluster was created successfully.

In this tutorial I created the simplest GKE cluster possible using Crossplane. There are other ways of creating a Kubernetes cluster and other cloud resources using Compositions for example. But this is for another time. For now I’ll leave you with this.

I’m excited to see where Crossplane adoption will get. Infrastructure world these days changes very fast but I see potential in Crossplane to be the defacto tool to manage Infrastructure in a modern way. Judging by the buzz in the last Kubecon Europe 2022, Crossplane has a bright future. New providers are being built and use-cases are becoming more popular. If I’m right or wrong only time will tell.

Thanks for reading!!!

Infrastructure
Platform Engineering
Crossplane
Kubernetes
Google Cloud Platform

--

--

Andre Almar
Quiqup Engineering

Written by Andre Almar

175 Followers
Writer for

I love to travel and I’m trying to change the world one line-of-code at a time.

Help

Status

About

Careers

Press

Blog

Privacy

Terms

Text to speech

Teams