Creating Custom Providers in Crossplane

A Tutorial For Absolute Beginners

Introduction & Goals

Are you interested in writing your own custom Crossplane provider and not sure how to get started? Then this tutorial can help you. This tutorial is meant to be the document I wish I had when I got started, and hopefully you will find it helpful as well.

What is Crossplane? It is an open source project that provides a framework to create a control plane to help manage Iaas and other API endpoints as Custom Resources in your Kubernetes cluster. It is maintained by the company UpBound, and is one of the many projects overseen by the Cloud Native Computing Foundation.

I recently flew to KubeCon EU 2022 in Valencia, Spain where I attended a session hosted by Hasan & Muvaffak from Upbound. They walked through how to create your own custom providers in their talk (Writing Crossplane Providers with Code Generation — Muvaffak Onuş & Hasan Türken) which is published on the CNCF YouTube Channel. They do a great job explaining all the necessary steps, and this tutorial is meant to complement the live demo with written explanations outlining the major components. Please note the outline below has been edited down to only focus on the fundamental steps pertaining to creating a PlanetScale database custom resource. The video tutorial contains additional steps to creating a PlanetScale password custom resource which will not be covered in this article.

Let’s Create a Custom-Provider

Step 0: Prerequisites

For this tutorial, you will need the following:

• Fundamental knowledge of git, kubernetes, & go.
• Go version 1.18
• A kubernetes cluster (ex. minikube)
• Account to an API enabled service (ex. PlanetScale)
• Go Client for API Service (ex: planetscale-go)

Step 1: Set Up the Provider Codebase

1. Sign into Github
2. Fork this template by selecting “Use this Template” — crossplane/provider-template

3. Clone template to local repository

4. Change Directory into your repository and run the following commands to configure your repository:

# Fetch [upbound/build] modules 
make submodule
# This command will update code with your custom provider name
# Replacing anything labeled as 'template'
# You can only run command this once when you initially setup your provider repository
# Format: make provider.prepare provider provider=xxx
make provider.prepare provider provider=PlanetScale
# This command will generate your api and controller files to define your Kinds under an API group
# Format: make provider.addtype provider=xxx group=yyy kind=zzz
# Create a Database Kind under Database Group
make provider.addtype provider=PlanetScale group=Database kind=Database
# Optional: Create a Password Kind under Branch Group
make provider.addtype provider=PlanetScale group=Branch kind=Password

5. Validate All the Directories and Code Were Generated:

/apis/planetscale.go
/apis/database/*
/internal/controller/planetscale.go
/internal/controller/database/*
# Optional: 
/apis/branch/*
/internal/controller/branch/*

Step 2: Configure Your First ‘Kind’

From the previous step, there were two ‘Kinds’ that were generated; database.database & branch.password. The Database Kind will be a custom resource that will manage a database instance in your PlanetScale account, and the Password Kind is an additional custom resource required to allow service accounts to interface with the Database. Note that the Password is NOT the account credentials used to make API calls to the PlanetScale service. For the remainder of the tutorial, only steps pertaining to creating a database custom resource will be covered.

We will begin by updating the following file: /api/database/v1alpha1/database_type.go

The *_type.go files defines the structs for type Parameters (configs ForProvider) and Observations (configs AtProvider).

The *Parameter struct defines the configurable fields that would be passed by the manifest yaml file under ‘ForProvider’. These are the desired state of your CRD, and Kubernetes will try to Create or Update based on these parameters. I recommend identifying the desired format of the manifest file to better understand how to format these structs.

1. Update Parameters

In /api/database/v1alpha1/database_type.go. At a minimum, these parameters should reflect the configurations required to create the resource. For PlanetScale Database, we will copy the CreateDatabaseRequest struct from the PlanetScale-Go Client.

The *Observation struct defines the fields passed from the service API to the custom resource and stored as ‘AtProvider’. These are the observed state of your CRD, and stored as metadata in the resource. Later in the tutorial you will develop the logic to compare the values between ‘ForProvider’ and ‘AtProvider’ to either trigger an update, create, or do nothing.

2. Update Observation

In /api/database/v1alpha1/database_type.go. These observations should reflect the configurations required to compare against the desired state to trigger an update. For PlanetScale Database, we will just observe the ‘State’ parameter for now.

3. Register the new Groups

Add groups to AddToSchemes in /apis/planetscale.go (ln 31–34)

4. Register the new Types in Setup():

/internal/controller/planetscale.go (ln 30–35)

5. Generate files

Generate the /api/group/zz_*.go files and /package/crds/*.yml manifest files. These files will define the functions and structs to interact with externally managed resources. Run the following command to do so:

make generate

Step 3: Set Up Credentials Secret for API Authentication

Almost all custom resources you create will require credentials to make an authenticated API call to the target service. This credential is stored in a Kubernetes Secret which will be referenced later by the ProviderConfig file.

Follow the steps below to setup the Access Token to interface with PlanetScale. Warning this is not the same as Service Token that is acquired through the PlanetScale UI. Access token is generated in a local file after cli is initialized.

1. Install pscale cli: Follow the instructions to setup pscale cli.

2. Initialize pscale cli: Run command pscale login.

3. Copy Access Token

After successful initialization of cli with your own credentials, copy your access-token from here: $HOME/.config/planetscale/access-token

4. Convert Your Access Token String to Base 64

echo -n 'my-access-token' | base64

5. Create Kubernetes Secret

Copy Your Credentials in manifest file pscale-token-secret.yml:

apiVersion: v1
kind: Secret
metadata:
  name: mysecret
type: Opaque
data:
  servicetoken: my_base_64_encoded_token

6. Deploy Your Secret:

kubectl apply -f pscale-token-secret.yml 

Step 4: Set Up Service & Connector for Controller

Follow these steps to update the /internal/controller/database/database.go file to define the client service and authentication method to target API service account.

1. Import the client package for your desired API service.

Example for PlanetScale:

# Command Line:
go get github.com/planetscale/planetscale-go
go get github.com/crossplane/crossplane-runtime/apis/common/v1
# Add to Import Package in database.go:
import (
"github.com/planetscale/planetscale-go/planetscale"
xpv1 "github.com/crossplane/crossplane-runtime/apis/common/v1"
)

2. Update NoOpService to PlanetScaleService

This var will define how to initialize the client with creds passed in from secrets.

3. Add NewPlanetScaleService to Setup() & Update connector{}

Step 5: Add Controller CRUD Operations

This is where the custom resource is defined with how it will Observe, Create, Update and Delete a PlanetScale database.

For each customer resource, the controller defines the condition by which each method is invoked, and how it implements each them. The Observe() methods is the most important for it will define the conditions to invoke the Create() and Update() methods. The Delete() method is invoked when the resource is deleted via kubectl delete commands.

1. Update External Struct for PlanetScale service (ln 135–138)

2. Update Observe() (ln 144–181):

3. Update Create()(ln 184–215):

The Create() function is triggered when the Observe() returns ResourceExists: false.

4. Update Delete()(ln 233–246):

5. Optional: Update Update()::

The tutorial does not address any steps to update a database. For any other custom resource that can be updated via API, add logical steps under this function. The Update() function is triggered when the Observe() returns ResourceUpToDate: false.

Step 6: Deploy Custom Provider, Provider Config & Custom Resources

Now that the resource parameters (/api/database/v1alpha1/database_type.go) and the controllers (/internal/controller/database/database.go) have been defined, it’s time to deploy your first custom resource.

1. Prepare Your K8s Cluster:

• Start Local Cluster: minikube start
• Generate All Required Manifest Files, : make generate
• Generate Cluster & Install Provider: make dev
• Double Check Secret is Deployed: kubectl apply -f pscale-token-secret.yml

2. Define & Deploy your ProviderConfig (examples/provider/config.yaml)

Remember to update secretRef with the namespace, name & key defined in your Secret. This config file will create a custom resource ProviderConfig which defines the secret to the service API. This will be referenced when deploying the PlanetScale database.

Deploy: kubectl apply -f examples/provider/config.yaml

3. Define & Deploy your PlanetScale Database

The following manifest file will deploy a PlanetScale database called ‘firstdb’, with description ‘my first db’ in the organization ‘my-org’. Replace these with values that reflect your situation. The providerConfigRef should reflect the name of the ProviderConfig deployed from above which references the access token secret. In this case, the ProviderConfig name is also ‘firstdb’.

Deploy: kubectl apply -f examples/sample/firstdb.yaml

Voila! Congratulations on deploying your first Crossplane custom resource; a PlanetScale Database! The firstdb ‘Database’ resource now manages this database in PlanetScale.

4. Clean Up

To delete your PlanetScale Database, run: kubectl delete -f examples/sample/firstdb.yaml

If you are done with your local cluster, run the following to reset & delete cluster configurations: make dev-clean

Conclusion

Thank you for following along! Hopefully you were able to learn more about creating custom providers with Crossplane. The open source project offers many powerful utilities to better manage cloud infrastructure as well as options to customize existing providers. That value compounds when the greater developer community builds and publishes more custom-providers for others to use. To learn more I recommend visiting the project site at crossplane.io.

See you around!