EXPEDIA GROUP TECHNOLOGY — ENGINEERING

Crossplane Implementation & Integration with AWS

Beyond Provisioning: One-stop solution for hybrid Kubernetes Infrastructure

In the last few years, tools like Terraform, Cloud Formation (AWS), and many more have had groundbreaking advancements for provisioning infrastructure. However, all these tools exist outside the Kubernetes ecosystem and the platform community is very much moving into Kubernetes. Crossplane bridges this divide by utilizing the Kubernetes control plane to provision multi-cloud infrastructure.

Why Crossplane?

The current world of automation demands IaC (Infrastructure as Code) and often leverages open-source technologies:

1. Terraform
2. Pulumi
3. Configuration Management Tools (Ansible/Chef/Puppet)
4. Pure old bash scripting or some other coding (Python/Node)

Or cloud vendor services like :

1. Cloud Formation (AWS)
2. Azure Resource Manager
3. Google Cloud Deployment Manager

But none of the above toolsets can exist as a part of the Kubernetes ecosystem and help spin up resources within Kubernetes itself.
Crossplane has come into the picture to solve this problem.

Crossplane Introduction

Crossplane is an open-source multi-cloud control plane that utilizes the Kubernetes control plane to provide an interface to various cloud providers. It integrates directly with Kubernetes and provisions cloud resources in the form of Kubernetes manifests. One of the most important advantages of Crossplane is being cloud-agnostic which enables a hybrid ecosystem rather than relying on a single technology for infrastructure provisioning.

Image by Andre Piacquadio from pexels

Implementation

Crossplane has a good amount of documentation to get started like the install guide and provision infrastructure guide. This way Crossplane can be installed and configured in a sandbox manner, but our use case requires a production-grade solution with standardized helm charts and security.

Pre-requisites

1. Kubernetes Cluster (If you do not have a k8s cluster follow this)
2. AWS account (With relevant access)
3. IAM Role with restricted access (To be used by Crossplane)

Install vanilla Crossplane

Enter the following series of commands on your shell to install Crossplane.

# 1. Create namespace dedicated to crossplane
kubectl create namespace-crossplane-system
# 2. Add Crossplane chart in the local helm repository
helm repo add crossplane-stable https://charts.crossplane.io/stable
# 3. Update our local repo
helm repo update
# 4. Install crossplane
helm install crossplane --namespace crossplane-system crossplane-stable/crossplane
# 5. Check for all crossplane components
kubectl get all -n crossplane-system

Now Crossplane is up and running. At this stage, crossplane can be integrated with any cloud provider since it is cloud-agnostic.

Crossplane Integration with AWS

GIF by CleverReach from giphy

IRSA

Let’s start by addressing security. Even if we allow Crossplane to provision resources, it still needs to be watched over for security and safety purposes. Therefore an important concept called IRSA (IAM Roles for Service Accounts) comes into the picture here. We incorporate IRSA with the provider package using controllerconfig. Don’t worry, definitions are coming next.

Provider

Providers are packaged controllers which are installed in the form of deployment and are catered by core crossplane controller through their own provider packaging mechanism.

Providers come with default values so in case we need to provide custom values like in our case as we need to give restricted access to crossplane to access our AWS, controller config is required

Install Provider and Controller Config

IAM role referred as a pre-requisite above is to be entered in the annotation below

It should be noted that in the above case the package is being pulled from a public repository. For the production scenario, we would have to ensure the package is pulled from a private repository, and to override it we would have to provide an image in controller config as well. As described above it overrides provider’s default values.

kubectl apply -f provider-controller-config.yaml

Once the provider is up and running, a lot of new CRD(Custom Resource Definitions) would be available w.r.t AWS and you would be able to spin up instances of those services. The following command would reflect all the new CRDs available now w.r.t Crossplane.

kubectl get crds -A

Last piece of the puzzle

To utilize those credentials to provision new resources, you must create a ProviderConfig with source: InjectedIdentity: A custom solution can be referenced below.

apiVersion: aws.crossplane.io/v1beta1
kind: ProviderConfig
metadata:
  name: crossplane-provider-config
spec:
  credentials:
    source: InjectedIdentity

GIF by Snanko from tenor

Provisioning AWS Resources

Now we are good to go with provisioning AWS resources. Crossplane supports a lot of AWS resources and its community is growing immensely.

IAM

Let's take a few use cases here where we’d be creating a whole IAM package in which IAM role, IAM policy, and IAM policy attachment would be created giving restricted access to route53 service.

kubectl apply -f iam-route53.yaml
iamrole.identity.aws.crossplane.io/crossplane-read-route53 created
iampolicy.identity.aws.crossplane.io/crossplane-read-route53 created
iamrolepolicyattachment.identity.aws.crossplane.io/crossplane-read-route53 created

Confirm the created resources

kubectl get iampolicy; kubectl get iamrole; kubectl get iamrolepolicyattachment
NAME                      ARN                                                        READY   SYNCED   AGE
crossplane-read-route53   arn:aws:iam::xxxxxxxxxxx:policy/crossplane-read-route53   True    True     30m
 
NAME                      READY   SYNCED   AGE
crossplane-read-route53   True    True     22m
 
NAME                      READY   SYNCED   ROLENAME                  POLICYARN                                                  AGE
crossplane-read-route53   True    True     crossplane-read-route53   arn:aws:iam::xxxxxxxxxx:policy/crossplane-read-route53   9m35s

S3

kubectl apply -f s3.yaml

Conclusion

So as proven Crossplane directly integrates very well with Kubernetes. The future of Crossplane is very ambitious as it keeps its infrastructure only in Kubernetes scope, exploits the Kubernetes control plane, and is cloud-agnostic.

GIF by SevenBucks from tenor