Installing cf-for-k8s on Google Cloud GKE

Capability of Kubernetes, Simplicity of Cloud Foundry, Google Cloud reliability — a match made in heaven?

In this tutorial, we aim to walk you through the steps that you need to take to install cf-for-k8s on your own Kubernetes clusters. But, for every step along the way we ask ourselves why. And by answering the why, we aim to educate you about the reasoning behind our steps while also communicating our many design decisions. Let’s begin with the first why -

Why Install CF-for-K8s?

In our experience, we’ve observed that small teams and startups often don’t pick up Kubernetes because of two reasons — one is the steep learning curves. Second, for fear of introducing complexity in the workflow of their developers.

By providing a “cf” wrapper around Kubernetes-based infrastructure, both these problems can be solved! The collective efforts of the Cloud Foundry community have resulted in the cf-for-k8s project which, when installed, allows application developers to deploy to Kubernetes using the tried and tested cf push workflow, which is known for its simplicity and ease of use. This way, teams no longer have to make a choice between simplicity, power, and the developer experience.

Kubernet-easy!

(psst — if you already have the cf, kubectl, bosh, gcloud CLIs, and ytt, kapp — skip ahead to the install process)

Prerequisites

There are some prerequisites to installing cf-for-k8s. Let’s quickly get into each of them. This section is meant for the truly uninitiated. You will need a handful of CLI tools to help with various parts of the installation.

You will need the cf CLI. Why?

The Cloud Foundry CLI is the tool that manages your local machine and helps connect to the remote Cloud Foundry endpoint. It is the official command-line client for Cloud Foundry.

Here are the instructions on how to install the client on your machine:

wget -q -O - https://packages.cloudfoundry.org/debian/cli.cloudfoundry.org.key | sudo apt-key add -echo "deb https://packages.cloudfoundry.org/debian stable main" | sudo tee /etc/apt/sources.list.d/cloudfoundry-cli.list
sudo apt-get update
sudo apt-get install cf7-cli

On a Mac, use Homebrew –

brew install cloudfoundry/tap/cf-cli

Next, you will need kubectl. Why?

Technically speaking, kubectl is the client for the remote Kubernetes API. It acts as the “control center” for your Kubernetes installation. You can conduct all control operations on your remote cluster by using kubectl.

curl -LO "https://storage.googleapis.com/kubernetes-release/release/$(curl -s https://storage.googleapis.com/kubernetes-release/release/stable.txt)/bin/linux/amd64/kubectl"
chmod +x ./kubectl
sudo mv ./kubectl /usr/local/bin/kubectl

Third, install gcloud. Why?

This tutorial assumes that you’re going to be using Google Cloud as your IaaS provider. When using Google Cloud, the gcloud command-line interface helps you create and manage all your resources on Google Cloud. Install this using the following command:

echo "deb [signed-by=/usr/share/keyrings/cloud.google.gpg] https://packages.cloud.google.com/apt cloud-sdk main" | sudo tee -a /etc/apt/sources.list.d/google-cloud-sdk.list
sudo apt-get install apt-transport-https ca-certificates gnupg
curl https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key — keyring /usr/share/keyrings/cloud.google.gpg add -
sudo apt-get update && sudo apt-get install google-cloud-sdk

If you’re on a Mac/Windows, use these instructions to install the Google Cloud SDK.

In this tutorial, we will specifically use it to create a static IP that will front the Kubernetes cluster. More details later.

Install the BOSH CLI. Why?

One step in the install process requires you to generate a YAML file. In case you would like to use a script to create this file with some defaults, you will need the bosh CLI.

The BOSH CLI is the command-line client for the core Cloud Foundry operations and releases management tool. In this installation process, it has only a small role to play.

Here’s how to install the BOSH CLI –

Download the right binary for your OS from https://github.com/cloudfoundry/bosh-cli/releases

Then, run the following commands:

chmod +x ./bosh
sudo mv ./bosh /usr/local/bin/bosh

YTT and KAPP

Finally, you will need 2 important tools from the k14s toolchain. These are ytt and kapp.

ytt is a YAML templating tool that is used to create the full YAML file needed for the cf-for-k8s installation. It is used to create a configuration file that parameterizes all the values and variables needed for installing cf-for-k8s.

Download the binary from https://github.com/k14s/kapp/releases

mv ytt-linux-amd64 ytt
chmod +x ./ytt
sudo mv ./ytt /usr/local/bin/ytt

kapp is a deployment tool that performs the actual installation of cf-for-k8s on the Kubernetes cluster. There are many advantages that the usage of kapp brings such as exempting custom resources, automatically ascertain create/update/delete operations, and providing deployment progress.

Download the binary from https://github.com/k14s/kapp/releases

mv kapp-linux-amd64 kapp
chmod +x ./kapp
sudo mv ./kapp /usr/local/bin/kapp

Creating a Kubernetes Cluster

Along with this handful of tools, we need a Kubernetes cluster on which to deploy cf-for-k8s. Here are the steps I took with Google Cloud to create a GKE cluster. I am using three nodes with 4vCPU each and 16 GB of RAM.

Please note:

• The cluster version should be any Kubernetes cluster in the version range 1.16.n to 1.18.n.
• The installation has worked with both 2 and 3 nodes in a cluster. The default docs prescribe a minimum of 5 nodes.

Now that we have all the tools and the cluster on which to deploy, we can begin the actual installation.

Preparing for the Installation

Set the kubectl context

In preparation for the installation, please ensure that the context of your kubectl command is set correctly.

You can verify this by using

kubectl config current-context

If you don’t find the cluster name on the console matching your target cluster, please change it. When using Kubernetes with Google Cloud, this command will help you configure the connection:

gcloud container clusters get-credentials <cluster-name> — zone us-central1-c — project summit-labs

Why is this needed? To be able to direct all the installation steps (and subsequently any governance/instructions that will be needed later).

Set kubectl config

Static IPs

Next, create a static IP.

Let’s answer the why first — This static IP will be externally visible and will serve as the ingress to the Kubernetes cluster. We will use this IP address to refer to the Cloud Foundry endpoint. We will include this IP address as part of the installation process so that the mapping is available as soon as the installation finishes.

How to create a static IP? On Google Cloud, indicate that you want to create an address along with the region in which you want it to be present. You can also specify a label with which you can identify the address. The exact command is

gcloud compute addresses create cf4k8sdemo --region us-central1

Once the static IP address is generated, create a local variable called IP. Assign the value to the local variable. This is for use later when generating deployment YAML files.

IP=$(gcloud compute addresses describe cf4k8sdemo --region us-central1 --format='value(address)')

Create Static IP on Google Cloud

There are a couple of alternative approaches as well. You can always modify the property of the ingress service post-installation. You can set the static IP as the parameter for the load balancer. The command for doing that is

kubectl patch svc istio-ingressgateway --namespace istio-system --patch '{"spec": { "loadBalancerIP": "$IP" }}'

The other approach is to use hostnames. Obviously, using static IP addresses is not desirable. For the same reason as websites having domain names and not IPs. :shrug: So, you can use a hostname and configure an ‘A’ record that redirects incoming requests to the ingress/load balancer that manages the Kubernetes cluster. (There will be a second tutorial that will provide instructions on using hostnames. Stay tuned!)

Installation

Now we get into the actual installation process.

Step 1:

Clone the cf-for-k8s repo. Why? Duh!

git clone https://github.com/cloudfoundry/cf-for-k8s.git

Step 2:

Switch into the cloned directory.

cd cf-for-k8s

Step 3:

Create a directory to use as a temporary/swap space for the YAML files. Assign a local variable named TMP_DIR the path to this directory. We will be using this directory to store the YAML files that we generate and use for the final installation.

mkdir -p ~/tempdir
TMP_DIR=~/tempdir

Step 4:

A YAML file containing the deployment manifest is needed. This will contain the details of the endpoint, our container registry credentials, and static ip among other things. You can generate this by yourself or make use of the auto-generate script that is included in the cf-for-k8s repo. This is a temporary measure meant for illustrative purposes only.

The command used to create the YAML file is:

./hack/generate-values.sh -d ${IP}.xip.io > $TMP_DIR/cf-values.yml

Don’t be alarmed when you see this message:

WARNING: The hack scripts are intended for the development of cf-for-k8s.

They are not officially supported product bits. Their interface and behavior may change at any time without notice.

Autogen Values & Set Static IP

Step 5:

A couple of additions have to be made to the cf-yalues.yml file that you created.

1. Add credentials that can help connect to your Container Registry. This could be Docker Hub or any private one.

app_registry:
    hostname: https://index.docker.io/v1/ 
    repository_prefix: "<my_username>" 
    username: "<my_username>" 
    password: "<my_password>"

2. Append the static IP address we generated before to the cf-values.yml file. This is to help assign the ingress gateway of the Kubernetes installation.

echo "istio_static_ip: \"$IP\"" >> $TMP_DIR/cf-values.yml

Step 6:

Next, use the ytt tool to generate the final template to be used for the installation.

ytt -f config -f $TMP_DIR/cf-values.yml > $TMP_DIR/cf-for-k8s-rendered.yml

Step 7:

Finally, use the kapp command to install cf-for-k8s on your Kubernetes cluster.

kapp deploy -a cf -f $TMP_DIR/cf-for-k8s-rendered.yml -y

cf-for-k8s Install

Wait for the installation to complete. It takes an estimated 8–10 minutes to complete.

After installation

CF Push Using cf-for-k8s

The ability to connect with the Cloud Foundry API endpoint is the best validation of the installation. You can do this by setting the endpoint of your local cf CLI as follows

cf api --skip-ssl-validation api.$IP.xip.io

This step will allow you to verify if the cf installation is functional. Next, log in to the instance to put the instance to some real use.

cf login

Create an org for your use and create spaces within them. Set the new space as your “target” to which you will deploy your applications.

cf create-org SampleOrg
cf create-space -o SampleOrg dev
cf target -o SampleOrg -s dev

The simplest app you can push is the Node.js app that is included within the cf-for-k8s directory. Use this command:

cf push test-node-app -p tests/smoke/assets/test-node-app

This will push a test application to the Kubernetes clusters. Use the URL endpoint generated and open the application in your browser.

Conclusion

With this installation, you can help developers in your org realize the benefits of the simplified ‘cf push’ process while deploying to modern infrastructure. The cf-for-k8s community has got positive feedback for this abstraction layer over Kubernetes. There are several ongoing projects that aim to improve this offering.

The best way to connect with the Cloud Foundry community is to join our Slack Workspace at https://slack.cloudfoundry.org/. Those in the Slack community help you get quickly connected with other members or someone from the Cloud Foundry Foundation.