One does not simply deploy Kubernetes to the cloud

Boromir

Creating an Oracle Container Engine (OKE) cluster manually can be a time consuming task. At a minimum, you need to create the following:

• A VCN, an Internet Gateway, a NAT gateway if you want to a private worker node deployment
• worker subnets with a set of security rules
• load balancer subnets with its security rules

All of this is very well documented here.

Assuming you have done everything correctly, you can now create a cluster and node pools.

If you need a cluster quickly, say for an experiment or a demo, you can also use the Quick Create option in the OCI console. But why do things manually when you can actually code it?

Introducing terraform-oci-oke

We recently published on github the terraform-oci-oke s̶a̶m̶p̶l̶e̶-̶o̶k̶e̶-̶f̶o̶r̶-̶t̶e̶r̶r̶a̶f̶o̶r̶m̶ (I̶’̶l̶l̶ ̶t̶r̶y̶ ̶t̶o̶ ̶m̶a̶k̶e̶ ̶i̶t̶ ̶l̶e̶s̶s̶ ̶a̶ ̶m̶o̶u̶t̶h̶f̶u̶l̶ ̶i̶n̶ ̶t̶h̶e̶ ̶f̶u̶t̶u̶r̶e̶ done) that uses Terraform and the Terraform OCI provider to automate the OKE deployment.

Some of the features include the following:

• provisioning of all the basic requirements such as VCN, gateways, subnets, security lists etc
• configurable network settings for your worker and load balancer subnets
• optional provisioning of your bastion with git, oci-cli, kubectl and helm installed and configured
• choice of running your worker nodes in private or public mode
• configurable number of node pools, node pool size, worker node shapes, and topologies
• optional addons such as the kubernetes dashboard, calico for network policy, helm
• optional creation of Authentication Token, which is used to create a Kubernetes Secret. You can then use the secret as an imagePullSecrets so that OKE worker nodes can pull images from private OCIR repos.

Let’s take it for a spin. Below is an example of what you can create with the terraform-oci-oke scripts.

OKE Cluster with Bastion

I’ll be assuming that you have (a) already configured your OCI account for terraform (b) created the policy for creating an OKE cluster. If not, follow the instructions here first.

First, clone the repo:

git clone https://github.com/oracle-terraform-modules/terraform-oci-oke.git tfoke
cd tfoke

Copy the terraform.tfvars.example to terraform.tfvars and edit the terraform.tfvars:

cp terraform.tfvars.example terraform.tfvars

Enter the following information:

• api_fingerprint
• api_private_key_path
• compartment_name
• compartment_ocid
• tenancy_ocid
• user_ocid
• ssh_private_key_path
• ssh_public_key_path

If you want to create the authentication token and the Kubernetes secret, you also need to provide the following:

• email_address
• tenancy_name
• username

All the terraform options are fully documented here.

You can now run terraform plan and apply:

terraform plan
terraform apply -auto-approve

By default, a cluster of 3 worker nodes will be created for you unless you changed the node_pools andnode_pool_quantity_per_subnet parameters.

When it’s done, terraform will print the following for you:

bastion_public_ips = XXX.XXX.XXX.XXX
kubeconfig = export KUBECONFIG=generated/kubeconfig
ocirtoken = sensitive
ssh_to_bastion = ssh -i ~/.ssh/id_rsa opc@XXX.XXX.XXX

The kubeconfig file will also be created under the generated directory so you can just set your KUBECONFIG environment variable and start using kubectl:

export KUBECONFIG=generated/kubeconfig

And verify that you can interact with the cluster:

kubectl get nodes
NAME        STATUS   ROLES   AGE   VERSION                                                                                                                                  
10.0.13.2   Ready    node    20h   v1.12.6                                                                                                                                  
10.0.23.2   Ready    node    20h   v1.12.6                                                                                                                                  
10.0.33.2   Ready    node    20h   v1.12.6

Access the dashboard:

demo/dashboard.sh
Access K8s Dashboard: http://localhost:8001/api/v1/namespaces/kube-system/services/https:kubernetes-dashboard:/proxy/

Open your browser and access the dashboard: http://localhost:8001/api/v1/namespaces/kube-system/services/https:kubernetes-dashboard:/proxy/ and use the kubeconfig in the generated folder to login.

Using the bastion

The bastion is installed with oci-cli and helm and also has kubectl and helm auto-completion. Just login and start interacting with your cluster:

ssh -i ~/.ssh/id_rsa opc@XXX.XXX.XXX
kubectl get nodes
NAME        STATUS   ROLES   AGE   VERSION                                                                                                                                  
10.0.13.2   Ready    node    20h   v1.12.7                                                                                                                                  
10.0.23.2   Ready    node    20h   v1.12.7                                                                                                                                  
10.0.33.2   Ready    node    20h   v1.12.7

You can turn on/off the bastion anytime.

Reusing, addons and community

The project is also designed to be reusable, which means you can clone and add your own private customizations.

Over the course of coming months, we also want to add a few addons to the project that will automate the deployment of additional software to the cluster and we want to do this as a community effort. Some of the addons we are thinking include the following:

• Istio
• MySQL operator
• API Gateways such as Ambassador and Kong
• WebLogic Operator

You can see the full list here. Of course, if you have an idea for an addon, we would love to hear from you. Or better, send us a pull request.