Rancher deployed Kubernetes on Oracle Cloud Infrastructure

Part two of a multi-part series on creating, monitoring, and managing Kubernetes clusters (hosted and non-hosted) on OCI.

Published in

The Startup

5 min readMay 6, 2020

In the previous article, we looked at the Oracle Container Engine for Kubernetes (OKE) Cluster Driver for Rancher, which enables Rancher to create hosted Kubernetes clusters where the cluster’s control plane components are managed for you by Oracle Cloud Infrastructure (OCI).

In this article, we’ll take a look at the new Oracle Cloud Infrastructure Driver included in Rancher 2.4.3+, which enables Rancher to create Kubernetes clusters out of raw compute and network resources on OCI.

Both cluster types can integrate with other infrastructure services such as dynamic load-balancing and block storage volumes (a topic for a subsequent post). Both can automatically recreate failed nodes. The main difference is about who runs and manages the control plane for Kubernetes (i.e. API server, scheduler, resource controllers, and etcd).

If You Already Run Rancher

If you’re already running Rancher locally or inside a Kubernetes cluster in HA mode and you just want to create clusters from Rancher, skip ahead and start from the Virtual Cloud Network section (after you create and configure your Oracle Cloud account).

Before You Begin

Try the steps in this article yourself by signing up for an Oracle Cloud Free Tier account and complete the following steps:

Generate an API signing key pair:

$ openssl genrsa -out ~/.oci/oci_api_key.pem 2048
$ chmod go-rwx ~/.oci/oci_api_key.pem
$ openssl rsa -pubout -in ~/.oci/oci_api_key.pem -out ~/.oci/oci_api_key_public.pem
$ cat ~/.oci/oci_api_key_public.pem | pbcopy

Sign in to the OCI console using your email and password. Navigate to Identity > Users > User Details > API Keys > Add Public Key and Paste the Public Key captured above.
Create a Compartment in the desired region by choosing Identity > Compartment > Create Compartment.

VCN Creation and Configuration

Before we can create a cluster, we need to create a Virtual Cloud Network (VCN) where we’ll deploy the clusters. Sign back in to the OCI console and navigate to Networking > Virtual Cloud Network. Select a compartment and click on the Networking Quickstart button. Choose the VCN with Internet Connectivity option. Enter a name and accept the defaults.

Additional Security List Configuration

Rancher requires that the cluster nodes created by the driver be reachable by the Rancher server. In the VCN in which you want to create clusters, create the following rules in the security lists of the node subnets.

Ingress:

22 (SSH) from the Rancher source IP to the node(s) being created (try starting with a CIDR of 0.0.0.0/0)
2376 (Docker) from the Rancher source IP to the node(s) being created (again, try starting with a CIDR of 0.0.0.0/0)
2376, 2379, 2380, 6443, and 10250 between all the node(s) (nodes will use private IP addresses for internal communication regardless if they use public IPs)

Egress:

Access to the Internet 0.0.0.0/0 to download scripts, and to pull images from the Docker repository (a private repo can be configured)

Your nodes can be created with public or private IP addresses, according to whether you specify public or private subnets when defining the templates in the subsequent section.

Create Cloud Credentials

Rancher uses cloud credentials to store your credentials for an OCI account, including the API signing key pair you created above.

Click the user profile button in the upper right corner and add an OCI Cloud Credential.

Create Node Template(s)

Rancher uses node templates to create the worker and control plane nodes that make up your cluster. When you create node templates, you specify configuration parameters like the availability domain, instance shape, and whether your nodes will attach to public or private subnets.

Click the user profile button in the upper right corner to add an OCI Node Template. Multiple templates allow for different configurations for the nodes in your cluster. For example, you could create a set of templates for each availability domain for the control plane nodes and do the same for the worker nodes.

Create multiple node templates with different options for different role nodes in your cluster

Deploy Kubernetes Cluster on OCI

Return to the Rancher Global view and click on the Add Cluster button. Select Oracle Cloud Infrastructure as your provider to begin the cluster creation process.

Choose the Oracle Cloud Infrastructure Provider

Configure the nodes in your cluster by clicking on the Add Cluster button and assigning templates to each of the roles (etcd, Control Plane, Worker). Rancher requires at least one node be assigned to each role. You can increase the number of each node that has the same configuration by adjusting the Count.

Assign each of the 3 roles to at least one node

In the Kubernetes Options, configure the version of Kubernetes you want as well as your choice for the network provider. The Private Registry Option optionally allows you to pull the images for cluster provisioning from a private Docker registry.

OCI Cloud Provider

In order to give yourself the option of dynamically provisioning load-balancers and block storage volumes from Kubernetes, ensure the cloud provider to external and the kubeproxy health check binds to all interfaces.

The easiest way to configure this is to choose the Edit as YAML option for the Cluster Options and ensure the following information is set:

rancher_kubernetes_engine_config:
...
  cloud_provider:
    name: external
  ...
  services:
    kubeproxy:
      extra_args:
        healthz-bind-address: 0.0.0.0

Once your cluster configuration is complete, click the Create button at the bottom of the page.

Cluster Created!

If everything goes as planned, the cluster will soon show as Active. If you have issues, double check the ingress/egress rules of your VCN’s security lists. Try temporarily opening up the source CIDRs to 0.0.0.0/0. If that fails, post a comment below and I’ll do my best to help! Jesse Millan

Wrapping Up

The cluster driver for Oracle’s hosted Kubernetes (OKE) is still the quickest and easiest path of launching Kubernetes on OCI from Rancher. However, the new infrastructure driver gives Rancher the ability to deploy Kubernetes clusters directly to compute and network resources. Be on the lookout for driver enhancements and follow-up posts including how to use the Cluster Configuration (YAML) to enable dynamic load-balancing and block storage volumes in your Rancher deployed clusters.

NOTE: When a cluster is deleted from Rancher, the corresponding compute image resources are also deleted. It is a good idea to periodically check the OCI console for stray resources — especially if you encounter unexpected quota issues or if your cluster or nodes failed to create.