Setting up kubernetes cluster in AWS with Private topology using KOPS 1.5.1.

Introduction:

KOPS is open source tool for deploying Kubernetes cluster with Amazon Web Services. In this blog, we will focus on launching cluster in private topology.

Prerequisite:

  1. Linux hands on.
  2. AWS Account with necessary permissions.
  3. Access key and Secret key of AWS account.

Step 1 | Install KOPS 1.5.1

Download KOPS 1.5.1,

$ wget  https://github.com/kubernetes/kops/releases/download/1.5.1/kops-linux-amd64

Change Permissions,

$ chmod a+x kops-linux-amd64

Move KOPS to /usr/local/bin

$ sudo mv kops-linux-amd64 /usr/local/bin/kops

Confirm KOPS installation,

$ kops
kops is kubernetes ops.
It allows you to create, destroy, upgrade and maintain clusters.
Usage:
kops [command]
Available Commands:
completion Output shell completion code for the given shell (bash)
create Create a resource by filename or stdin
delete delete clusters
describe describe objects
edit Edit resource
export export clusters/kubecfg
get list or get objects
import import clusters
replace Replace a resource by filename or stdin
rolling-update rolling update clusters
secrets Manage secrets & keys
toolbox Misc infrequently used commands
update update clusters
upgrade upgrade clusters
validate Validate Cluster
version Print the client version information
Flags:
--alsologtostderr log to standard error as well as files
--config string config file (default is $HOME/.kops.yaml)
-h, --help help for kops
--log_backtrace_at traceLocation when logging hits line file:N, emit a stack trace (default :0)
--log_dir string If non-empty, write log files in this directory
--logtostderr log to standard error instead of files (default false)
--name string Name of cluster
--state string Location of state storage
--stderrthreshold severity logs at or above this threshold go to stderr (default 2)
-v, --v Level log level for V logs
--vmodule moduleSpec comma-separated list of pattern=N settings for file-filtered logging
Use "kops [command] --help" for more information about a command.
root@opcito-ThinkPad-E460:/home/opcito#

Step 2 | Install Kubectl

Download Kubectl,

$ wget https://storage.googleapis.com/kubernetes-release/release/v1.5.2/bin/linux/amd64/kubectl

Change Permissions,

$ chmod a+x kubectl

Move kubectl to /usr/local/bin,

$ sudo mv kubectl /usr/local/bin/kubectl

Confirm Kubectl installation,

$ kubectl 
kubectl controls the Kubernetes cluster manager.
Find more information at https://github.com/kubernetes/kubernetes.
Basic Commands (Beginner):
create Create a resource by filename or stdin
expose Take a replication controller, service, deployment or pod and
expose it as a new Kubernetes Service
run Run a particular image on the cluster
set Set specific features on objects
Basic Commands (Intermediate):
get Display one or many resources
explain Documentation of resources
edit Edit a resource on the server
delete Delete resources by filenames, stdin, resources and names, or
by resources and label selector
Deploy Commands:
rollout Manage a deployment rollout
rolling-update Perform a rolling update of the given ReplicationController
scale Set a new size for a Deployment, ReplicaSet, Replication
Controller, or Job
autoscale Auto-scale a Deployment, ReplicaSet, or ReplicationController
Cluster Management Commands:
certificate Modify certificate resources.
cluster-info Display cluster info
top Display Resource (CPU/Memory/Storage) usage
cordon Mark node as unschedulable
uncordon Mark node as schedulable
drain Drain node in preparation for maintenance
taint Update the taints on one or more nodes
Troubleshooting and Debugging Commands:
describe Show details of a specific resource or group of resources
logs Print the logs for a container in a pod
attach Attach to a running container
exec Execute a command in a container
port-forward Forward one or more local ports to a pod
proxy Run a proxy to the Kubernetes API server
cp Copy files and directories to and from containers.
Advanced Commands:
apply Apply a configuration to a resource by filename or stdin
patch Update field(s) of a resource using strategic merge patch
replace Replace a resource by filename or stdin
convert Convert config files between different API versions
Settings Commands:
label Update the labels on a resource
annotate Update the annotations on a resource
completion Output shell completion code for the given shell (bash or zsh)
Other Commands:
api-versions Print the supported API versions on the server, in the form of
"group/version"
config Modify kubeconfig files
help Help about any command
version Print the client and server version information
Use "kubectl <command> --help" for more information about a given command.
Use "kubectl options" for a list of global command-line options (applies to all
commands).

Step 3 | Setting up AWS environment

Configure AWS with Secret Key and Access Key which is having necessary permissions as mentioned in prerequisite. You can also create the seperate user for KOPS with this script.

Configure AWS,

$ aws configure

Provide right Secret Key , Access Key and Region.

If you are using script [Which is more recommended],

$ curl -O https://raw.githubusercontent.com/kubernetes/kops/master/hack/new-iam-user.sh
$ sh new-iam-user.sh <group> <user>
$ aws iam list-users

Note down secret key and access-key for this user.

Step 4 | Create Hosted Zone for Cluster

We will require hosted zone associated with Route 53 which must be publically resolvable. (However, KOPS also allows to use private DNS which is more tricky.).

Create Hosted Zone. We are using Terraform..in,

$ aws route53 create-hosted-zone --name ikod.in --caller-reference 2017-02-24-11:12 --hosted-zone-config Comment="Hosted Zone for KOPS"

You can refer this blog to create public hosted zone and transfer your domain to Route 53.

Step 5 | Create State Store

KOPS internally uses Terraform . So we required external state store for storing states of a cluster. We are using Amazon S3 for storing state.

$ aws s3api create-bucket --bucket ikod-in-state-store --region us-west-2

Step 6| Creating Cluster

Setup environment variable for STATE STORE and cluster name NAME:

$ export NAME=cluster.ikod.in
$ export KOPS_STATE_STORE="s3://ikod-in-state-store"

Forming custom KOPS command to use private topology,

Parameters :

— cloud aws : We are launching cluster in AWS.

— zones us-west-2a,us-west-2b,us-west-2c : This describes availability zones for nodes. [Note that this availability zones should be available for AWS account. You can check it through, aws ec2 describe-availability-zones — region us-west-1 ] — node-count 5 : The number kubernetes nodes.

— node-size t2.medium : Size of kubernetes nodes.

— dns-zone ikod.in : Hosted zone which we created earlier.

— master-size m3.medium : Size of a Kubernetes master node.

— master-zones us-west-2a,us-west-2b,us-west-2c : This will tell kops to spread masters across those availability zones. Which will give High Availability to KOPS cluster.

— topology private : We define that we want to use a private network topology with kops.

— networking calico : We tell kops to use Calico for our overlay network.Overlay networks are required for this configuration.

— bastion : Add this flag to tell kops to create a bastion server so you can SSH into the cluster.

KOPS Command,

kops create cluster \
--cloud aws \
--node-count 5 \
--node-size t2.medium \
--master-size m3.medium \
--zones us-west-2a,us-west-2b,us-west-2c \
--master-zones us-west-2a,us-west-2b,us-west-2c \
--dns-zone ikod.in \
--topology private \
--networking calico \
--bastion \
${NAME}

Kops will default to ~/.ssh/id_rsa.pub for backend access. You can override this with — ssh-public-key /path/to/key.pub

Cluster configuration

Above command will create a blueprint for the cluster. You can edit configurations with,

$ kops edit cluster ${NAME}

Apply the changes,

$ kops update cluster ${NAME} --yes

Cluster creation will take up to 10 minutes.

Step 7| Verify cluster and Start using Cluster

Check your API’s are working with kubectl,

$ kubectl get nodes

It should provide you the number of nodes in response.

To access admin via bastion host,

$ exec ssh-agent bash
$ ssh-add ~/.ssh/id_rsa
$ ssh -A admin@bastion.cluster.ikod.in
$ ssh admin@<master_private_ip>
Show your support

Clapping shows how much you appreciated Jakir Patel’s story.