Creating High Available Baremetal Kubernetes cluster with Kubeadm and Keepalived (Simple Guide)

kvaps

This guide is a free interpretation of official Creating Highly Available Clusters with kubeadm for Stacked control plane nodes. I don’t like this difficult form which used there, so I wrote this article.

If you have any questions, or something is not clear, please refer to the official documentation or ask the Google. All steps described here in the short and simple form.

Input data

We have 3-nodes cluster:

  • node1 (10.9.8.11)
  • node2 (10.9.8.12)
  • node3 (10.9.8.13)

We will create one fault-resistant cluster-ip for them:

  • 10.9.8.10

Then install etcd and kubernetes cluster on them.

LoadBalancer setup

First, we need to install keepalived on all three nodes:

apt-get -y install keepalived

Then write config /etc/keepalived/keepalived.conf:

vrrp_instance VI_1 {
state MASTER
interface eth0
virtual_router_id 1
priority 100
advert_int 1
authentication {
auth_type AH
auth_pass iech6peeBu6Thoo8xaih
}
virtual_ipaddress {
10.9.8.10
}
}

Enable and start keepalived on all thee nodes

systemctl start keepalived
systemctl enable keepalived

Now we can check that one of node have 10.9.8.10 address on eth0 interface.

Deploying Kubernetes cluster

UPD: This article was written for v1.12, and despite the fact that it was adapted for v1.13, now creating cluster is much easier, check this more simple guide.

Make sure that you have latest kubernetes packages installed on all nodes:

apt-get -y install kubeadm kubelet kubectl

Also stop keepalived daemon on all nodes except the last one:

systemctl stop keepalived

First node

Now we will generate kubeadm-configs (we need separated config for each master node):

CLUSTER_IP=10.9.8.10
NODES=(node1 node2 node3)
IPS=(10.9.8.11 10.9.8.12 10.9.8.13)
POD_SUBNET="192.168.0.0/16"
for i in "${!NODES[@]}"; do
HOST=${IPS[$i]}
NAME=${NODES[$i]}
INITIAL_CLUSTER=$(
for j in "${!NODES[@]}"; do
echo "${NODES[$j]}=https://${IPS[$j]}:2380"
done | xargs | tr ' ' ,
)
cat > kubeadm-config-${NODES[$i]}.yaml <<EOT
apiVersion: kubeadm.k8s.io/v1beta1
kind: ClusterConfiguration
kubernetesVersion: stable
apiServer:
certSANs:
- "${CLUSTER_IP}"
controlPlaneEndpoint: "${CLUSTER_IP}:6443"
etcd:
local:
extraArgs:
initial-cluster: "${INITIAL_CLUSTER}"
initial-cluster-state: new
name: ${NODES[$i]}
listen-peer-urls: "https://${IPS[$i]}:2380"
listen-client-urls: "https://127.0.0.1:2379,https://${IPS[$i]}:2379"
advertise-client-urls: "https://${IPS[$i]}:2379"
initial-advertise-peer-urls: "https://${IPS[$i]}:2380"
serverCertSANs:
- "${NODES[$i]}"
- "${IPS[$i]}"
peerCertSANs:
- "${NODES[$i]}"
- "${IPS[$i]}"
networking:
podSubnet: "${POD_SUBNET}"
EOT
done

Init etcd on first node, generate ca certs and admin config

kubeadm="kubeadm --config=kubeadm-config-${HOSTNAME}.yaml"
$kubeadm init phase preflight
$kubeadm init phase certs all
$kubeadm init phase kubelet-start
$kubeadm init phase kubeconfig kubelet
$kubeadm init phase etcd local
$kubeadm init phase kubeconfig admin
systemctl start kubelet

Copy generated certs and kubeadm configs to other control plane nodes

NODES="node2 node3"
CERTS=$(find /etc/kubernetes/pki/ -maxdepth 1 -name '*ca.*' -o -name '*sa.*')
ETCD_CERTS=$(find /etc/kubernetes/pki/etcd/ -maxdepth 1 -name '*ca.*')
for NODE in $NODES; do
ssh $NODE mkdir -p /etc/kubernetes/pki/etcd
scp $CERTS $NODE:/etc/kubernetes/pki/
scp $ETCD_CERTS $NODE:/etc/kubernetes/pki/etcd/
scp /etc/kubernetes/admin.conf $NODE:/etc/kubernetes
scp kubeadm-config-$NODE.yaml $NODE:
done

Second node

Init etcd on second node

kubeadm="kubeadm --config=kubeadm-config-${HOSTNAME}.yaml"
$kubeadm init phase preflight
$kubeadm init phase certs all
$kubeadm init phase kubelet-start
$kubeadm init phase kubeconfig kubelet
$kubeadm init phase etcd local
systemctl start kubelet

Third node

Init kubernetes master with the last etcd on the last node:

(make sure that you have loadbalanser ip targeted to this node)

kubeadm init --config kubeadm-config-${HOSTNAME}.yaml

First and Second nodes

Now we can deploy kubernetes master on the first two nodes

kubeadm="kubeadm --config=kubeadm-config-${HOSTNAME}.yaml"
$kubeadm init phase kubeconfig all
$kubeadm init phase control-plane all
$kubeadm init phase mark-control-plane

And start keepalived daemon also:

systemctl start keepalived

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade