Multi-minion Kubernetes Cluster on Fedora Cloud on AWS

This blog is an extension of “Testing Kubernetes with an Atomic Host” by Jason Brooks. While Jason’s blog covers the essential steps to setup a cluster on Atomic host, I am going to focus primarily on setting up the cluster on Fedora cloud on AWS ie.,Fedora Base Cloud launched using one of the pre-configured AMIs.Though Atomic Host is pre-configured to run docker containers and has Kubenetes pre-installed, yum is disabled in these instances. Thus I prefer starting with Fedora Cloud base instance and then installing the required & latest packages. In addition, let us also have a quick view of managing the containers and images using Cockpit dashboard.
1. Launch Fedora Cloud on Amazon

From the AWS Console, choose the Fedora base cloud version say “ami-100b6f78″ AMIN . Launch 2 or more instances, one for the kubernetes master and the other(s) for the minion(s). Ensure the required TCP ports are opened across these instances in the AWS security groups.

2. Pre-configure instances

On the master and the minion instances, set a common root password for all the instances say “password” and bring down the firewall.

sudo su 
passwd 
systemctl disable iptables firewalld

On the master and the minion instances, install network managers.

yum install -y NetworkManager
systemctl status NetworkManager
systemctl start NetworkManager
systemctl enable NetworkManager

Ensure the instances can ssh to each other using password and a common root password is set across all the instances.

yum install -y sshpass
echo "password" > ~/rootpassword

Edit the /etc/ssh/sshd_config file on all the instances and set PasswordAuthentication

# To disable tunneled clear text passwords, change to no here!
PasswordAuthentication yes
#PermitEmptyPasswords no
#PasswordAuthentication no
Restart the ssh service.
service sshd restart
Check in the instances are reachable through ssh and ensure ~/.ssh/id_rsa.pub files exist in all the instances. If not follow the steps here to generate the ssh keys.
sshpass -p 'password' ssh root@
3. Install & Configure Cockpit
Cockpit helps to monitor and administer multiple linux systems through web browser. It provides jounal of the various operations performed across these systems. To bring up Cockpit, on the master and the minion instances, install cockpit package.
yum install -y cockpit
On the master, bring up the cockpit service.
yum install -y cockpit
systemctl enable cockpit.socket
systemctl start cockpit.socket
4. Install & Configure Kubernetes
Bring up Kubernetes cluster using Ansible following the same steps as for Atomic Host. In addition, due to Kubernetes bug #3187 , if the minion hostname is not the same as its IP address, when a pod is deployed, the pod gets scheduled but the deployment is stuck in “pending” state. As a temporary resolution, in the /etc/kubernetes/controller-manager file on the master, the kubelet_addresses configuration should contain the ‘hostname -f’ value instead of the IP address. ie.,
KUBELET_ADDRESSES=“—machines=minion-hostname-f“
Similarly, in the /etc/kubernetes/kubelet file on the minion, the kubelet_hostname configuration should contain the IP address of the minion. ie.,
KUBELET_HOSTNAME=“—hostname_override=minion-public-ip"
Once this is updated, restart the kubernetes master as below:
for SERVICES in etcd kube-apiserver kube-controller-manager kube-scheduler; do
systemctl restart $SERVICES
systemctl enable $SERVICES
systemctl status $SERVICES
done
Start the kubelet & kubelet-proxy on the minions as below:
for SERVICES in kube-proxy kubelet docker; do
systemctl restart $SERVICES
systemctl enable $SERVICES
systemctl status $SERVICES
done
Once the services come up, login to the cockpit server at http://master-ip:9090 using the root user and root password. Add the minions to the managed hosts. Now you can see the Docker containers running within the minions.