Provision the K8S Cluster
This article is the first one in the Running Kafka in Kubernetes publication. In this article, you will see the how setup a new Rancher cluster with 2 nodes, and configure NFS mount on each of those nodes.
EC2 Instances
We use EC2 in Oregon region for all our servers.
Here is the what we use:
- kafka-rancher-nfs-server. This is a t2.micro Ubuntu 18.04 instance that we use to run NFS server.
- kafka-rancher-master. This is a t2.medium Ubuntu 18.04 instance that we use to run Rancher master cluster.
- kafka-rancher-kafka-1. This is a t2.medium Ubuntu 18.04 instance that we use to run the control-plane, etcd, and worker role for our kafka cluster.
- kafka-rancher-kafka-2. This is a t2.medium Ubuntu 18.04 instance that we use to run the worker role for our kafka cluster.
Here you go, the cluster is up. To get more details on how to provision Cluster, you can refer to https://rancher.com/docs/rancher/v2.x/en/cluster-provisioning/.
Install NFS Server
Now, you will see how we configure NFS server and mount the NFS into each node. Thanks to https://www.tecmint.com/install-nfs-server-on-ubuntu/ for the hints to configure NFS.
Login to kafka-rancher-nfs-server and run these commands
sudo apt update
sudo apt install nfs-kernel-server
sudo mkdir -p /mnt/nfs_share
sudo chown -R nobody:nogroup /mnt/nfs_share
sudo chmod 777 /mnt/nfs_share/
sudo vim /etc/exports
Add this line in the /etc/exports file
/mnt/nfs_share 172.31.0.0/16(rw,sync,no_subtree_check,no_root_squash)
172.31.0.0
is the private IP range of the four EC2 instances mentioned above.
Continue with these commands
sudo exportfs -a
sudo systemctl restart nfs-kernel-server
sudo ufw allow from 172.31.0.0/16 to any port nfs
Install NFS Client in Rancher Nodes
The NFS server is fully configured. Now, let’s configure the NFS client.
Login to the kafka-rancher-kafka-1 and kafka-rancher-kafka-2 and run the following commands
sudo apt update
sudo apt install nfs-common
sudo mkdir -p /mnt/nfs_clientshare
sudo mount 172.31.19.148:/mnt/nfs_share /mnt/nfs_clientshare
Verify the NSF Folder
Nothing is certain until the verification is done.
Create these folders and files in NFS Server
Verify the two files exist in both nodes
Conclusion
You have just seen how we provisioned the Rancher clusters and and configure NFS Server and Client.
Do take a look at the next article https://medium.com/kafka-in-kubernetes/automating-storage-provisioning-41034e570928