CoreOS Etcd and Fleet with Encryption and Authentication

This tutorial gives you a complete walk through on how to get your CoreOS cluster up and running with Etcd Client-to-server transport security over HTTPS, client-to-server authentication with HTTPS client certificates, and Transport security & client certificates in a cluster (Etcd peer to peer communication). You’ll learn how to set up your own CA (Certificate Authority), generate certificates for each node and clients, configure CoreOS nodes, and finally interact with Etcd with the authentication and certs.

This tutorial is based on examples found in this article: https://github.com/coreos/etcd/blob/master/Documentation/security.md

Here is a CoreOS tutorial on Etcd-security:

https://coreos.com/docs/distributed-configuration/etcd-security/

We’ll use the following four steps to achieve our goal.

1. Generate certs

First you’ll need to create a Certificate Authority (CA), and then generate certificates specific to each host in your cluster and clients that will be using Etcd.

For more information, you may consult this useful site https://github.com/coreos/etcd-ca.

Use the following commands to setup a CA, and initialize it. Make sure that you have “docker” running either locally or remotely.

root@myserver:/# docker pull golang
root@myserver:/# docker run -i -t golang /bin/bash
root@docker:/# git clone https://github.com/coreos/etcd-ca
root@docker:/# cd etcd-ca
root@docker:/# ./build
root@docker:/# ./bin/etcd-ca init

2. Generate certificates for each host

Now that you’ve your CA up and running, you’re ready to generate certificates. Continuing in the same docker container, issue the following commands to generate certificates.

Note: The IP address and the hostname (host1.example.com) must be same as what your CoreOS node is configured to. What happens is that when Etcd validates the SSL authentication, it will match it up to the IP address and hostnames. If these change on your host, you’ll need to generate a new set of keys for that particular host.

Let’s generate certs for each host using the CLI commands below.

root@myserver:/# cd bin
root@myserver:/# ./etcd-ca new-cert —ip 198.27.74.208 host1.example.com
root@myserver:/# ./etcd-ca sign host1.example.com

Use the following command to output the host ca chain, and save this file as: ca.crt.

root@myserver:/# ./etcd-ca chain host1.example.com

Use the following command to output host1’s certificate and private key. Then, save the certificate as “key.crt” and the private key as “key.key”.

root@myserver:/# ./etcd-ca export —insecure host1.example.com

3. Write cloud configuration for each host

Here, we will write cloud configurations for each host. You’ll use this cloud-config file to startup your CoreOS host1. For this, you’ll need your CA certificate (ca.crt) and signed key pair (key.crt, key.key)

1. Repeat step 2 from above.
2. Fill out the keys in the cloud-config below.

4. Check to see that everything works

The following commands will setup Etcd testing parameters. After that, you can proceed with Etcd configurations to see that everything is working properly.

On your command prompt, export out these environment variables so they can be used in the cURL commands below. Just cut and paste these into your command prompt. You will need to fill out the correct IPs where your certs are located. You can do this on one of the CoreOS nodes, or you can do this on your local computer (make sure that your local computer can reach these IPs).

LEADER_IP=198.27.74.208
FOLLOWER_IP=142.4.216.210
KEY_CA_CERT=/home/core/ca.crt
KEY_CERT=/home/core/key.crt
KEY_KEY=/home/core/key.key

Etcd Tests

You can now test the configuration using HTTPS:

• Test one — making a call to the leader

curl —cacert ${KEY_CA_CERT} —cert ${KEY_CERT} —key ${KEY_KEY} -L https://${LEADER_IP}:4001/v2/keys/foo2 -X PUT -d value=bar -v

• Test two — making a call and getting the leader’s IP

curl —cacert ${KEY_CA_CERT} —cert ${KEY_CERT} —key ${KEY_KEY} -L https://${LEADER_IP}:4001/v2/leader -v

• Test three — making a call to a follower that will be automatically redirected to the leader.

curl —cacert ${KEY_CA_CERT} —cert ${KEY_CERT} —key ${KEY_KEY} -L https://${FOLLOWER_IP}:4001/v2/keys/foo2 -X PUT -d value=bar -v

• Test four — getting a value from the leader

curl —cacert ${KEY_CA_CERT} —cert ${KEY_CERT} —key ${KEY_KEY} -L https://${LEADER_IP}:4001/v2/keys/foo2 -X GET -v

• Test five — getting a value from a follower

curl —cacert ${KEY_CA_CERT} —cert ${KEY_CERT} —key ${KEY_KEY} -L https://${FOLLOWER_IP}:4001/v2/keys/foo2 -X GET -v

Fleet Test

• Test one: listing machines in the CoreOS cluster

fleetctl —endpoint=https://${FOLLOWER_IP}:4001 —etcd-cafile=${KEY_CA_CERT} —etcd-certfile=${KEY_CERT} —etcd-keyfile=${KEY_KEY} list-machines

Conclusion

We now have a CoreOS cluster up and running with fully-encrypted Etcd peer to peer communication and Etcd client-to-server transport security over HTTPS as well as, Etcd client-to-server authentication with HTTPS client certificates.

—

ManagedKube built k8sBot, a point-and-click interface for Kubernetes in Slack. Now, software developers and novice k8s users can get meaningful Kubernetes information with just one click in Slack, where you’re already talking with your team. With k8sBot, you can retrieve pod status, get pod logs, and get troubleshooting recommendations based on real-time information from your cluster’s Kubernetes API.

Learn more and start a free trial of k8sBot at managedkube.com