Kubernetes recipe: how to setup baremetal cluster (1.8.1) with Kubespray (2.3.0) and deploy Ingress Controller with Letsencrypt support on ScaleWay 2017-10-30
UPDATE 2017–11–18: Read new version of guide (with GlusterFS Native Storage Service)
1. Create new server on ScaleWay
Create new server on ScaleWay
Remember your Public IP (X.X.X.X) and Private IP (Y.Y.Y.Y)
Test connection to the server
ssh root@X.X.X.X
exit
2. Install requirements on local computer
virtualenv venv
source venv/bin/activate
pip install ansible netaddr
3. Install kubernetes cluster
Clone Kubespray
git clone https://github.com/kubernetes-incubator/kubespray.gitcd kubespraygit checkout -b tags/v2.3.0
Create ansible inventory file (replace X.X.X.X and Y.Y.Y.Y)
cat <<EOF > inventory/inventory.cfg
# Change ansible_ssh_host to your Public IP, ip to Private IP
master ansible_ssh_host=X.X.X.X ip=Y.Y.Y.Y[kube-master]
master[etcd]
master[kube-node]
master[k8s-cluster:children]
kube-node
kube-master
EOF
Change kubernetes definitions
vim ./inventory/group_vars/k8s-cluster.yml-kube_network_plugin: calico
+kube_network_plugin: flannel
Run kubernetes installation
ansible-playbook -u root -i inventory/inventory.cfg cluster.yml -b -v
Check installation
ssh root@X.X.X.Xkubectl get all --all-namespacesexit
4. Deploy Ingress Controller (nginx-ingress)
Login to the master
ssh root@X.X.X.X
Run mandatory commands (from https://github.com/kubernetes/ingress-nginx/blob/master/deploy/README.md#mandatory-commands)
curl https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/namespace.yaml \
| kubectl apply -f -
curl https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/default-backend.yaml \
| kubectl apply -f -
curl https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/configmap.yaml \
| kubectl apply -f -
curl https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/tcp-services-configmap.yaml \
| kubectl apply -f -
curl https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/udp-services-configmap.yaml \
| kubectl apply -f -
Install with RBAC roles (from https://github.com/kubernetes/ingress-nginx/blob/master/deploy/README.md#install-with-rbac-roles)
curl https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/rbac.yaml \
| kubectl apply -f -curl https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/with-rbac.yaml \
| kubectl apply -f -
Deploy Service (change Y.Y.Y.Y to your Private IP)
cat <<EOF | kubectl create -f -
apiVersion: v1
kind: Service
metadata:
name: ingress-nginx
namespace: ingress-nginx
spec:
type:
ports:
- name: http
port: 80
targetPort: 80
protocol: TCP
- name: https
port: 443
targetPort: 443
protocol: TCP
selector:
app: ingress-nginx
externalIPs:
- Y.Y.Y.Y
EOF
Check that nginx-ingress-controller and default-http-backend is running
kubectl get pods -n ingress-nginxNAME READY STATUS RESTARTS AGE
default-http-backend-66b447d9cf-4b8d9 1/1 Running 0 2m
nginx-ingress-controller-c5b844d96-vr6bk 1/1 Running 0 1m
Check that nginx-ingress-controller was binded to 80, 443 port
netstat -tuplntcp 0 0 Y.Y.Y.Y:80 0.0.0.0:* LISTEN 20771/hyperkube
tcp 0 0 Y.Y.Y.Y:443 0.0.0.0:* LISTEN 20771/hyperkube
Check default-http-backend
curl X.X.X.X
default backend - 404
4. Deploy Let’sencrypt Service for Ingress
Deploy kube-lego (change lego.email)
cat <<EOF | kubectl create -f -
apiVersion: v1
kind: Namespace
metadata:
name: kube-lego
---
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
name: ingress-secret-admin
rules:
- apiGroups: [""]
resources: ["secrets"]
verbs:
- get
- watch
- list
- create
- update
- patch
- apiGroups: [""]
resources: ["services"]
verbs:
- get
- create
- apiGroups: ["extensions"]
resources: ["ingresses"]
verbs:
- get
- watch
- list
- create
- update
- patch
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
name: kube-lego
roleRef:
kind: ClusterRole
name: ingress-secret-admin
apiGroup: rbac.authorization.k8s.io
subjects:
- kind: ServiceAccount
name: default
namespace: kube-lego
---
apiVersion: v1
metadata:
name: kube-lego
namespace: kube-lego
data:
# modify this to specify your address
lego.email: "your@email.com"
# configure letsencrypt's production api
lego.url: "https://acme-v01.api.letsencrypt.org/directory"
kind: ConfigMap
---
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: kube-lego
namespace: kube-lego
spec:
replicas: 1
template:
metadata:
labels:
app: kube-lego
spec:
containers:
- name: kube-lego
image: jetstack/kube-lego:0.1.5
imagePullPolicy: Always
ports:
- containerPort: 8080
env:
- name: LEGO_EMAIL
valueFrom:
configMapKeyRef:
name: kube-lego
key: lego.email
- name: LEGO_URL
valueFrom:
configMapKeyRef:
name: kube-lego
key: lego.url
- name: LEGO_NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
- name: LEGO_POD_IP
valueFrom:
fieldRef:
fieldPath: status.podIP
readinessProbe:
httpGet:
path: /healthz
port: 8080
initialDelaySeconds: 5
timeoutSeconds: 1
EOF
Check kube-lego is running
kubectl get pods -n kube-legoNAME READY STATUS RESTARTS AGE
kube-lego-674567867b-5wmfh 1/1 Running 0 25s
5. Deploy example application
Add an A record in your DNS and wait about three hours for your changes to propagate
Type: A
Name: echo.example.com (your domain name)
Value: X.X.X.X (Public IP)
Deploy echoserver (replace echo.example.com with your domain name)
cat <<EOF | kubectl create -f -
apiVersion: v1
kind: Namespace
metadata:
name: echoserver
---
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: echoserver
namespace: echoserver
spec:
replicas: 1
template:
metadata:
labels:
app: echoserver
spec:
containers:
- image: gcr.io/google_containers/echoserver:1.0
imagePullPolicy: Always
name: echoserver
ports:
- containerPort: 8080
---
apiVersion: v1
kind: Service
metadata:
name: echoserver
namespace: echoserver
spec:
ports:
- port: 80
targetPort: 8080
protocol: TCP
selector:
app: echoserver
---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: echoserver
namespace: echoserver
annotations:
kubernetes.io/tls-acme: "true"
kubernetes.io/ingress.class: "nginx"
spec:
tls:
- hosts:
# CHANGE ME
- echo.example.com
secretName: echoserver-tls
rules:
# CHANGE ME
- host: echo.example.com
http:
paths:
- path: /
backend:
serviceName: echoserver
servicePort: 80
EOF
Check for letsencrypt certificate
kubectl get pods -n kube-legoNAME READY STATUS RESTARTS AGE
kube-lego-674567867b-5wmfh 1/1 Running 0 25kubectl logs -f kube-lego-674567867b-5wmfh -n kube-lego
Check ingress resources
kubectl get ing --all-namespacesNAMESPACE NAME HOSTS ADDRESS PORTS AGE
echoserver echoserver echo.example.com Y.Y.Y.Y 80, 443 5m
kube-lego kube-lego-nginx echo.example.com Y.Y.Y.Y 80 5m
Test deployment
curl https://echo.example.com