Kubernets with Prometheus and Grafana on HetznerCloud
Recently I started to mess around with kubernetes. I did it many times in the last year, but actually never build anything bigger with it. As I’m writing this I’m just starting out with the whole thing. I think I will make an article-series out of this, as it might be too much for one article. Some parts of this article are based on the work of xetys (David Steiman) (obviously we will be using his CLI) and on his article at his blog.
Table of Contents
- Let’s get started
- hetzner-cube configuration
- deploying the dashboard
- deploying rook for storage
- deploying prometheus
- deploying grafana
- configuring grafana
- importing the default prometheus dashboard
Let’s get started
First, you will need an account at https://www.hetzner.com/cloud. Thereafter you have to create a Project via “Add Project” inside the CloudConsole we will name it demo. Then you need to create an API-Token through “Access” > “API-Tokens” > “Generate API Token”, store it somewhere safe, as it’s only displayed once.
Now let’s configure the hetzner-cube cli (currently you need go installed to use it).
Open the shell of you favor (I’m using the Windows PowerShell, as I’m doing the setup on Windows).
After that you need to add an ssh-key:
Now we are ready to spin up the virtual machines and install the needed dependencies. The best thing about it: hetzner-cube does this for us:
This will create a simple cluster with one master and one worker. Of course, this is not the high-availability setup-method, but It’s the cheapest one (each instance costs 0,005 EUR per hour if you delete them you pay only the used hours). This will take a few minutes. When it’s done you will see to servers under the “Servers” Tab on the Dashboard:
We will configure our local kubectl to work with the created cluster:
deploying the dashboard
Since the cluster is up and running and our local kubectl is running we can deploy the kubernets-dashboard to see what's going on. If you don’t need the dashboard you can skip to the next step. (Based on this GitHub issue, thanks to radutopala)
To confirm it worked run:
The second entry is the one we are after. Time to create a user for it: Create a YAML file (“admin-user.yaml”) with the following contents:
and execute it:
We need to bind the ServiceAccount to the ClusterRole “cluster-admin”, so we need another file (“admin-user-cluster-role.yaml”) (you could also use the existing one as this are one-time commands):
great. The last step is to create the login token:
As with the Hetzner-API-Token, store this token somewhere safe, as it’s the login-token for the kubernetes-dashboard.
Now we can open a local proxy to the dashboard through the kubectl command:
If you open 127.0.0.1:8001/api/v1/namespaces/kube-system/services/https:kubernetes-dashboard:/proxy/#!/login in your browser you should see the dashboard login. Select token and enter the previously created one.
deploying rook for storage
To deploy the rook storage we can use hetzner-cube again:
This should complete in a few seconds. In the dashboard under “Storage Classes” you should seek “rook-block”, and there should be a “rook”-namespace under “Namespaces”.
Of course, kubernetes has its own package-manager called helm. So it’s pretty straight forward to deploy prometheus. First, install the helm-addon:
After that, you need to get a local copy of the prometheus-chart as we need to modify some values.
Edit the “values.yaml” file, and change the following: Inside the “server”-block is a “persistentVolume”-block which holds the “storageClass”-property. We uncomment it and change it to “rook-block”, as we are using “rook” as our storage provider. Makes sense, doesn’t it?
Now let’s deploy prometheus with the help of helm:
This should be pretty fast, you can check if it worked by visiting the dashboard. You should have a “prometheus” namespace to select inside the “Namespace” dropdown. Also, there should be two “Persistent Volumes” under the appropriate menu-point.
To our luck, there is also a chart for grafana. And again we are going to edit the “values.yaml” file:
Under the “persistence”-nlock we set the following:
Let’s deploy grafana with the help of helm:
Now we need to port-forward the grafana-dashboard to our local machine, for that we need the instance name of grafana, we can get it through:
and then when can open the proxy:
Now we can navigate to http://127.0.0.1:3000/ and we should see the grafana UI.
In the dashboard go to “Add data source” and enter the following:
Click “Save & Test”.
importing the default prometheus dashboard
Select the freshly created Kubernetes Datasource and go to the “Dashboard”-Tab. Click on Import on the desired dashboards (ex. “Prometheus 2.0 Stats”). If you go back to the Dashboards-Site and click on the dropdown labeled “Home” you can now select “Prometheus 2.0 Stats”. This dashboard shows stats about prometheus itself, we want to see the kubernetes metrics.
To import them we click on “+”-sign an select “Import”. We are going to use a dashboard hosted hat the garfana-dashboard store called Kubernetes All Nodes. Just paste the URL inside the input and you should get a new interface.
You could change the name if you want. In the “prometheus”-input select our created “Kubernetes” Datasource and click “Import”.
And that's it. You can now play around with the whole cluster. If you would like to delete the whole thing and don’t pay any more just run the following command:
While I was writing this the cluster costs me about 0,03 EUR. Thanks for reading and till next time!