Deploy a KEEP Node to Kubernetes and Monitor with Prometheus & Grafana

4 min readJul 29, 2020

This guide aim to walk you through the steps involved in:

Getting you KEEP ECDSA node running in Kubernetes with Helm
Monitoring with Prometheus metrics
Installing a dashboard in Grafana

This guide is aimed at a slightly more technical audience, and makes some assumption. If you’re just getting started with running KEEP nodes one of the many VPS guides linked in the #staking-resources channel will likely be more suitable

This guide has developed running against a test local Kubernetes cluster (k3d), however this guide will be applicable for any Kubernetes cluster whether hosted in a public cloud or bare metal. You will likely want to adjust some of the parameters for your specific cloud in areas such as load-balancing and persistent volume configuration, please see the chart configuration for options to support this.

You can also adjust this guide for deploying the core/beacon node using the alternative keep-client chart.

Prerequists

This guide assumes some pre-existing setup these assumption are:

K8s cluster running and accessible with kubectl
helm installed, see here for guide
Ethereum wallet authorised and funded with KEEP and ETH
KEEP Docker images with metrics support (all recent builds should be fine or you can build your own)
Up to date version of golang installed on your system

Prometheus Install

You can skip this section if you already have Prometheus configured within your cluster

To get Prometheus & Grafana setup quickly we’re going to install the prometheus-operator chart. You can view the chart for all the available options, but for our example a default install is enough.

We use the Helm chart from the stable repo and install with all defaults. This will enable Prometheus, Grafana and a host of other exports for Kubernetes.

$ helm repo add stable https://kubernetes-charts.storage.googleapis.com/
$ kubectl create ns monitoring
$ helm install -n monitoring prometheus-operator stable/prometheus-operator

Node Setup

For the node setup we’re again going to be using Helm to make installation of our node simple and easy to upgrade in the future.

This install is using one approach for providing the config & wallet please see the README of the chart repo for examples and more information

Start by downloading the charts locally

$ git clone https://github.com/ajcrowe/keep-helm-chart
$ cd keep-helm-chart/keep-ecdsa

Create a local.yaml and copy you wallet locally to wallet.json

$ touch local.yaml
$ cp /path/to/my/wallet.json wallet.json

Edit the local.yaml with the required parameters. You can find all available options here

You will need to replace the Ethereum configuration with valid endpoints and specify the Ethereum address of you wallet.

# set our Ethereum backend settings
config:
  ethereum:
    url: "wss://<network>.infura.io/ws/v3/<mykey>"
    urlRpc: "https://<network>.infura.io/v3/<mykey>"
    account: "0xmyaddress"# set your wallet password
# or set with --set keyPass.password="password" on install
keyPass:
  password: "password"# enables the metrics section of our config.toml
metrics:
  enabled: true# enables the creation of a ServiceMonitor resource
serviceMonitor:
  enabled: true
  additionalLabels
    release: prometheus-operator# set an image which has metrics support builtin
image:
  repository: ajcrowe/keep-ecdsa
  tag: master

Once these are in place you can install the chart with

$ helm install keep-ecdsa -f local.yaml .

View the logs to validate the node is running and connected with the network

$ kubectl logs -f -l app.kubernetes.io/instance=keep-client
...
22:43:06.883  INFO keep-net-l: number of connected peers: [83] libp2p.go:237
...

Grafana Dashboard

This section covers getting a Grafana dashboard installed to provide a quick overview of our nodes health and performance

Start by installing required tools and cloning the dashboard template

go get -u github.com/grafana/grizzly/cmd/grr
go get -u github.com/jsonnet-bundler/jsonnet-bundler/cmd/jbgit clone https://github.com/ajcrowe/keep-grafana-dashboard
cd keep-grafana-dashboard

Next we need to access Grafana. We can do with this the kubectl port-forward command to access our clusters local instance.

For a production setup you will want to use an Ingress or LoadBalancer service to expose this or ship your metrics to a hosted service like Grafana Cloud

$ kubectl port-forward svc/prometheus-operator-grafana 8080:80 -n monitoring
Forwarding from 127.0.0.1:8080 -> 3000
Forwarding from [::1]:8080 -> 3000

Connect to http://localhost:8080and login with admin / prom-operator. At this point you can validate you’re recieving metrics from the node

Select “Explore” from the menu

Then enter connected_peers_count and run the query, you should see something similar to this

With this verified we can install our dashboard

$ export GRAFANA_URL="http://admin:prom-operator@localhost:8080
$ grr apply keep-dashboard.jsonnet

You can then browse to the newly installed dashboard and you should be presented with something like this

Final Thoughts

This guide was aiming to get you up and running with a basic Kubernetes install and a starter dashboard. You will likely want to modify and add additional functionality to your node in areas such as alerting and network configuration.

If you have any feedback, or suggestions on specific areas you’d like future guides please let me know in the comments or dia the KEEP discord my handle is ajc