Running Ethereum’s Execution & Consensus Nodes on Azure Kubernetes Service
Now that Ethereum’s merge is behind us, where Eth1 and Eth2 are now one single network with a consensus layer and an execution layer, the task of running your own node got a little bit harder. Instead of just running one piece of software that previously was responsible for everything (such as Go Ethereum or Parity), now only handles execution and transactions, and another piece that is responsible for the proof-of-stake consensus is required.
In this article we will be using Terraform to deploy a simple Azure Kubernetes Service, and in it we will be using Helm charts to start and run a Go Ethereum service as the execution node and Lighthouse service as the consensus node.
Prerequisites
Will be using Terraform and its azurerm provider, so we will be needing the following installed on our workstation:
As we will be interacting with a Kubernetes cluster and using Helm charts to deploy our service, it would be a good idea to also have on our workstation:
Example Repository
A complete example Terraform script, which creates a private network, an Azure Kubernetes Service cluster, and deploys the two services using Helm charts, is available in the following GitHub repository:
The script
For brevity, I will only cover the area of the Terraform script that specifically creates the cluster and the shared secret on the cluster, and only the Helm templates that are specific to Go Ethereum and Lighthouse.
We will start by creating an Azure Kubernetes Service cluster with a minimum version of 1.24, which is the first version to have the MixedProtocolLBService feature graduated from alpha status (it reached beta in 1.24, which makes it available in AKS). We need that feature because both our services need to expose TCP and UDP ports to the internet for discovery purposes.
Also, because we want to be able to use well known public IP addresses (again for discovery purposes), we will be using a custom virtual network (either an existing one or a new one), and we will need to grant the cluster’s identity the ability to access the security group that contains those IP addresses. Also, because the virtual network is not managed by Kubernetes, a Network Security Group with rules to allow communication to the private network from the internet will also be required.
Next, we will create a secret which will be alter mounted to the service as the jwt-secret the consensus node uses to authenticate to the execution node.
Now we can start working on the services. We will be creating two Helm charts, one for each service. Both services will be defined as a StatefulSet , due to the long synchronization both require, which we don’t want it re-running each time a pod is recreated and the need for each of the nodes to have a fixed identity in the Ethereum network.
Go Ethereum’s StatefulSet definition:
Lighthouse’s StatefulSet definition:
Note that both definitions have the jwt-secret mounted to them and a persistent volume that will be used to store the synchronized state between pod terminations.
To allow Lighthouse to communicate with Go Ethereum internally, without leaving the boundaries of the cluster, we will create a headless service for Go Ethereum, which will allow Lighthouse to resolve the generated internal hostname.
The last part is to expose both services to the internet using a LoadBalancer service with a few Azure specific tricks (annotations) that will set the name of the resource groups that contains the public IP that should be used (the actual IP is in the .spec.loadBalancerIP field), enable mixed protocol support for the service and give it an external DNS label under the <region>.cloudapp.azure.com domain. Each service will have a selector that will use the statefulset.kubernetes.io/pod-name with the name of the single pod of the service (which is instance name with a -0 suffix, as that is the only pod instance).
Go Ethereum’s LoadBalancer definition:
Lighthouse’s LoadBalancer definition:
And the definition of the pod selector:
The last part is just to run terraform apply to deploy everything to Azure (you might need to do az login if haven’t done so lately).
Testing The Deployment
The synchronization of both the consensus and later the execution nodes will take time, probably a few days (execution node synchronization only starts when the consensus node synchronization is complete).
You will be able to check the logs of both the pods (using either kubectl or the azure portal) and see whether the synchronization is complete or not, and once they are, you will be able to test your node using the JSON-RPC calls you used with your old, pre-merge node.
Conclusion
The example discussed in this document will get you up and running in a relatively simple manner, but for high load/high reliability scenarios this deployment won’t be enough, as those scenarios will require a Kubernetes cluster with more than node and maybe more than one instance of each of the services deployed for redundancy. As for managing a production grade Kubernetes cluster in a production environment, this example is far from it (monitoring, logging, internal network policies, etc.), so you should probably use this deployment for development and testing purposes only.
