Spinning up a Tezos blockchain with Kubernetes (k8s)

Overview

Tezos is a public blockchain network that includes innovative features such as on-chain governance and proof of stake. Smart contracts on Tezos are compiled to Michelson, a stack-based, functional language that lends itself to formal verification.

A private or permissioned chain of Tezos provides a controlled environment for development, testing, or consortium-governed networks popular in enterprise settings.

The instructions laid out in this article, in conjunction with the provided scripts, will create a fully working independent Tezos chain by performing the following high-level steps:

1. Prepare your machine to run the genesis node
2. Deploy local configuration to initiate the chain
3. Invite others to join your chain

Key Resources

• Github repo: https://github.com/tqtezos/tezos-k8s
• Multicluster documentation: https://github.com/tqtezos/tezos-k8s/blob/1.0/MULTICLUSTER.md
• About zerotier: https://www.zerotier.com/manual/
• About kubernetes: https://kubernetes.io/docs/home/
• About minikube: https://minikube.sigs.k8s.io/docs/contrib/principles/

Prerequisites

Kubernetes and its underlying Docker containers will do most of the work in setting up the environment needed for a healthy Tezos deployment. That being said, there are a few things needed in order to bootstrap our way there in the first place:

• minikube: for running kubernetes on your local machine. Experienced kubernetes users who have the appropriate cluster setup can adapt the configuration for their environment.
• docker: needed to generate blockchain constants like public/private keys
• python3: required to run the deployment helper scripts written for this article

With that we’re ready to continue!

Networking

In order for the Tezos nodes in our private chain to talk to each other, they must be discoverable on a network. The way that we have chosen to implement this for the sake of this tutorial is with the help of ZeroTier. In particular, we use ZeroTier to create a virtual area network which will host every Tezos node in our chain. This eliminates the need for complicated port forwarding and kubernetes networking configurations.

Why use ZeroTier in particular? Although it is not strictly required, (you can in principle manually set up the network with some k8s-fu and a bit of configuration with your router and NAT), the use of a VPN saves a lot of work. ZeroTier strikes a good balance between being reasonably open-source and easy to use.

To prepare a ZeroTier network for deployment:

1. Visit the ZeroTier website, and register an account
2. Create an API token by clicking the Generate new token button under the Account tab
3. Save the generated token (e.g. “yEflQt726fjXuSUyQ73WqXvAFoijXkLt”)
4. Visit the network tab tab, and create a new network
5. Save the 16 character network token (e.g. “1c33c1ced02a5eee”)

You will want to have these two tokens ready for use in later commands, so let’s put these into shell variables:

$ ZT_TOKEN=yEflQt726fjXuSUyQ73WqXvAFoijXkLt
$ ZT_NET=1c33c1ced02a5eee

Deploying

We should now have everything we need in order to begin the process of generating the needed kubernetes deployment.

If you’re using minikube, now is a good time to make sure that your minikube is running:

$ minikube start

Let’s choose a name for our chain. For the sake of this tutorial, we will use a UUID as a chain name. For your own chain, you can choose whatever you like here, as long as it’s valid ascii characters:

$ CHAIN_NAME=$(uuidgen)

To create the k8s deployment, we have written a small python script to generate kubernetes yaml files tailored for your blockchain and api keys. Let’s install that into a python venv:

python3 -m venv .venv-tezos-k8s
source .venv-tezos-k8s/bin/activate
pip install mkchain==1.0

We are now ready to generate our kubernetes config with the following commands:

mkchain generate-constants --zerotier-network $ZT_NET --zerotier-token $ZT_TOKEN $CHAIN_NAME
mkchain create $CHAIN_NAME > config.yaml

Explanation of the commands:

• First, we generate some constants that are needed for the tezos blockchain to run, e.g. public/private keys, baker account info, genesis block, etc. These constants get placed into files named `$(CHAIN_NAME)_chain.yaml` and `$(CHAIN_NAME)_chain_invite.yaml`. The _chain.yaml file contains your private keys, and the _invite.yaml contains public keys for generating invitations
• Then, we use these constants to generate a kubernetes deployment yaml file and redirect that into config.yaml

You may inspect the generated config.yaml at this point to see ~600 or so lines of kubernetes configuration. If you are satisfied that there’s nothing malicious or broken within the config, you may now apply it with kubectl:

$ kubectl apply -f config.yaml

At this point, the provided yaml will make your kubernetes cluster run a series of jobs to boot up the chain. In particular, it will:

• Get a zerotier ip
• Generate a node identity
• Activate the protocol
• Bake the first block
• Start the node and a baker to validate the chain

At this point, you can inspect your node with kubectl. If all goes well, you should see something like this:

$ kubectl -n tqtezos get pods -o wide
NAME READY STATUS RESTARTS AGE
activate-job-nkdl4 1/1 Completed 0 5m
tezos-bootstrap-node-58b559c67f-kjncz 3/3 Running 0 5m

If you want to inspect the logs, you can do so with the following command:

$ kubectl -n tqtezos logs -l appType=tezos -c tezos-node --tail -1

You should see output similar to the following:

Nov 13 18:36:14 — node.main: starting the Tezos node (chain = 6AFEB14F-39F3–4D93–8FFF-E1BC9D52D9F7)
Nov 13 18:36:14 — node.main: disabled local peer discovery
Nov 13 18:36:14 — node.main: read identity file (peer_id = idsRNbwVNAEA5PV8fDccxpa5EsF4bW)
Nov 13 18:36:14 — main: shell-node initialization: bootstrapping
Nov 13 18:36:14 — main: shell-node initialization: p2p_maintain_started
Nov 13 18:36:14 — block_validator_process_external: Initialized
Nov 13 18:36:14 — block_validator_process_external: Block validator started on pid 10
Nov 13 18:36:15 — validator.block: Worker started
Nov 13 18:36:15 — node.validator: activate chain NetXXhPDYhjY7eX
Nov 13 18:36:15 — p2p.maintenance: Too few connections (0)
Nov 13 18:36:15 — validator.chain: Worker started for NetXXhPDYhjY7
Nov 13 18:36:15 — node.chain_validator: no prevalidator filter found for protocol
Nov 13 18:36:15 — node.chain_validator: PtYuensgYBb3G3x1hLLbCmcav8ue8Kyd2khADcL5LsT5R1hcXex
Nov 13 18:36:15 — prevalidator.NetXXhPDYhjY7.PtYuensgYBb3: Worker started for NetXXhPDYhjY7.PtYuensgYBb3
Nov 13 18:36:15 — node.main: starting RPC server on ::ffff:0.0.0.0:8732 (tls = false)
Nov 13 18:36:15 — node.main: the Tezos node is now running
Nov 13 18:36:20 — p2p.maintenance: Too few connections (0)
Nov 13 18:36:25 — p2p.maintenance: Too few connections (0)
Nov 13 18:36:30 — p2p.maintenance: Too few connections (0)
Nov 13 18:36:35 — p2p.maintenance: Too few connections (0)
Nov 13 18:36:40 — p2p.maintenance: Too few connections (0)

Assuming everything checks out, congratulations! You now have your very own Tezos blockchain, all to yourself.

Inviting Others

Of course, it can be lonely being the only one on your chain. If you want to add friends or others in general, you can use the mkchain command to help with that:

$ mkchain invite $CHAIN_NAME > join-$CHAIN_NAME.yaml

You may then share the resulting yaml file with whoever you wish to have in your chain. Be careful with this! Should the file, or other sensitive information like your ZeroTier tokens fall into the wrong hands, it is possible for outsiders to view block information on your chain.

In order for your guests to join, they will need to install a kubernetes environment and apply the invite yaml to their cluster.

$ kubectl apply -f join-$CHAIN_NAME.yaml

At this point additional nodes will be added in a full mesh topology.

Congratulations! You now have a multi-cluster Tezos based permissioned chain.

Check that the nodes have matching heads by comparing their hashes:

kubectl get pod -n tqtezos -l appType=tezos -o name | \
while read line
do
  kubectl -n tqtezos exec $line -c tezos-node -- \
  /usr/local/bin/tezos-client rpc get /chains/main/blocks/head/hash
done

Special thanks to Nicolas Ochem from Midl Dev.