Ethereum on Google Cloud Platform
Blockchain Platforms on Google Cloud Platform
Google’s super-fast network, powerful VMs and SSDs coupled with its state-of-the-art container technologies make Google Cloud Platform (GCP) an unparalleled destination for Blockchain platforms.
Google could do better in ensuring that developers are aware of this.
Late last year, I wrote a story describing how to deploy Ethereum to GCP. I’d not used Ethereum between then and today. This story updates that previous post with easier ways to run Ethereum on GCP using: (a) Container-Optimized OS; (b) using Kubernetes Engine.
You’ll need a GCP Project and there are free options. You will need Cloud SDK (aka gcloud
) installed. Let’s get started.
Container-Optimized OS
Container-Optimized OS permits you to deploy a Docker image to a Google Compute Engine VM. This is the simplest way to run the ethereum/client-go
image:
INSTANCE=[[YOUR-INSTANCE]] # cos-ethereum
ZONE=[[YOUR-ZONE]] # us-west1-c
PROJECT=[[YOUR-PROJECT]]gcloud beta compute instances create-with-container ${INSTANCE} \
--boot-disk-size=500GB \
--boot-disk-type=pd-ssd \
--container-image=ethereum/client-go \
--container-arg="--rpc" \
--container-arg="--rpcaddr=0.0.0.0" \
--container-restart-policy=always \
--container-mount-host-path=\
mount-path=/root,\
host-path=/tmp/client-go,\
mode=rw \
--image-family=cos-stable \
--image-project=cos-cloud \
--zone=${ZONE} \
--project=${PROJECT}
NB To avoid exhausting disk, the container uses 500GB (the entire default Quota) of SSD Persistent Storage. Reduce the
boot-disk-size
or change theboot-disk-type
to pd-standard as you prefer.NB If you would prefer to use the test network add
— container-arg="— testnet”
before the existing— container-arg
flags. If you would prefer to use Rinkeby add— container-arg=" — rinkeby"
before the existing--container-arg
flags.
The VM should be running in under 30 seconds, you can peek into its state with:
gcloud compute ssh ${INSTANCE} \
--command="sudo journalctl --unit=konlet-startup --follow" \
--project=${PROJECT}
NB
konlet-startup
is the systemd service that corresponds to the Docker image deployment in the Container-Optimized OS.
Success looks like:
Created a container with name 'cos-ethereum' and ID: fdd6...
Starting a container with ID: fdd6...
Saving welcome script to profile.d
And you may confirm the container is running:
gcloud compute ssh ${INSTANCE} \
--command="docker ps --format='[{{.ID}}] {{.Names}}: {{.Status}}'" \
--project=${PROJECT}[a8c710e75ae3] cos-ethereum: Up 8 minutes
Then, you may use gcloud
to start the ssh
port-forward to the instance:
gcloud compute ssh ${INSTANCE} \
--ssh-flag="-L 8545:localhost:8545" \
--project=${PROJECT}
And — from a different shell — you can connect to the Ethereum node running on the instance:
docker run \
--rm \
--interactive \
--net=host \
--tty \
ethereum/client-go \
attach http://localhost:8545
Which should yield:
Welcome to the Geth JavaScript console!instance: Geth/v1.8.9-unstable-0fe47e98/linux-amd64/go1.10.2
modules: eth:1.0 net:1.0 rpc:1.0 web3:1.0>
Running Ethereum on GCP leverages Google Cloud Logging. You will automatically be able to view and query logs.
https://console.cloud.google.com/logs/viewers
NB Slightly confusingly (!), you must select “Global” as the resource and you should then see “gcplogs-docker-driver” as an option. This will — as above — provide the container’s logs.
Everything you see via Cloud Console is accessible from the CLI too. In this case, you may run the following command to view the container’s logs:
FILTER="resource.type=\"global\" "\
"logName=\"projects/${PROJECT}/logs/gcplogs-docker-driver\" "gcloud logging read "${FILTER}" \
--project=$PROJECT \
--format="value(jsonPayload.data)" \
--order=asc
You should log entries like this:
INFO [05-23|00:47:38] Imported new state entries count=1920 elapsed=8.560ms processed=2993520 pending=6080 retry=0 duplicate=31 unexpected=283
INFO [05-23|00:47:39] Imported new state entries count=1920 elapsed=110.058ms processed=2995440 pending=5488 retry=0 duplicate=31 unexpected=283
INFO [05-23|00:47:39] Imported new state entries count=1306 elapsed=11.663ms processed=2996746 pending=6196 retry=0 duplicate=31 unexpected=283
INFO [05-23|00:47:39] Imported new block receipts count=691 elapsed=286.192ms number=1358549 hash=3633f9…19e595 size=1.94mB ignored=0
...
...
Fatal: Error starting protocol stack: write /root/.ethereum/rinkeby/geth/chaindata/004464.ldb: no space left on device
Fatal: Error starting protocol stack: write /root/.ethereum/rinkeby/geth/chaindata/004464.ldb: no space left on device
NB What you *don’t* want are log entries of the form “no space left on device” as these terminate the container. If you receive these you’ll need to increase the size of the boot-disk-size
flag or grab less data.
You also get basic monitoring *but* of the VM not the container(s) specifically:
You may exit
the JavaScript console then delete the Ethereum VM with:
gcloud compute instances delete ${INSTANCE} \
--zone=${ZONE} \
--project=${PROJECT}
Kubernetes Engine
If you would prefer to use a Kubernetes (Engine) cluster, the commands are as straightforward.
Assuming you are authenticated against a cluster, the following steps create a namespace for Ethereum, create a disk (actually a Kubernetes PersistentVolumeClaim), deploy a single Go Ethereum (aka “geth”) node and port-forward to your local workstation so that you may attach to it:
NAMESPACE=ethereum
kubectl create namespace ${NAMESPACE}
kubectl apply --filename=deployment.yaml --namespace=${NAMESPACE}
NB If you would prefer to use the test network insert
"--testnet"
(including the quotation marks) between lines 33–34. If you’d prefer to use Rinkeby network insert"--rinkeby"
instead.
This should result in:
persistentvolumeclaim "ethereum" created
deployment.extensions "ethereum" created
service "ethereum" created
And you should be able to see the Persistent Volume Claim created:
https://console.cloud.google.com/kubernetes/storage
And the Deployment:
https://console.cloud.google.com/kubernetes/workload
Running Ethereum on GCP yields some useful, automatic benefits including, Console monitoring:
Logging:
https://console.cloud.google.com/logs/viewer
Some Stackdriver goodness (although this is even better with the recently-announced Stackdriver Kubernetes Monitoring — not enabled on this cluster). If you’ve not configured Stackdriver for the project, you’ll need to do that first and you may use the free tier:
https://console.cloud.google.com/monitoring
We’ll use ssh port-forwarding (through gcloud
) to access the Go Ethereum node from our workstation without opening a firewall.
SERVICE=ethereum
NAMESPACE=ethereumNODE=$(\
kubectl get nodes \
--output=jsonpath='{.items[0].metadata.name}')PORT=$(\
kubectl get services/${SERVICE} \
--namespace=${NAMESPACE} \
--output=jsonpath='{.spec.ports[?(@.name=="default")].nodePort}')echo ${PORT}gcloud compute ssh ${NODE} \
--ssh-flag="-L ${PORT}:localhost:${PORT}" \
--project=${PROJECT}
NB These nebulous looking commands grab one of the Kubernetes cluster’s node names; determine which NodePort the Ethereum Service is being served on; and then uses
gcloud
to port-forward that node’s port to the localhost’s (same) port. If you prefer you could use —ssh-flag=“-L 8545:localhost:${PORT}”
and then in the subsequent command replace${PORT}
with8545
too. Your choice!NB In my case
echo ${PORT}
(this time) returned30873
.
Then, from your local workstation, run the geth
JavaScript client and attach to the node running on the Kubernetes cluster:
docker run \
--rm \
--interactive \
--tty \
--net=host \
ethereum/client-go attach http://localhost:${PORT}
NB In my case, the value of
${PORT}
is30873
.
And you should then be put in the Ethereum JavaScript console:
Welcome to the Geth JavaScript console!instance: Geth/v1.8.9-unstable-0fe47e98/linux-amd64/go1.10.2
modules: eth:1.0 net:1.0 rpc:1.0 web3:1.0>
When you’re done, it’s simplest to whack the Kubernetes namespace which will delete the Deployment, the Service and the PeristentVolumeClaim:
kubectl delete namespace ${NAMESPACE}
18–05–25 Aside: Wallets
I uncovered an issue using Wallets passed to Kubernetes (Engine) as Secrets. Providing data to Pods using Secrets (and ConfigMaps) is considered a good practice with Kubernetes. However, the implementation of both Secrets and ConfigMaps when (volume) mounted into Pods is to present file content as symbolic links. This appears to be a problem with Wallets and Ethereum:
https://github.com/ethereum/go-ethereum/issues/16793
If you’d like to use existing wallet file(s) when you deploy to Kubernetes, this should be as simple as creating a Secret (you may use ConfigMaps similarly but Secrets provide opacity for… secrets and other confidential info) and then mounting the Secret as a volume mount for the Ethereum node to access. However, as above, this does not work correctly with Ethereum. So, you will need the additional step in the ‘hacky workaround’ below.
First, create the Secret:
kubectl create secret generic keystore \
--from-file=PATH/TO/YOUR/keystore \
--namespace=$NAMESPACE
NB Your path should end in a directory called
keystore
and this directory and its wallet(s) will be encoded in the Secret.
My hacky workaround is to use init containers to copy the wallet(s) from the Secret (called /keystore
) into an emptyDir (called /cache
) volume *before* the Ethereum node starts and configure Ethereum to look for the wallet in /cache
rather than /keystore
(because this doesn’t work). This works because copying the symbolic link duplicates the underlying file.
NB See the next section ‘SSD’ to configure the
datadir
as SSD rather than HDD.NB Line #31–39 create an init container called
init-service
.init-service
uses Alpine to copy files of the formUTC*
from/keystore
to/cache
. These directories are volume mounted. The volumes are defined (for the Pod not specific containers) in lines #66–75.keystore
is the Secret containing wallet file(s) mounted as symbolic link(s) that do not appear to work with Ethereum if used directly.cache
is an emptyDir volume and, if Ethereum uses the wallet(s) when copied here, it does work!NB In line #46, Ethereum is configured to use the
/cache
directory for wallets with--keystore=/cache
.
SSD
The Kubernetes Deployment used above includes a PersistentVolumeClaim
spec and line #7 defines storageClassName
of standard
. This corresponds to regular Persistent Disk.
If you’d prefer to use faster SSD, we must first register the new Storage class in Kubernetes Engine:
and:
kubectl apply --filename=ssd.yaml
Now, when you query the Kubernetes Engine storage classes, a new type pd-ssd
called ssd
should be added:
Then, we can revise the Deployment file’s line #7 and change storageClassName: standard
to storageClassName: ssd
.
NB Default “Persistent Disk SSD (GB)” Quotas for GCP Projects are 500 GB (gigabytes) per zone. If you’ve not increased these quotas, you will be unable to provision a 500 GiB (!) disk when you Apply the Deployment. For this reason, please also *reduce* the value of line #12 in the Deployment to e.g. 400Gi (~429GB) which should be sufficient *unless* you have other SSD PD in your Project in the zone. Or, request a Quota increase:
To effect this change we must delete and recreate the Deployment:
kubectl delete --filename=deployment.yaml --namespace=${NAMESPACE}
kubectl apply --filename=deployment.yaml --namespace=${NAMESPACE}
NB There are 4 Compute Engine Disks: 3 are “Standard Persistent disk” one for each of the 3 Nodes in the Kubernetes cluster. The 4th disk is 430GB (==400GiB) and is of type “SSD persistent disk”. It was created by the PersistentVolumeClaim when we applied the Deployment.
Conclusion
In this post, I’ve provided two straightforward ways to run a single Ethereum node on Google Cloud Platform.
Feedback is always welcome.
That’s all!