Trillian on Google Cloud Platform

Daz Wilkin
May 26, 2018 · 5 min read

As my better-educated colleague — Annie — pointed out yesterday, “Trillian” is named after a character in “The Hitchhiker’s Guide to the Galaxy” this is (a) excellent; (b) not Greek which is a refreshing change having been submerged in Kubernetes. The eponymous project is a Google OSS contribution: Trillian.

This post is a quick-hit walking through all the hard-work that’s been done by this team in developing Trillian and in making it straightforward to deploy to Kubernetes.

Please see the README on the Trillian GitHub here for the source of these instructions.

Setup

  1. Enable “container” and “spanner” APIs
  2. Request additional Quota for IPs in your preferred zone (you need ≥12)
  3. Create a single-zone (for now) Kubernetes cluster.
  4. Download Trillian
PROJECT=[[YOUR-PROJECT]]
BILLING=[[YOUR-BILLING]]
gcloud projects create ${PROJECT}gcloud beta billing projects link $PROJECT \
--billing-account=$BILLING
for SERVICE in container spanner
do
gcloud services enable ${SERVICE}.googleapis.com \
--project=${PROJECT}
done

For this project (${PROJECT}), you will need ≥12 IP addresses as 12 machines (4+2+4+2) are created by the Kubernetes cluster. The default Quota is 8 and so you may need to request more Quota. You may do so through the Cloud Console:

https://console.cloud.corp.google.com/iam-admin/quotas?project=${PROJECT}&service=compute.googleapis.com&metric=In-use%20IP%20addresses&location=${ZONE}

Image for post
Image for post
Cloud Console: Default Zonal IP Quota is 8

Select the Quota that represents the “In-use IP Address” for the zone you wish to use, click the “Edit Quotas” and request ≥12 IPs. Await the email confirming the Quota has been approved.

NB Do not proceed until you receive confirmation of the Quota increase because the scripts will fail without sufficient Quota.

Here’s a screenshot *after* the Quota was approved and *after* I’d run ./create.sh as shown in the next section:

Image for post
Image for post
Cloud Console: Increased Zonal IP Quota now 11

Trillian

WORKING_DIR=[[YOUR-WORKING-DIRECTORY]]cd ${WORKING_DIR}
mkdir go
export GOPATH=$PWD/go
export PATH=$PATH:$GOPATH/bin
go get github.com/google/trillian
cd go/src/github.com/google/trillian/examples/deployment/kubernetes

Please edit the config.sh file to correctly reflect your preferences:

export PROJECT_NAME=[[YOUR-PROJECT]]
export CLUSTER_NAME=trillian-01
export REGION=[[YOUR-PREFERRED-ZONE]] # This needs suff Quota
export ZONE=${REGION}-b
export CONFIGMAP=trillian-cloudspanner.yaml

We are ready to begin the 2 phase deployment.

First, run:

./create.sh

This provisions the Google Cloud Platform resources: Kubernetes cluster and a Cloud Spanner instance.

Here’s the Kubernetes cluster:

Image for post
Image for post
Kubernetes Engine Cluster w/ 4 Node Pools

If the cluster does not provision with 4 Node Pools as shown above, you should revisit the instructions. The deployment will not succeed without this configuration.

A soupcon of Cloud Spanner:

https://console.cloud.google.com/spanner/instances/trillian-spanner/databases?project=${PROJECT}

Image for post
Image for post
Cloud Spanner

And, a couple of container images:

https://console.cloud.google.com/gcr/images/${PROJECT}/

Image for post
Image for post
Container Registry

If everything appears to have completed successfully, you may proceed to the deployment phase with:

./deploy.sh

This script builds the container images and pushes them to Google Container Registry. Then it creates Kubernetes Deployments that leverage these images.

If this script succeeds, you will be presented with an enumeration of all the Kubernetes resources on the command-line (not reproduced here) and, you may confirm the Deployments (is proceeding | have completed) using Cloud Console:

Image for post
Image for post
All Workloads: etcd and Trillian’s Log Server and Log Signer

And:

Image for post
Image for post
Workload: trillian-logserver-deployment

And:

Image for post
Image for post
Workload: trillian-logsigner-deployment

This brings you to the step in the GitHub README “Next steps”. We’ll want to do something useful with our Trillian deployment to Kubernetes (backed by Cloud Spanner).

You may not yet be using Trillian’s minimal container images but — hopefully — you will soon be using them. When you are, you can’t ssh into a trillian-log-server Pod and run curl commands because the Pods have no shell and don’t include curl. Instead, you can port-forward to test the Trillian API

Trillian API

LOG_POD=$(\
kubectl get pods \
--selector=io.kompose.service=trillian-log \
--output=jsonpath='{.items[0].metadata.name}')

Knowing that these Pods all expose port 8091 as an HTTP endpoint, we can then setup port-forwarding from your local machine to the Pod. For convenience, we’ll use the same port on either end:

kubectl port-forward ${LOG_POD} 8091:8091Forwarding from 127.0.0.1:8091 -> 8091
Forwarding from [::1]:8091 -> 8091
Handling connection for 8091
Handling connection for 8091
Handling connection for 8091

From another shell, you should then be able to issue the command cited in the Trillian tutorial:

HOST=localhost
PORT=8091
RESPONSE=$(curl \
--silent \
--request POST \
http://${HOST}:${PORT}/v1beta1/trees \
--data '{ "tree":{ "tree_state":"ACTIVE", "tree_type":"LOG", "hash_strategy":"RFC6962_SHA256", "signature_algorithm":"ECDSA", "max_root_duration":"0", "hash_algorithm":"SHA256" }, "key_spec":{ "ecdsa_params":{ "curve":"P256" } } }')
TREE_ID=$(echo ${RESPONSE} | jq --raw-output .tree_id)echo ${RESPONSE} \
| jq .

And, all being well, you should receive something of the form:

{
"tree_id": "1234567890123456789",
"tree_state": "ACTIVE",
"tree_type": "LOG",
"hash_strategy": "RFC6962_SHA256",
"hash_algorithm": "SHA256",
"signature_algorithm": "ECDSA",
"storage_settings": {
"@type": "type.googleapis.com/spannerpb.LogStorageConfig",
"num_unseq_buckets": "4",
"num_merkle_buckets": "16"
},
"public_key": {
"der": "[[REDACTED]]"
},
"max_root_duration": "0s",
"create_time": "2018-00-00T00:00:00.000000000Z",
"update_time": "2018-00-00T00:00:00.000000000Z"
}

And, then you may:

curl \
--silent \
--request POST \
http://${HOST}:${PORT}/v1beta1/logs/${TREE_ID}:init \
| jq .

And you should receive something of the form:

{
"created": {
"timestamp_nanos": "1527642133340965256",
"root_hash": "[[REDACTED]]",
"key_hint": "[[REDACTED]]",
"log_root": "[[REDACTED]]",
"log_root_signature": "[[REDACTED]]"
}
}

I’ll update this post next week with something for us to use it for. One of the members of the Trillian team has a nifty example :-)

2018–06–01: Update

Following on from above… with ${TREE_ID} and having called ${TREE_ID}:init:

VALUE="[[SOME-STRING]]"
EXTRA="[[SOME-STRING]]"
EVALUE=$(echo ${VALUE} | base64)
EEXTRA=$(echo ${EXTRA} | base64)
curl \
--silent \
--request POST \
http://${HOST}:${PORT}/v1beta1/logs/${TREE_ID}/leaves \
--data "{ \"leaf\": { \"leaf_value\": \"${EVALUE}\", \"extra_data\": \"${EEXTRA}\" } }" \
| jq .

Or for some bashity to get a random strings of 256 characters...

EVALUE=$(\
cat /dev/urandom \
| tr --complement --delete 'a-zA-Z0-9 ' \
| fold --width=256 \
| head --lines=1 \
| base64 --wrap=0\
)

More soon!

Conclusion

Google Cloud - Community

Google Cloud community articles and blogs

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch

Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore

Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store