Google Trillian for Noobs (1b)

The Missing Manual series

Published in

Google Cloud - Community

5 min readJul 5, 2019

Last week I documented what I hope is the simplest possible Trillian personality. Yesterday, I documented adding an inclusion proof. Today we’ll split the main.go into a client and a server and reconnect them using gRPC. This is a gRPC rather than Trillian work but, it helps to evolve the personality.

Setup

You’ll need the Database and Trillian Servers described in my previous post.

A gRPC-based Personality Server

This time, either clone gRPC:

git clone \
--single-branch \
--branch=gRPC \
git@github.com:DazWilkin/simple-trillian-log-1.git

Or you may just run the Docker Compose file:

docker-compose --file=./deployment/docker-compose.yml up

NB In either case you will need to have created the Database.

Docker Compose will create 7 services:

Name                       State  Ports
--------------------------------------------------------------------
adminer                    Up     0.0.0.0:7777->8080/tcp
basic-personality-client   Up     0.0.0.0:9997->9999/tcp
basic-personality-server   Up     50051/tcp
                                  0.0.0.0:8080->8080/tcp
                                  0.0.0.0:9998->9999/tcp
db                         Up     3306/tcp
prometheus                 Up     0.0.0.0:9090->9090/tcp
trillian-log-server        Up     0.0.0.0:54051->50051/tcp, 8080/tcp
trillian-log-signer        Up     50051/tcp, 8080/tcp

I’ll not describe adminer and db much further. Suffice to say that the db service exposes port 3306 within (!) the network created by Docker Compose to the Trillian Log Server and Log Signer.

The basic-personality-client exposes a zPages endpoint internally on basic-personality-client:9999 that will be exposed on your host on :9997. You can browse this endpoint using either localhost:9997/debug/rpcz or localhost:9997/debug/tracez.

The basic-personality-server exposes :50051 internally (only) and this is used by basic-personality-client to access it using gRPC. This service also exposes :8080 (http) and a zPages endpoint internally on basic-personality-server:9999 that will be exposed on your host on :9998. You can browse this endpoint using either localhost:9998/debug/rpcz or localhost:9998/debug/tracez.

The prometheus services exposes its endpoint on :9090 (see below).

The trillian-log-server exposes its gRPC endpoint on the host on :54051. This is to provide a host-accessible endpoint to use the createtree binary to create new logs.

Prometheus

Currently only the Trillian services are instrumented. You can access the Prometheus server on:

http://localhost:9090
http://localhost:9090/graph
http://localhost:9090/status
http://localhost:9090/targets
http://localhost:9090/-/healthy

NB Because Prometheus is part of the Docker Compose network, the service is able to reference the Trillian services using their network names (trillian-log-server and trillian-log-signer). You will need to create port bindings to expose these services’ :8080 ports on the host if you wish to access their /metrics endpoints directly.

zPages

The trivial addition of OpenCensus’ zPages package, provides details of gRPC calls including traces. Google’s convention is to pluralize (pluralise?) these with a ‘-z’. So, you can browse:

http://localhost:9998/debug/rpcz
http://localhost:9998/debug/tracez

And — for the basic-personality-server (exposed on :9998) you should see:

And:

NB For tracez the first column is in-flight|running traces so you have to refresh at the right time. The latency histogram provides more opportunity for drilldown.

Channelz

The Basic Personality gRPC server exposes a channelz endpoints over its gRPC endpoint (:50051). If you wish to explore the functionality of channelz, please see gdebug. The simplest solution is to revise the Docker Compose file to expose this port to the host so that you may run gdebug on the host and so that gdebug can access the Basic Personality service.

Protobuf

The essence of the changes in this branch of the repo are a result of the addition of protobuf definitions for the messages our original client (main.go) was shipping to and from the Trillian Log Server. We’d refactored this code so it appeared to be interacting with a server when, in fact this was purely a naming|package conceit. You’ll recall we have 3 functions with similar signatures. Of the form:

server.put(&Request{
  thing: *thing,
  extra: *extra,
 }

After creating protobuf definitions of these functions and messages. Then new client code looks similar now using a gRPC client proxy:

client.PutThing(ctx, &pb.ThingRequest{
   Thing: &pb.Thing{...},
   Extra: &pb.Extra{...},
})

and the gRPC server proxy is just:

func (s *server) PutThing(ctx context.Context, r *pb.ThingRequest) (*pb.ThingResponse, error) { return s.put(r) }

I’m not wanting to bore you with a gRPC tutorial but, for completeness, pb.ThingRequest and pb.ThingResponse types are described in the protobuf definition along with our renamed — but otherwise similar — functions PutThing, GetThing, and WaitThing.

syntax = "proto3";package google.basicpersonality.v1;option go_package = "github.com/DazWilkin/basic-personality/protos";
option java_package = "com.google.dazwilkin.basicpersonality";service BasicPersonality {
  rpc PutThing(ThingRequest) returns (ThingResponse) {};
  rpc GetThing(ThingRequest) returns (ThingResponse) {};
  rpc WaitThing(ThingRequest) returns (ThingResponse) {};message ThingRequest {
  Thing thing = 1;
  Extra extra = 2;
}
message ThingResponse {
  string status = 1;
}
message Thing {
  string name = 1;
}
message Extra {
  string name = 1;
}

All that we need do is provide the above definition to the protobuf compiler (protoc) , configure it to generate Golang bindings for us and, we’re almost done.

We then need to decouple the original main.go into a client and a server that each use the machine-generated gRPC bindings for BasicPersonality and let the gRPC runtime take care of passing messages between them.

Docker Compose

As mentioned previously, the Docker Compose file is extended to run the basic-personality-client and basic-personality-server alongside the Database, a Trillian Log Server and Log Signer, and Prometheus. We configure each with the gRPC port (:50051) to bind them together.

The client is configured to run forever so albeit with 15-second pauses. When you up the Docker Compose file, be careful not to leave it running as you’ll eventually blow up the database.

Conclusion

We now have a very basic Trillian personality called “Basic Personality” that is implemented as Golang gRPC server that itself uses gRPC to communicate with the Trillian services. For simplicity, we generated a Golang gRPC client that integrates with our Basic Personality server.

Because Basic Personality has a protobuf service definition and exposes a gRPC endpoint using this definition, you could take the protobuf and generate a client in any gRPC language to talk with the service.

In the next steps, I think I’m going to (a) instrument these services using OpenCensus; (b) generate a non-Golang gRPC client. I Googled Rust gRPC and it seems non-trivial for a Rust beginner. If anyone reading this would like to show how this could be done, I’d be interested.