Google Trillian for Noobs (1b)
The Missing Manual series
Last week I documented what I hope is the simplest possible Trillian personality. Yesterday, I documented adding an inclusion proof. Today we’ll split the main.go
into a client and a server and reconnect them using gRPC. This is a gRPC rather than Trillian work but, it helps to evolve the personality.
Setup
You’ll need the Database and Trillian Servers described in my previous post.
A gRPC-based Personality Server
This time, either clone gRPC
:
git clone \
--single-branch \
--branch=gRPC \
git@github.com:DazWilkin/simple-trillian-log-1.git
Or you may just run the Docker Compose file:
docker-compose --file=./deployment/docker-compose.yml up
NB In either case you will need to have created the Database.
Docker Compose will create 7 services:
Name State Ports
--------------------------------------------------------------------
adminer Up 0.0.0.0:7777->8080/tcp
basic-personality-client Up 0.0.0.0:9997->9999/tcp
basic-personality-server Up 50051/tcp
0.0.0.0:8080->8080/tcp
0.0.0.0:9998->9999/tcp
db Up 3306/tcp
prometheus Up 0.0.0.0:9090->9090/tcp
trillian-log-server Up 0.0.0.0:54051->50051/tcp, 8080/tcp
trillian-log-signer Up 50051/tcp, 8080/tcp
I’ll not describe adminer
and db
much further. Suffice to say that the db
service exposes port 3306
within (!) the network created by Docker Compose to the Trillian Log Server and Log Signer.
The basic-personality-client
exposes a zPages endpoint internally on basic-personality-client:9999
that will be exposed on your host on :9997
. You can browse this endpoint using either localhost:9997/debug/rpcz
or localhost:9997/debug/tracez
.
The basic-personality-server
exposes :50051
internally (only) and this is used by basic-personality-client
to access it using gRPC. This service also exposes :8080
(http) and a zPages endpoint internally on basic-personality-server:9999
that will be exposed on your host on :9998
. You can browse this endpoint using either localhost:9998/debug/rpcz
or localhost:9998/debug/tracez
.
The prometheus
services exposes its endpoint on :9090
(see below).
The trillian-log-server
exposes its gRPC endpoint on the host on :54051
. This is to provide a host-accessible endpoint to use the createtree
binary to create new logs.
Prometheus
Currently only the Trillian services are instrumented. You can access the Prometheus server on:
http://localhost:9090
http://localhost:9090/graph
http://localhost:9090/status
http://localhost:9090/targets
http://localhost:9090/-/healthy
NB Because Prometheus is part of the Docker Compose network, the service is able to reference the Trillian services using their network names (
trillian-log-server
andtrillian-log-signer
). You will need to createport
bindings to expose these services’:8080
ports on the host if you wish to access their/metrics
endpoints directly.
zPages
The trivial addition of OpenCensus’ zPages package, provides details of gRPC calls including traces. Google’s convention is to pluralize (pluralise?) these with a ‘-z’. So, you can browse:
http://localhost:9998/debug/rpcz
http://localhost:9998/debug/tracez
And — for the basic-personality-server
(exposed on :9998
) you should see:
And:
NB For tracez the first column is in-flight|running traces so you have to refresh at the right time. The latency histogram provides more opportunity for drilldown.
Channelz
The Basic Personality gRPC server exposes a channelz endpoints over its gRPC endpoint (:50051
). If you wish to explore the functionality of channelz, please see gdebug. The simplest solution is to revise the Docker Compose file to expose this port to the host so that you may run gdebug on the host and so that gdebug can access the Basic Personality service.
Protobuf
The essence of the changes in this branch of the repo are a result of the addition of protobuf definitions for the messages our original client (main.go
) was shipping to and from the Trillian Log Server. We’d refactored this code so it appeared to be interacting with a server
when, in fact this was purely a naming|package conceit. You’ll recall we have 3 functions with similar signatures. Of the form:
server.put(&Request{
thing: *thing,
extra: *extra,
}
After creating protobuf definitions of these functions and messages. Then new client code looks similar now using a gRPC client
proxy:
client.PutThing(ctx, &pb.ThingRequest{
Thing: &pb.Thing{...},
Extra: &pb.Extra{...},
})
and the gRPC server proxy is just:
func (s *server) PutThing(ctx context.Context, r *pb.ThingRequest) (*pb.ThingResponse, error) { return s.put(r) }
I’m not wanting to bore you with a gRPC tutorial but, for completeness, pb.ThingRequest
and pb.ThingResponse
types are described in the protobuf definition along with our renamed — but otherwise similar — functions PutThing
, GetThing
, and WaitThing
.
syntax = "proto3";package google.basicpersonality.v1;option go_package = "github.com/DazWilkin/basic-personality/protos";
option java_package = "com.google.dazwilkin.basicpersonality";service BasicPersonality {
rpc PutThing(ThingRequest) returns (ThingResponse) {};
rpc GetThing(ThingRequest) returns (ThingResponse) {};
rpc WaitThing(ThingRequest) returns (ThingResponse) {};message ThingRequest {
Thing thing = 1;
Extra extra = 2;
}
message ThingResponse {
string status = 1;
}
message Thing {
string name = 1;
}
message Extra {
string name = 1;
}
All that we need do is provide the above definition to the protobuf compiler (protoc
) , configure it to generate Golang bindings for us and, we’re almost done.
We then need to decouple the original main.go
into a client and a server that each use the machine-generated gRPC bindings for BasicPersonality
and let the gRPC runtime take care of passing messages between them.
Docker Compose
As mentioned previously, the Docker Compose file is extended to run the basic-personality-client
and basic-personality-server
alongside the Database, a Trillian Log Server and Log Signer, and Prometheus. We configure each with the gRPC port (:50051
) to bind them together.
The client is configured to run forever so albeit with 15-second pauses. When you up
the Docker Compose file, be careful not to leave it running as you’ll eventually blow up the database.
Conclusion
We now have a very basic Trillian personality called “Basic Personality” that is implemented as Golang gRPC server that itself uses gRPC to communicate with the Trillian services. For simplicity, we generated a Golang gRPC client that integrates with our Basic Personality server.
Because Basic Personality has a protobuf service definition and exposes a gRPC endpoint using this definition, you could take the protobuf and generate a client in any gRPC language to talk with the service.
In the next steps, I think I’m going to (a) instrument these services using OpenCensus; (b) generate a non-Golang gRPC client. I Googled Rust gRPC and it seems non-trivial for a Rust beginner. If anyone reading this would like to show how this could be done, I’d be interested.