[Tutorial, Part 3] How to develop Go gRPC microservice with HTTP/REST endpoint, middleware, Kubernetes deployment, etc.
This is the continuation of the Part 2. The result of previous part is gRPC service and HTTP/REST endpoint. This part is dedicated to how to add middleware to gRPC service and HTTP/REST endpoint as well. You can find full source code for Part 3 here.
Step 1: Add Uber zap logger
First step is to replace standard Go log by Uber zap. Zap is supported by gRPC middleware framework that is the reason to use it for this tutorial.
Create logger.go file in the “pkg/logger” folder with the following content:
Update “pkg/cmd/server.go” to add zap logger initialization:
Replace standard logger by zap for “pkg/protocol/grpc/server.go” file. See difference here.
Replace standard logger by zap for “pkg/protocol/rest/server.go” file. See difference here.
Step 2: Add gRPC service logging/tracing middleware
There is great library to add middleware to gRPC service:
We are going to use it for logging/tracing.
Create logger.go file in the “pkg/protocol/grpc/middleware” folder with the following content:
Update “pkg/protocol/grpc/server.go” file with the following content to add logging/tracing to gRPC server:
The result project structure should looks like this:
Lets check how logging works.
Start gRPC server with DEBUG (-log-level=-1) logging level:
cd cmd/servergo build .server.exe -grpc-port=9090 -http-port=8080 -db-host=<HOST>:3306 -db-user=<USER> -db-password=<PASSWORD> -db-schema=<SCHEMA> -log-level=-1 -log-time-format=2006-01-02T15:04:05.999999999Z07:00
Open another terminal to build and run HTTP/REST client:
cd cmd/client-restgo build .client-rest.exe -server=http://localhost:8080
Go back to gRPC server and take a look at the terminal output. If we see something like this:
API server listening at: 127.0.0.1:2345
{"level":"info","ts":"2018-09-16T09:54:16.8474966+03:00","msg":"starting HTTP/REST gateway..."}
{"level":"info","ts":"2018-09-16T09:54:16.8484985+03:00","msg":"starting gRPC server..."}
{"level":"info","ts":"2018-09-16T09:54:16.8554788+03:00","msg":"pickfirstBalancer: HandleSubConnStateChange: 0xc00016c060, READY","system":"grpc","grpc_log":true}
{"level":"debug","ts":"2018-09-16T09:54:28.5287447+03:00","msg":"finished unary call with code OK","peer.address":"[::1]:49838","grpc.start_time":"2018-09-16T09:54:28+03:00","system":"grpc","span.kind":"server","grpc.service":"v1.ToDoService","grpc.method":"Create","peer.address":"[::1]:49838","grpc.code":"OK","grpc.time_ms":99.84400177001953}
{"level":"debug","ts":"2018-09-16T09:54:28.6258332+03:00","msg":"finished unary call with code OK","peer.address":"[::1]:49838","grpc.start_time":"2018-09-16T09:54:28+03:00","system":"grpc","span.kind":"server","grpc.service":"v1.ToDoService","grpc.method":"Read","peer.address":"[::1]:49838","grpc.code":"OK","grpc.time_ms":38.01499938964844}
{"level":"debug","ts":"2018-09-16T09:54:28.6695219+03:00","msg":"finished unary call with code OK","peer.address":"[::1]:49838","grpc.start_time":"2018-09-16T09:54:28+03:00","system":"grpc","span.kind":"server","grpc.service":"v1.ToDoService","grpc.method":"Update","peer.address":"[::1]:49838","grpc.code":"OK","grpc.time_ms":42.224998474121094}
{"level":"debug","ts":"2018-09-16T09:54:28.6873289+03:00","msg":"finished unary call with code OK","peer.address":"[::1]:49838","grpc.start_time":"2018-09-16T09:54:28+03:00","system":"grpc","span.kind":"server","grpc.service":"v1.ToDoService","grpc.method":"ReadAll","peer.address":"[::1]:49838","grpc.code":"OK","grpc.time_ms":16.808000564575195}
{"level":"debug","ts":"2018-09-16T09:54:28.7319168+03:00","msg":"finished unary call with code OK","peer.address":"[::1]:49838","grpc.start_time":"2018-09-16T09:54:28+03:00","system":"grpc","span.kind":"server","grpc.service":"v1.ToDoService","grpc.method":"Delete","peer.address":"[::1]:49838","grpc.code":"OK","grpc.time_ms":41.12799835205078}then everything works fine.
Step 3: Add middleware for HTTP/REST gateway
We are going to add two middleware for HTTP/REST gateway:
- Request-ID
- Logging/Tracing
Request-ID injects unique ID to HTTP request context. Logging/Tracing middleware is using ID to relate HTTP response with corresponding HTTP requests.
Create request-id.go file in the “pkg/protocol/rest/middleware” folder with the following content:
Then create logger.go file in the “pkg/protocol/rest/middleware” folder with the following content:
Update “pkg/protocol/rest/server.go” file with the following content to add logging/tracing to HTTP gateway:
The result project structure should looks like this:
Lets check how logging works.
Start gRPC server with DEBUG (-log-level=-1) logging level:
cd cmd/servergo build .server.exe -grpc-port=9090 -http-port=8080 -db-host=<HOST>:3306 -db-user=<USER> -db-password=<PASSWORD> -db-schema=<SCHEMA> -log-level=-1 -log-time-format=2006-01-02T15:04:05.999999999Z07:00
Open another terminal to build and run HTTP/REST client:
cd cmd/client-restgo build .client-rest.exe -server=http://localhost:8080
Go back to gRPC server and take a look at the terminal output. If we see something like this:
API server listening at: 127.0.0.1:2345
{"level":"info","ts":"2018-09-16T10:28:30.0875991+03:00","msg":"starting HTTP/REST gateway..."}
{"level":"info","ts":"2018-09-16T10:28:30.0886057+03:00","msg":"starting gRPC server..."}
{"level":"info","ts":"2018-09-16T10:28:30.0936242+03:00","msg":"pickfirstBalancer: HandleSubConnStateChange: 0xc000052070, READY","system":"grpc","grpc_log":true}
{"level":"debug","ts":"2018-09-16T10:28:42.3320283+03:00","msg":"request started","request-id":"PC/5P2xo0mYah-000001","http-scheme":"http","http-proto":"HTTP/1.1","http-method":"POST","remote-addr":"[::1]:50181","user-agent":"Go-http-client/1.1","uri":"http://localhost:8080/v1/todo"}
{"level":"debug","ts":"2018-09-16T10:28:42.4345359+03:00","msg":"finished unary call with code OK","peer.address":"[::1]:50178","grpc.start_time":"2018-09-16T10:28:42+03:00","system":"grpc","span.kind":"server","grpc.service":"v1.ToDoService","grpc.method":"Create","peer.address":"[::1]:50178","grpc.code":"OK","grpc.time_ms":100.51200103759766}
{"level":"debug","ts":"2018-09-16T10:28:42.4345359+03:00","msg":"request completed","request-id":"PC/5P2xo0mYah-000001","http-scheme":"http","http-proto":"HTTP/1.1","http-method":"POST","remote-addr":"[::1]:50181","user-agent":"Go-http-client/1.1","uri":"http://localhost:8080/v1/todo","elapsed-ms":102.5076}
{"level":"debug","ts":"2018-09-16T10:28:42.5072257+03:00","msg":"request started","request-id":"PC/5P2xo0mYah-000002","http-scheme":"http","http-proto":"HTTP/1.1","http-method":"GET","remote-addr":"[::1]:50181","user-agent":"Go-http-client/1.1","uri":"http://localhost:8080/v1/todo/29"}
{"level":"debug","ts":"2018-09-16T10:28:42.5457395+03:00","msg":"finished unary call with code OK","peer.address":"[::1]:50178","grpc.start_time":"2018-09-16T10:28:42+03:00","system":"grpc","span.kind":"server","grpc.service":"v1.ToDoService","grpc.method":"Read","peer.address":"[::1]:50178","grpc.code":"OK","grpc.time_ms":37.51100158691406}
{"level":"debug","ts":"2018-09-16T10:28:42.5457395+03:00","msg":"request completed","request-id":"PC/5P2xo0mYah-000002","http-scheme":"http","http-proto":"HTTP/1.1","http-method":"GET","remote-addr":"[::1]:50181","user-agent":"Go-http-client/1.1","uri":"http://localhost:8080/v1/todo/29","elapsed-ms":38.5138}
{"level":"debug","ts":"2018-09-16T10:28:42.5467358+03:00","msg":"request started","request-id":"PC/5P2xo0mYah-000003","http-scheme":"http","http-proto":"HTTP/1.1","http-method":"PUT","remote-addr":"[::1]:50181","user-agent":"Go-http-client/1.1","uri":"http://localhost:8080/v1/todo/29"}
{"level":"debug","ts":"2018-09-16T10:28:42.590229+03:00","msg":"finished unary call with code OK","peer.address":"[::1]:50178","grpc.start_time":"2018-09-16T10:28:42+03:00","system":"grpc","span.kind":"server","grpc.service":"v1.ToDoService","grpc.method":"Update","peer.address":"[::1]:50178","grpc.code":"OK","grpc.time_ms":42.492000579833984}
{"level":"debug","ts":"2018-09-16T10:28:42.590229+03:00","msg":"request completed","request-id":"PC/5P2xo0mYah-000003","http-scheme":"http","http-proto":"HTTP/1.1","http-method":"PUT","remote-addr":"[::1]:50181","user-agent":"Go-http-client/1.1","uri":"http://localhost:8080/v1/todo/29","elapsed-ms":43.4932}
{"level":"debug","ts":"2018-09-16T10:28:42.5913265+03:00","msg":"request started","request-id":"PC/5P2xo0mYah-000004","http-scheme":"http","http-proto":"HTTP/1.1","http-method":"GET","remote-addr":"[::1]:50181","user-agent":"Go-http-client/1.1","uri":"http://localhost:8080/v1/todo/all"}
{"level":"debug","ts":"2018-09-16T10:28:42.6090751+03:00","msg":"finished unary call with code OK","peer.address":"[::1]:50178","grpc.start_time":"2018-09-16T10:28:42+03:00","system":"grpc","span.kind":"server","grpc.service":"v1.ToDoService","grpc.method":"ReadAll","peer.address":"[::1]:50178","grpc.code":"OK","grpc.time_ms":17.74799919128418}
{"level":"debug","ts":"2018-09-16T10:28:42.6111404+03:00","msg":"request completed","request-id":"PC/5P2xo0mYah-000004","http-scheme":"http","http-proto":"HTTP/1.1","http-method":"GET","remote-addr":"[::1]:50181","user-agent":"Go-http-client/1.1","uri":"http://localhost:8080/v1/todo/all","elapsed-ms":19.8139}
{"level":"debug","ts":"2018-09-16T10:28:42.6121388+03:00","msg":"request started","request-id":"PC/5P2xo0mYah-000005","http-scheme":"http","http-proto":"HTTP/1.1","http-method":"DELETE","remote-addr":"[::1]:50181","user-agent":"Go-http-client/1.1","uri":"http://localhost:8080/v1/todo/29"}
{"level":"debug","ts":"2018-09-16T10:28:42.6541939+03:00","msg":"finished unary call with code OK","peer.address":"[::1]:50178","grpc.start_time":"2018-09-16T10:28:42+03:00","system":"grpc","span.kind":"server","grpc.service":"v1.ToDoService","grpc.method":"Delete","peer.address":"[::1]:50178","grpc.code":"OK","grpc.time_ms":42.05500030517578}
{"level":"debug","ts":"2018-09-16T10:28:42.6541939+03:00","msg":"request completed","request-id":"PC/5P2xo0mYah-000005","http-scheme":"http","http-proto":"HTTP/1.1","http-method":"DELETE","remote-addr":"[::1]:50181","user-agent":"Go-http-client/1.1","uri":"http://localhost:8080/v1/todo/29","elapsed-ms":42.0551}then everything works fine. Terminal output contains both gRPC and HTTP/REST gateway logs.
Resume for Part 3
That’s all for Part 3. We have added middleware for both gRPC server and HTTP/REST gateway.
Source code for Part 3 is available here.
Part 4 is going to be dedicated how to add Kubernetes deployment configuration with health check and how to build and deploy project to Google Cloud.
Thanks!
