gRPC with curl
gRPC is a fast, efficient, flexible RPC framework built on top of protocol buffers and uses HTTP2 for bi-directional streaming. The only real way to invoke the remote procedure is to use a generated gRPC client which internally does all the marshaling to encode the protbuf messages into gRPC’s wire format:
The procedure described below is a mechanism to invoke a remote GRPC call using and http2-enabled curl and nghttp2 client alone.
This does not serve any real practical purposes other than an investigation into dissecting what goes on in the RPC. The only usage for this is if running a full gRPC client is not possible and what is available is the serialized protocol buffer message to transmit.
You can run the sample here by either installing protobuf and gRPC or entirely through the docker container salrashid123/grpc_curl.
You can find the code and samples here:
QuickStart using docker
src/Dockerfile image includes the precompiled gRPC python client/server and curl+http2 clients. If you want, you can use this docker file entirely to test with.
First familiarize and verify curl and nghttp2 works from within the container
$ docker run salrashid123/grpc_curl curl \
-v — http2 \
https://www.google.com/ $ docker run salrashid123/grpc_curl nghttp \
-vn https://www.google.com/
Start the gRPC server
Now start a GRPC server which will accept the inbound request from curl. If you would rather run this directly on your laptop, please see the instructions towards the end of this article:
docker run -p 50051:50051 \
salrashid123/grpc_curl python /app/server.pyMake the binary file with the delimited gRPC message
Now, lets make the raw gRPC file to transmit. I used a script “message_util.py” as a utility to help with the encoding…you can view what i’ve done there or look towards the end of the article for details on wireformatting
mkdir gcurlcd gcurldocker run -v `pwd`:/tmp/gcurl/ \
-t salrashid123/grpc_curl \
python /app/message_util.py write /tmp/gcurl/frame.bin
Invoke the gRPC server with curl:
Now we’ve got a delimied frame with an embedded gRPC message…lets transmit it with curl and http/2 to the gRPC server we started earlier:
In the you should be left with two files in the ‘gcurl’ folder you created earlier or ran the dockerfile commands: “frame.bin” and “resp.bin”
Verify the response message
The gRPC server will respond back with a message which is saved as “resp.bin”..lets again use the utlity to decode it:
$ docker run -v `pwd`:/tmp/gcurl/ -t salrashid123/grpc_curl python /app/message_util.py read /tmp/gcurl/resp.binGot wire_message: 00000000120a1048656c6c6f2c206a6f686e20646f6521
Proto Decode: Hello, john doe!
What that shows is the response getting decoded via script.
Now, lets do this without docker because this is how we’ll show in detail the wireformatting:
Invoke gRPC with curl locally
The following steps outlines how to call the gRPC server with curl if curl, protoc, gRPC server runs localy
Installing curl, if necessary
First step is to install curl and/or nghttp2 clients that are http/2 aware.
NOTE: first check the verison of curl you have because
curl 7.58.0 already has HTTP/2 support
$ curl — help | grep http2
— http2 Use HTTP 2
otherwise, its the hard way..
Verify curl is enabled with http/2:
curl -v --http2 https://www.google.com/
* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)Installing gRPC support for python
Setting up client to run the gRPC server locally as well as the client to generate and save the protobuf files is preferably done through virtualenv:
Generate the gRPC wireformat binary file
The first step is to actually write the protobuf message in the wireformat.
The following python code write a protobuf message and converts it to the wireformat:
to invoke this command, simply run:
python message_util.py write frame.binThe above snippet writes the message to a binary file in your local directory. For manual encoding to wireformat starting with just the protobuf:
Start with the protbuf in a file by itself (no encoding done as above; just save a binary file with req.SerializeToString())
then
so the Delimited-Message is
000000000b0a046a6f686e1203646f65Run a gRPC server
You can either run the gRPC server directly if you have gRPC tools available:
cd src
python server.pyor via dockerfile
docker run -p 50051:50051 \
salrashid123/grpc_curl \
python /app/server.pyTransmit the wireformat binary file
Now that we have a file ‘frame.bin’ which is the data we want to transmit, transmit and save the output to ‘resp.bin’:
Note: `main.esodemoapp2.com` matches the certificates SAN and points back to localhost:
Decode the Response
The response message is alo in formatted so do the inverse of encoding
xxd -p resp.bin
00000000120a1048656c6c6f2c206a6f686e20646f6521which is:
- compression : 00
- message length: 00000012 -> 18 decimal
- message: 0a1048656c6c6f2c206a6f686e20646f6521
You can use the message utility file do this decoding using the protobuf decoder to do the delimited message -> proto decoder:
Invoking using the gRPC clients
Invoking directly
Assuming you have setup the virtualenv and installed grpc tools, you can call the actual gRPC client/server directly as usual:
python server.pythen (note, main.esodemoapp2.com should be bound on /etc/hosts to 127.0.0.1)
python client.py --host main.esodemoapp2.com 50051Invoking using docker image
THe same client-server python gRPC scripts are present in a docker image
Server:
docker run -p 50051:50051 \
salrashid123/grpc_curl \
python /app/server.pyClient
docker run \
--net=host \
--add-host main.esodemoapp2.com:127.0.0.1 \
-t salrashid123/grpc_curl \
python /app/client.py main.esodemoapp2.com 50051Appendix
gRPC Environment Variables
You can set some environment variables if you use client.py library for gRPC.
export GRPC_TRACE=all
export GRPC_VERBOSITY=DEBUGTCPTraces for request and response gRPC calls
The following traces captures the request and response streams while done over a plain HTTP call:
EchoRequest

EchoResponse

gRPC Streaming
The gRPC server also has response streaming enabled on the “/echo.EchoServer/SayHelloStream” endpoint.
curl -vv -k --raw --http2 \
-H "Content-Type: application/grpc" \
-H "TE: trailers" \
--data-binary @frame.bin https://main.esodemoapp2.com:50051/echo.EchoServer/SayHelloStream \
-o resp.binwhere the response is in the format:
Using Golang http2 client
Using python hyper client
hyper is an experimental http2 client for python. You can use this as well but at the time of writing 8/25/17, I have not been enable to negotiate a proper SSL connection ALPN to proceed.
