gRPC with curl

salmaan rashid
Aug 26, 2017 · 5 min read

gRPC is a fast, efficient, flexible RPC framework built on top of protocol buffers and uses HTTP2 for bi-directional streaming. The only real way to invoke the remote procedure is to use a generated gRPC client which internally does all the marshaling to encode the protbuf messages into gRPC’s wire format:

The procedure described below is a mechanism to invoke a remote GRPC call using and http2-enabled curl and nghttp2 client alone.

This does not serve any real practical purposes other than an investigation into dissecting what goes on in the RPC. The only usage for this is if running a full gRPC client is not possible and what is available is the serialized protocol buffer message to transmit.

You can run the sample here by either installing protobuf and gRPC or entirely through the docker container salrashid123/grpc_curl.

You can find the code and samples here:


QuickStart using docker

First familiarize and verify curl and nghttp2 works from within the container

$ docker run salrashid123/grpc_curl curl \
-v — http2 \
https://www.google.com/
$ docker run salrashid123/grpc_curl nghttp \
-vn https://www.google.com/

Start the gRPC server

docker run -p 50051:50051 \
salrashid123/grpc_curl python /app/server.py

Make the binary file with the delimited gRPC message

mkdir gcurlcd gcurldocker run -v `pwd`:/tmp/gcurl/ \
-t salrashid123/grpc_curl \
python /app/message_util.py write /tmp/gcurl/frame.bin

Invoke the gRPC server with curl:

In the you should be left with two files in the ‘gcurl’ folder you created earlier or ran the dockerfile commands: “frame.bin” and “resp.bin”

Verify the response message

$ docker run  -v `pwd`:/tmp/gcurl/ -t salrashid123/grpc_curl  python /app/message_util.py    read /tmp/gcurl/resp.binGot wire_message: 00000000120a1048656c6c6f2c206a6f686e20646f6521
Proto Decode: Hello, john doe!

What that shows is the response getting decoded via script.

Now, lets do this without docker because this is how we’ll show in detail the wireformatting:

Invoke gRPC with curl locally

Installing curl, if necessary

NOTE: first check the verison of curl you have because

curl 7.58.0 already has HTTP/2 support

$ curl — help | grep http2
— http2 Use HTTP 2

otherwise, its the hard way..

Verify curl is enabled with http/2:

curl -v --http2 https://www.google.com/
* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)

Installing gRPC support for python

Generate the gRPC wireformat binary file

The following python code write a protobuf message and converts it to the wireformat:

to invoke this command, simply run:

python message_util.py write frame.bin

The above snippet writes the message to a binary file in your local directory. For manual encoding to wireformat starting with just the protobuf:

Start with the protbuf in a file by itself (no encoding done as above; just save a binary file with req.SerializeToString())

then

so the Delimited-Message is

000000000b0a046a6f686e1203646f65

Run a gRPC server

cd src
python server.py

or via dockerfile

docker run -p 50051:50051 \
salrashid123/grpc_curl \
python /app/server.py

Transmit the wireformat binary file

Note: `main.esodemoapp2.com` matches the certificates SAN and points back to localhost:

Decode the Response

xxd -p resp.bin 
00000000120a1048656c6c6f2c206a6f686e20646f6521

which is:

  • compression : 00
  • message length: 00000012 -> 18 decimal
  • message: 0a1048656c6c6f2c206a6f686e20646f6521

You can use the message utility file do this decoding using the protobuf decoder to do the delimited message -> proto decoder:


Invoking using the gRPC clients

Invoking directly

python server.py

then (note, main.esodemoapp2.com should be bound on /etc/hosts to 127.0.0.1)

python client.py --host main.esodemoapp2.com 50051

Invoking using docker image

Server:

docker run -p 50051:50051 \
salrashid123/grpc_curl \
python /app/server.py

Client

docker run \
--net=host \
--add-host main.esodemoapp2.com:127.0.0.1 \
-t salrashid123/grpc_curl \
python /app/client.py main.esodemoapp2.com 50051

Appendix

gRPC Environment Variables

export GRPC_TRACE=all
export GRPC_VERBOSITY=DEBUG

TCPTraces for request and response gRPC calls

EchoRequest

EchoResponse

gRPC Streaming

curl  -vv -k --raw --http2 \
-H "Content-Type: application/grpc" \
-H "TE: trailers" \
--data-binary @frame.bin https://main.esodemoapp2.com:50051/echo.EchoServer/SayHelloStream \
-o resp.bin

where the response is in the format:

Using Golang http2 client

Using python hyper client

Google Cloud Platform - Community

A collection of technical articles published or curated by Google Cloud Platform Developer Advocates. The views expressed are those of the authors and don't necessarily reflect those of Google.

)

salmaan rashid

Written by

Google Cloud Platform - Community

A collection of technical articles published or curated by Google Cloud Platform Developer Advocates. The views expressed are those of the authors and don't necessarily reflect those of Google.

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade