Building gRPC endpoints with Protobufs

In this article I’ll be going over how to use protobufs to build a remote procedure call (gRPC) service and client. Before I do that, I will explain what a protobuf is.

Protocol Buffers (Protobufs)are Google’s language-neutral, platform-neutral, extensible mechanism for serializing structured data — think XML, but smaller, faster, and simpler. You define how you want your data to be structured once, then you can use special generated source code to easily write and read your structured data to and from a variety of data streams and using a variety of languages. — Google Developer Doc

Basically, this framework can be used to design a schema that will serve as a contract between a service and a client. It also dictates how the 2 different systems should interact. Tons of engineers use protocol buffers as an Interface Design Language to design APIs.

Cool right?

As a Software Engineer, the workflow for creating a service always looks something like this…

1. Scaffold a service with boilerplate code
2. Add endpoint
3. Write test for the endpoint
4. Build client to hit the endpoint
5. Write test for the client
6. Write documentation on how to use both the endpoint/client

Assuming the endpoint business logic is straightforward, each step probably takes about 30 minutes. So creating 1 endpoint would take roughly 3 hours to complete. Imagine now with protobuf You could easily do steps 1, 2, 4, & 5 with a few CLI commands. Now, it would only take you 1 hour total with protobufs. Now you have an extra 2 hours to practice your milly rock 😆

Milly rock on any block, even if it is the Great Wall of China.

In the next sections I’m going to walk you through the basics of how to setup a gRPC service using protobuf.Make sure to set up a virtual environment before you start (virtualenv).

Click here to see how to setup a virtualenv

Create .proto file

First we need to define the interface we want for the service, and the schema for how messages (requests) will be passed. Below is a hellomars proto. You can see the service name is defined as HelloMars and has an rpc endpoint called SayHello . This endpoint takes in a HelloRequest message and returns a HelloReply message, which is defined below the service codeblock. Both HelloRequest and HelloReply take a string value for their respective key values `name` and `message`.

Codegen library

Now that we have our service interface, we can now use grpcio-tools and protoc to generate the service and client library in python. CLI commands don’t actually write the endpoint but give you the code to do it and as a developer all you may need to add is the business logic.

If you run theprotoc help command you can see that you also generate code for other languages by just adding more cli options. For example, if I wanted to use ruby instead my cli option would be --ruby_out={someFileOutputName}. After running the command, two files should be created hellomars_pb2_grpc.py and hellomars_pb2.py . These files will serve as libraries to help construct our service and client.

Implement service file

We now have a library we can leverage to generate the service and client. First, we’ll create the service by extending theHelloMarsServicer from hellomars_pb2_grpc.py . Below is the sample code needed to create the service and implement the endpoint we defined in the hellomars.proto

You can see that we are following the same interface defined by our hellomars protobuf. We have a service called HelloMars and it has 1 function/endpoint called SayHello where the request param will be a HelloRequest python dict and the function will be returning a HelloReply object as the response.

Implement client

Awesome, now we have a service! Let’s build a client so we can send a request to it.

In this code snippet, I’m sending my name (Hugh!) as a message using the hellomars_pb2_grpc.HelloMarsStub class. This python class has all the available request functions a client needs to interact with the given service. In this case we instantiate the class and call SayHello() method with HelloRequest as a param.

Try it out

We now have everything we need now to make call and get response. So all we need to do is instantiate is running the 2 commands in separate terminal window.

# Start service
$ python hellomars_server.py
# Make call to service
$ python hellomars_client.py
>> Mars client received! Hello, Hugh!!

Dope! Now you have officially created your first gRPC service with a client! To see the sample code we’ve generated click here.

Me after building a gRPC endpoint

Written next door 🖥