Create a Public API with gRPC

6 min readMar 21, 2020

In this article, I’ll demonstrate how to create a public API with gRPC. The application code itself will be relatively simple and won’t include authentication, with more focus instead on the tooling and methodology around building the service as a whole and exposing the protobufs so that consumers can generate clients from the same protos to use the service.

Learning Objectives

Create a Protocol Buffer library
Create a gRPC service implementation (Java)
Create a gRPC client implementation (Node)

Requirements

Create a Protocol Buffer Library

We first want to create the protobufs that will describe the API we want to build. In this example, I’m creating a service to convert money between currencies. Let’s say this is part of a larger numbers-oriented service.

Initialize the project with protop

If you haven’t installed protop before, you can do so either from Homebrew ( brew tap protop/io && brew install protop) or from the source following the instructions here.

$ mkdir numbers-protos && cd $_
$ protop init

Following the prompt, I’ll create a basic protop manifest:

Organization name (required): awesomelabs
Project name (required): numbers
Initial revision (default 0.1.0):

We won’t have to do anything with it after this point, but here’s the protop.json that was generated:

{
  "organization" : "awesomelabs",
  "name" : "numbers",
  "version" : "0.1.0",
  "include" : [ "." ],
  "dependencies" : { }
}

Create some protos

Now, I’ll create a small proto file to define a money service. Since this tutorial isn’t about writing protos themselves, I’ll skip going through it in depth; here’s the proto I’ve created:

This basically just defines a few messages that hold information about money and a service with a single RPC to convert a value from one currency to the other.

To make it available to other projects, I can either publish locally (protop link), publish to an external registry (protop publish), or publish to a VCS platform like Github (just create a normal Git project, and push it up). I’ll do the latter — and you can find that repository here. Now, the library is available to any other project that wants to use it. In the next part, I’ll demonstrate doing that to create a gRPC service that implements the MoneyService proto.

Create a gRPC Service Implementation (Java)

Initialize the application with Gradle

Setting up a Java project is a little more involved than the previous step, but I’m using Gradle to make it easier. Here’s what that looks like for me using Gradle 6.2.1:

$ mkdir numbers-service && cd $_
$ gradle initSelect type of project to generate:
  1: basic
  2: application
  3: library
  4: Gradle plugin
Enter selection (default: basic) [1..4] 2Select implementation language:
  1: C++
  2: Groovy
  3: Java
  4: Kotlin
  5: Swift
Enter selection (default: Java) [1..5] 3Select build script DSL:
  1: Groovy
  2: Kotlin
Enter selection (default: Groovy) [1..2] 1Select test framework:
  1: JUnit 4
  2: TestNG
  3: Spock
  4: JUnit Jupiter
Enter selection (default: JUnit 4) [1..4] 4Project name (default: numbers-service):
Source package (default: numbers.service): awesomelabs.numbers> Task :init
Get more help with your project: https://docs.gradle.org/6.2.1/userguide/tutorial_java_projects.htmlBUILD SUCCESSFUL in 48s
2 actionable tasks: 2 executed

Make it a protop project as well

Next, I’ll make this into a protop project so that I can import the numbers-protos library. Instead of going through the protop init interface, I can also of course just create a protop.json manually. Here’s what that looks like so far:

{
  "organization" : "awesomelabs-internal",
  "name" : "numbers-service",
  "version" : "0.1.0",
  "include" : [ "." ],
  "private": true,  
  "dependencies" : { }
}

Now, I just need to add my protos library as a dependency. Since the awesomelabs/numbers library was published to a Github repository, I’ll use that URL instead of a version, prepended with “git:” so protop knows that it’s definitely a Git repository (note: Medium is wrapping the lines here):

"dependencies": {
  "awesomelabs/numbers": "git:https://github.com/jefferyshivers/numbers-protos"
}

Sync the external protobuf dependencies

This should be enough to run protop sync and import the dependencies for the project:

$ protop sync
Syncing external dependencies.
Done syncing.

And now if I look inside the project’s protop path in the project directory, I can see that the dependency was imported as expected from Github:

$ ls .protop/path/awesomelabs/numbers/
README.md money.proto protop.json

I’ll add that path to my Gradle configuration so that the protoc compiler can use those protos to generate the classes needed for the service implementation. After setting up my build.gradle to use the protobuf plugin, I just need to add the protop path as a proto source directory:

sourceSets {
    main {
        proto {
            srcDirs += ".protop/path"
        }
        ...
    }
}

To make things easier so that I don’t have to run protop sync manually, I can create a simple command line task in Gradle, and make it execute before protoc generates anything:

You’ll notice that I added a flag --git-refresh; this tells protop to refresh the Git sources every time sync is run. When it does this, it will re-clone the repository rather than pulling. While this might seem expensive, it means you can be as specific as you need with any dependencies down to the commit (e.g. https://github.com/<repo_name>/tree/<commit_sha>) without worrying that using --git-refresh will pull newer revisions if you don’t want them. Since I wasn’t that specific with this dependency, I’m ok with getting the latest version from the master branch every time.

When we build the project now, the output logs should look something like this:

$ gradle clean && gradle buildBUILD SUCCESSFUL in 720ms
1 actionable task: 1 executed> Task :protop
Syncing external dependencies.
Done syncing.BUILD SUCCESSFUL in 4s
12 actionable tasks: 12 executed

Implement the gRPC service

Finally, the generated protos (in this case, Java classes) should have been generated such that I can actually write out the service. Here’s a simple implementation of the service with the convert RPC call:

You can see the full project here; the main files to look at are the build.gradle, awesomelabs.numbers.App, and awesomelabs.numbers.MoneyService. And now I should be able to start the service:

gradle clean && gradle runBUILD SUCCESSFUL in 1s
1 actionable task: 1 executed> Task :protop
Syncing registered dependencies.
Done syncing.> Task :run
Server started on port 8080
<===========--> 87% EXECUTING [15s]
> :run
|

Create a gRPC Client Implementation (Node)

Now that the money service is running, it should be possible to create a gRPC client and interact with it. Since it doesn’t matter what language the service or client are in, I’ll do this part in Node.

mkdir numbers-node-client && cd $_

Create a new protop project

As with the service implementation, I’ll create a protop manifest and add the awesomelabs/numbers package as a dependency.

{
  "name": "numbers-node-client",
  "version": "0.1.0",
  "organization": "awesomelabs",
  "include": [ "." ],
  "dependencies": {
  "awesomelabs/numbers": "git:https://github.com/jefferyshivers/numbers-protos"
  }
}

If you’re new to gRPC (or to using it with Node), I created the client to pretty closely mirror the example Node client in the official gRPC documentation. Here’s what it ended up looking like:

As you can see, I am dynamically loading the protos from the protop path, and then creating a service client. The gRPC library allows me to use JSON to build the request protos and will handle serializing them in the RPC. Now, running the script in the project directory should result in console output similar to the following (with different numbers since they are always randomized):

$ node .
Original: 7.3406037480265685 USD
Converted: 5.33649466631217 MXN

And that’s it! I can continue to create any number of service implementations or clients using that original protobuf library without adding any fancy pipelines to generate artifacts.

Conclusion

I hope that this example helps demonstrate that protocol buffers and gRPC can be easier to work with than they initially might seem. They are great technologies with somewhat intimidating first impressions, but ultimately, with the right tools and development patterns, they can make your life building robust projects — whether it’s a public API, an internal microservice mesh, or even just a simple library of protocol buffers for others to build upon.

Examples on Github

I’ve published all the examples used in this tutorial to Github. Feel free to clone them, or use them as a basis for a new project. I’m also interested to see other examples out there, so please let me know if you’d like for me to share what you’ve created and I’ll gladly link to it here!