How We Were Able to Quickly Write Java Clients for Our Microservices

Published in

Knerd

6 min readApr 17, 2018

In this post we will talk about how we were able to reduce the amount of time it takes to build client libraries for our Java HTTP services. We will introduce the connection library we built, the motivation behind it and cover some of its main features.

Some benefits of a microservice oriented architecture are how the smaller codebases make microservices easier to scale, debug, and understand, which also makes refactors and component upgrades easier. One disadvantage to this type of architecture is the upfront cost it takes to spin up a new service. Prioritizing architectural changes and refactors at a product driven company can be very tricky, even more so at smaller tech organizations. At Knewton, we have historically done a pretty good job at providing common libraries that we use across many services, which cut down on the time it takes to spin up a new service. However, one area where we were lacking was having a quick way to create clients for HTTP services, or more generally, an HTTP connection library.

Requirements

Before we set out to build the HTTP connection library, creatively called HttpClient Library, we had the following list of high-level requirements:

Quick and Easy: Since our primary motivation was to cut down the time it takes to deploy a new service we wanted the library to be quick and easy to integrate within our tech stack.
Performant and Resilient: Like any connection library this one too had to be fast and also resilient to transient errors and service outages with retryability and load balancing.
Observable: Client calls needed to be traceable and metered
Flexible: Since we were planning on porting existing services to this new client library we wanted the library to offer flexibility to service owners, including the ability to choose from different low level HTTP clients like OkHttp and Apache.

Approach

To satisfy a lot of the requirements listed above we decided to base the HttpClient Library on Feign + Ribbon, both originally introduced by Netflix. Feign is a library that can “inflate” or proxy your annotated interfaces and generates templated request payloads, making it a great fit for REST APIs. Ribbon is an RPC library with built in support for Eureka discovery as well as load balancing. Eureka also happens to be what we use for service discovery. This allowed us to check a lot of the requirements listed above without too much work on our end. Let’s take a look at our requirements list again and explain how we were able to meet some of them.

Quick and Easy

Feign allows us to quickly autogenerate client side HTTP payloads without much work by reading a user defined annotated interface. This allowed us to have a common interface for both the client and the server. We placed JAX-RS, (a JAVA API Spec for creating RESTful web services) annotations on the interface which can be used by both Feign and a server like Jetty. For example an interface could look something like this:

@Path(“/users”)
@Produces(MediaType.APPLICATION_JSON)
@Consumes(MediaType.APPLICATION_JSON)public interface IUserResource {
    …
    @GET
    @Path(“{userId}”)
    User getUser(@PathParam(“userId”) UserId userId) 
            throws NotFoundException;
    …
}

Passing that interface through the Feign instance factory would create a java proxy object from this interface which would allow the client library to generate the proper payload with all the query, body or path parameters set.

The server side implementation could look something like this:

public class UserResource implements IUserResource {
    …
    @Override
    public User getUser(UserId userId) throws NotFoundException {
        return userDao.getUser(userId);
    }
    …
}

This pattern allowed us to keep our client code — and more importantly documentation — up to date with the server. This also makes server code a little more readable since annotations now go on the interface.

Through the use of Guice (Google’s dependency injection framework), integrating services can also configure and provide these clients to other services very easily. The providing service simply needs to implement a Guice module that would look something like the following:

public class UserClientModule extends PrivateModule {
    …
    @Override
    protected void configure() {
        install(new HttpClientLibraryModule(config));
    }    @Provides
    @Exposed
    @Singleton
    public IUserResource provideIUserResource(
            IHttpClientFactory clientFactory) {
        return clientFactory.create(IUserResource.class);
    }
}

The services which will then be calling an HTTP service would simply need to install this module on their injector to get access to specific resources.

Resiliency through Exception Propagation

A common mistake engineers would make when calling remote HTTP endpoints was to forget to handle exceptional conditions. The advantage of offering a higher level client is that you can explicitly force caller services to handle certain kinds of exceptions. In the case of Java, this can be done through the use of checked exceptions. Another advantage is that you can go beyond the general exceptions and provide more specialized service specific exceptions. This was a common pattern we follow in other services that we implemented with Thrift, through the use of a common exception library. We wanted to carry over that pattern and build exception propagation from the server to the client from the get go.

Propagating these exceptions to the client allows us to force the caller to deal with certain kinds of exceptions in order to retry in a more meaningful way or abort the current operation in the calling service more gracefully. With the common client/server interface described above, this is even easier to do.
Combined with a common exception hierarchy, exceptions can also indicate whether an operation is retryable, which can then be handled by the library instead of at the application layer.

To achieve this, we bundle an exception mapper on the server side which serializes parts of the exception into the body of the response, which is then deserialized on the client side. The client decoder will inspect the exception and decide whether it’s retryable or not. If it is, it will proceed to retry the operation with some backoff, or otherwise it will propagate the exception back up to the application layer. This allows us to write client code that looks like this without any extra effort from the developers who are integrating with the library:

try {
    serviceBClient.getUser(userId);
} catch (NotFoundException e) { // create the user if not present
    serviceBClient.createUser(userId);
    // Deal with any other checked exceptions here
} catch (UnretryableException e) {
    // gracefully handle this
}

Extra care was needed when crossing to a client service that is written in another language so we only partially serialize exceptions in JSON and we make sure that the equivalent types exist in the receiving service’s programming language. Of course, the client side could also just completely ignore the extra exception content.

Observable

To make visibility and measurability easier we integrated the connection library with our internal distributed tracing library as well as our internal metrics library which reports metrics to graphite. Fortunately Feign makes it easy to provide a custom invocation handler. The handler gets invoked on every proxy method invocation which allows us to record metrics before and after a method invocation proceeds.

Flexible

Something we had to deal with was migrating existing client code to the new library. Existing services were using various flavors of lower level HTTP clients, like OkHTTP and Apache, some with custom request and response interceptors. Fortunately, Ribbon allows users to specify what low level HTTP client they want to use so all we had to do was to bundle two separate modules for our clients to choose from.

To conclude, we hope that some of the ideas here will prove to be useful to the rest of the tech community. We’ve been using the HttpClient Library for the past year now and we’ve certainly seen the time it takes to spin up a new service significantly go down. Debugging problems has also become a little easier since the same patterns are now used across codebases, so they have more consistent behavior.

Special thanks to Kevin Rathbun for his contributions.