A simple Retrofit-style JSON-RPC client

Implementing a Retrofit for JSON-RPC over web-socket.

In our company we write in Kotlin and use RxJava for asynchronous operations in Android. When I joined the company as a junior Android developer more than two years ago, I did not have any experience with Kolin what so ever, and never even heard of Rx. I had to learn it there by writing code at work and after a few months I was hooked. Now I literally have no idea how to create Android apps without Rx.

One of my colleagues, who was my mentor at that time, used to say:

If you have a problem — Rx it. Now you have an Observable<Problem>.

But this post is not about RxJava. So, bear with me.

Slightly more than a year ago I was working on Android library for network communication, where I had to implement a simple Rx-wrapper around OkHttp web-socket client. A client would listen to various notifications from the server and handle them appropriately.

Since web-socket provides a bi-directional communication channel over http, in addition to server notifications, we needed to implement a simple RPC over web-socket that would allow the client to request data from the server over the same connection. And because we already used JSON everywhere, we decided not to reinvent the wheel and just use already existing JSON-RPC protocol.

Having chosen the protocol, we narrowed down the problem to implementing a convenient JSON-RPC client over web-socket.

JSON-RPC

JSON-RPC describes a way to implement asynchronous requests to the server using Json format (the word asynchronous here means that responses can come out of order). It’s very generic and defines a few simple types and commands.

Here are examples of request and response in JSON-RPC 2.0:

--> {
        "jsonrpc": "2.0", 
        "method": "subtract", 
        "params": {
            "minuend": 42, 
            "subtrahend": 23
        }, 
        "id": 3
    }
<-- {
        "jsonrpc": "2.0", 
        "result": 19, 
        "id": 3
    }

If client sends multiple requests, responses might come in a different order and we need to match a response to its request. To address this problem, client generates a unique (within the scope of current connection) id and assigns it to request. This id is later returned back to the client “as-is” as part of response.

The id can be anything, from string to a complex object (protocol does not restrict the type). And because sometimes it might be problematic for the server to handle any type, one should stick to a primitive type. JSON-RPC 2.0 doesn’t require all requests to have an id. It can be omitted if request does not imply any response.

The name of the remote method that is being called is specified by method parameter, and request payload is passed via params as un-ordered arg-value pairs.

Response contains either result or error field and has the same id that was retained from request. The protocol doesn’t impose any restrictions on the result, it should be up to your API. But error is strictly defined as an object with code, message and data fields.

Both request and response have "jsonrpc": "2.0" marker, which can be used to distinguish JSON-RPC related stuff from other data.

More information about JSON-RPC can be found on Wikipedia and official website.

RxWebSocket

So far there have been written tons of reactive wrappers around different non-reactive libraries with ugly APIs. I found more than 10 different implementations of RxWebSocket in different languages on Github (some are actually for okhttp web-socket). I’m not saying that web-socket API is ugly, but web-socket is inherently reactive, it is basically a stream of frames. So, it is quite natural that people wrap it into Rx.

Back in 2016, because there were no alternatives, we created our own RxWebsocket with the following API:

interface RxWebSocket<T> {
    fun sendMessage(message: String): Single<Unit>
    fun observeMessages(): Observable<T>
    fun observeState(): Observable<RxWebSocketState>
}

This interfaces provided an abstraction that allowed us to interact with web-socket in a reactive manner. We can send or receive messages with any content and monitor socket state. The next task was to implement a JSON-RPC client with it.

JsonRpcClient — API

Our client only need to have a single method and since we are working over a “reactive web-socket”, we this single method should return an Observable<T>. If we look again at the protocol that we are trying to implement, each request will have one or no response. To express this semantics the return type should actually Maybe<T>, but to keep things simple I’ll just use Single<T>.

interface JsonRpcClient {
    fun <R> call(request: JsonRpcRequest, type: Type): Single<R>
}

And of course, we need to define classes for request, response and error:

class JsonRpcRequest(
  val id: Long,
  val method: String,
  val params: Map<String, Any?> = emptyMap()
)
class JsonRpcResponse(
  val id: Long,
  val result: JsonElement,
  val error: JsonRpcError
)
class JsonRpcError(
  val code: Int,
  val message: String
)

JsonRpcClient — implementation

We need to implement the following algorithm:

1. Serialize and send JSON-RPC request.
2. Subscribe to the stream of incoming messages.
3. Take the first JSON-RPC response where response.id == request.id.
4. Parse response and cast it to target type R or return error.

Here is an implementation with RxJava 2. It is a bit too long but once you take a closer look, it should not be a problem to understand it.

A simple implementation of JSON-RPC client over RxWebSocket

Our implementation uses Gson library for serialization / deserialization. However, you can always use Jackson, Moshi or whatever you like. In fact, client shouldn’t be aware of serialization at all: serializer and deserializer could use completely different mechanisms and should be passed to the client as interfaces.

UPD: Above is the updated version of the code. The only thing that changed was when we subscribe to captureResponse(). We should do it before we send request to the websocket and not after.

Removing boilerplate

Having implemented our client this way, we have to write not so beautiful code when making requests.

val client: JsonRpcClient = ...
val request = JsonRpcRequest(
                id = generateUniqueId(),
                method = "myMethod",
                params = mapOf(
                           "param1" to 100,
                           "param2" to "value",
                           "param3 to listOf(1, 2, 3)
                )
)
val responseType = typeToken<MyResponse>()
val result = client.call<MyResponse>(request, responseType)

I see at least three obvious disadvantages:

• We need to generate unique request ids.
• We need to manually build a request object for all requests.
• Code has poor readability and error-prone

To make our lives easier, we can steal the approach used in Retrofit.

interface MyService {
    @JsonRpc("myMethod")
    fun myMethod(@JsonRpc("param1") a: Int, 
                 @JsonRpc("param2") b: String, 
                 @JsonRpc("param3") c: List<Int> = emptyList()
    ): Single<MyResponse>
  
}

Annotations allow us to build the request at invocation time via reflection. All method definitions are in one place. And since we are using Kotlin, we can provide default parameter values.

The easiest way to implement annotation-based approach, is to use reflection. We can create a proxy class, which will implement MyService interface and intercept all method invocations at runtime. Reflection allows us to get access to the method being invoked, arguments and return type.

Retrofit for JSON-RPC

With this implementation of runtime annotation handling the same request from the above can be significantly simplified:

val client: JsonRpcClient = ...
val service = createJsonRpcService(MyService::class, client)
val result = service.myMethod(100, "value", listOf(1, 2, 3, 4))

What can be improved?

Our client is still rather simple and doesn’t know anything about request timeouts and connection failures. We do not need to wait for response

• after connection was closed;
• after certain timeout.

In addition, when comparing with HTTP client, there is no response caching. All of these can be implemented by the client itself (to make it smarter), as an extension to it or just as wrapper (although, caching will require some modifications of the protocol).

I will not go into details here, but you can easily implement timeouts via Single.timeout() operator of RxJava. Connection failures can be handled using RxWebSocket.observeState().

Conclusion

We created a simple JSON-RPC client, that allows you to interact with remote server over web-socket connection, as if you had a simple HTTP protocol and Retrofit.

Although, this approach is using reflection and adds some unnecessary overhead, the performance impact is negligible. In exchange we get the code that is easier to read and maintain.

All the code in one gist: http://bit.ly/2gkpIhN