Type-driven generation of RESTful API bindings in OCaml

Back in October I finished my first software engineering internship with Cryptosense , a French cryptographic security company. It went largely without incident, despite me speaking slightly less French than your standard multilingual “wet floor” sign.

Overall it was a highly positive experience, and I walked away with a far better understanding of how CS work is done by professional operations. While I’d love to talk about that, right now I’m going to go into a side project I’ve been working on for the past few months.

Lots of exciting opportunities in CS right now involve working with the cloud: web services, social networking and the IoT represent some of the most innovative sectors of one of the most innovative industries in America. As a huge fan of having employment prospects, I’d really like to get in on this, and as a huge fan of self flagellation I’d like to do so with OCaml.

I’ll get started by writing an application find the street address of a point on the Earth, because there happens to be an endpoint in the Google Maps API which does exactly that. Unfortunately, Google does not maintain an OCaml client library for their Maps API, and the community-maintained bindings are meant to be used in client-side code via js_of_ocaml, so I’ll have to write my own bindings. OCaml, however, does not really have a whole lot of (nice) libraries for writing network code, with most of them being very systems-oriented. I don’t want to worry about network headers or manually building GET and POST messages, I would rather use a library that lets me quickly throw together bindings to REST API’s while:

1. Safely assembling complex GET and POST messages
2. Automatically checking that responses represent well-formed JSON/XML documents
3. Marshalling responses into native OCaml data types (I don’t want to deal with Yojson.Safe.Util every time I want to access a member of a JSON object)
4. Handling my networking boilerplate for me
5. Telling me when I’m fucking up: missing a required parameter, not handling exceptional cases, etc.

Netblob accomplishes all of these things: implemented as a ppx_deriving plugin, the netblob annotation may be attached to a record type declaration, which will generate a function that calls the specified endpoint with parameters described by the type declaration. The compiler will emit an error if the function is called without all the required arguments, which are all guaranteed to be reasonable, and the return value is an error-aware data structure, so the compiler is able to guarantee that you’re handling the case of poorly encoded responses.

Boilerplate

In order to build a complete application, we have to build structures to describe the JSON responses described here.

First, I should define a JSON schema to hold addresses, complete with an [@@deriving yojson] annotation to generate JSON (de)serializers for this type. I only want the formatted_address field, so everything else can just be given the Yojson.Safe.json type, as I don’t want to write types for them (this means that ppx_deriving_yojson will make sure that the fields are present and that they are valid JSON, but it will not check that they are schemas)

type address =
  { address_components : Yojson.Safe.json
  ; formatted_address : string
  ; geometry : Yojson.Safe.json
  ; place_id : Yojson.Safe.json
  ; types : Yojson.Safe.json
  }
[@@deriving yojson]

I’ll also need a wrapper type, because apparently there’s no idiomatic way to let an HTTP client know that they fucked up.

type http_result =
  { results : address list
  ; status : string
  }
[@@deriving yojson]

The last step is where Netblob comes in. In order to create a Netblob binding, you have to provide 4 things:

1. The request schema: GET and POST parameters, path parameters (e.g. example.com/page/20), default values, etc.
2. An HTTP verb
3. The API endpoint, e.g. https://maps.googleapis.com/maps/api/geocode/json
4. The format of the expected response (may be Text, Xml, or Json) and an optional function to decode the response into a native OCaml data structure. This allows you to tell Netblob to automatically ensure that it receives a well-formed response, then guarantee that the response adheres to a particular schema and make the data available as a high-performance structure.

For our example, we’re going to make bindings to the Google Maps Geocoding API, at https://maps.googleapis.com/maps/api/geocode/json. Requests to this endpoint usually look like this:

https://maps.googleapis.com/maps/api/geocode/json?latlng=37.23,-41.42&key=SOME_API_KEY

Fortunately, this is quite a simple schema: latlng may be encoded as a pair of floats, and key is just a string.

The GET parameters may be nicely encoded into an OCaml record type, and the Netblob annotation adds the rest of what we need:

type address_of_coords =
  { latlng : float * float
  ; key : string
  }
[@@deriving
  netblob
    { url = "https://maps.googleapis.com/maps/api/geocode/json"
    ; meth = `Get
    ; format = `Json http_result_of_yojson
    }]

The whole program looks like this:

So now we have a fully functioning OCaml binding to a simple REST endpoint, complete with error handling (mostly through error-aware return types option and result) and automatic response validation/marshalling.

Why?

Why should you use Netblob (or something like it) for REST development? Because life is too short to write network code by hand, and it’s also too short to deal with the myriad problems which can arise from sloppily constructing API bindings. Netblob attempts to balance automation with control by providing a predictable system for expanding type declarations into workable network code. You should never have to throw together URI’s or request headers in your application, but few developers maintain OCaml client libraries for their API’s. With Netblob, you can quickly and easily build robust bindings to any REST API without worrying about performance or writing HTTP code by hand.