Constraint based APIs for async/await

New patterns enabled by async functions.

The more I live with async/await the more I discover new patterns and leave behind many old ones.

One constraint of working with async/await is that we only have a single return value from async functions. In the old callback world, you could pass more than one success object to the callback.

In addition to multiple callback results, you also had the option of returning a stream from your function (while still accepting a callback) which allowed you to do hybrid APIs that accepted steam input/output in addition to function parameters and callbacks.

I used this heavily in request. The goal of request was always to reduce the amount of typing you had to do in order to use HTTP. But even as flexible as this API became, there was some common boilerplate you can find almost everywhere people use request.

request(url, {json: true}, (err, resp, body) => {
  if (err) return cb(err)
  if (resp.statusCode !== 200) return cb(new JSONError(body))
  // success!
  console.log(body)
})

In practice, I tend to write small functions on top of request to handle this boilerplate. But as I started taking async/await for granted these constraints began to weigh on me more and more. Eventually, I started thinking about an alternative approach.

Constraints instead of features.

request implements many, many, features which you enable with different input options. When enabled, some of these features also impose constraints (enabling JSON decoding means non-JSON responses will fail) but these features are so general purpose that the constraints can’t expand enough to what you normally want (enabling JSON decoding DOES NOT impose that the status code must be 200).

When I wrote request I had already spent a lot of time with HTTP. I had done a lot in Python and eventually worked on Node.js Core HTTP adding features that also impacted request. request has always been an HTTP client from the perspective of someone who knows HTTP. But after 8 years of using request I have a much better idea of what a better pattern is for everyday use.

One big shift in my thinking is that the client shouldn’t offer features, it should offer constraints. I want to tell the client what is acceptable and build a single narrow path to success the client can impose without any other code on my part.

This fits well with the constraints of async/await. The more we constrain the success path the easier it is to get to a single return value and the closer that return value is to what the user wants it to be.

Introducing bent

let request = bent()
let response = await request(url)

This looks like nothing, but it’s actually already quite constrained by bent’s defaults. The method is GET, the status code must be 200, the return value is the response object (which is also a stream). But let’s do a bit more.

let api = bent('https://api.site.com/v1', 'json')
let obj = await api('/entry/point.json')

This is something I’m surprised I’ve never seen in HTTP clients given how common it is for a client to only operate on a single API service in a given code file. With bent, we create an async function prepared just for this API service, which is a JSON service at a base URL. The return values are now decoded JSON, any other response will generate an error.

Maybe there’s already a fancy computer science term for “function that returns an async function” but that’s not the part of this pattern that interests me. I’m much more interested in moving towards APIs that ask for “what is allowed” rather than just “what do you want to add.”

let put = bent('PUT', 'https://api.site.com/v1', 201)
await put('/upload', Buffer.from('example'))

This function will also accept a stream.

await put('/upload', fs.createReadStream('./file.txt'))

Another interesting thing about async/await is that we get to normalize input values (Buffer, JSON, or Stream). With callbacks you had an incentive to use the pipe interface when working with streams because it could often get you out of also requiring a callback. With async/await there’s no such benefit. In fact, you want the return value to remain the same between different input types so stream just becomes another parameter.

P.S. Like all the modules I write now, there’s 100% code coverage and automated releases w/ semantic-release. While this module is new, it’s actually pretty solid and on good footing 😀