API Design — Deriving Future

Måns Bernhardt
May 7, 2018 · 11 min read

In this article, we will explain what asynchronous operations are, why they are useful, and how their APIs are typically designed. Specifically, we will look at an API for performing network requests. Step by step, we will improve this API, first by introducing the Result<T> type, and later on, by abstracting the asynchronous operation itself, by deriving the Future<T> type. We will also show how these two types could be implemented and extended with useful operations. In the end, this will lead to code that is more composable, easier to reason about as well as to maintain.

Asynchronous operations

A common reason for an API to be asynchronous is that it models access to external services. These services often end up accessing some hardware. As most hardware have much longer latencies than the main CPU, it does make sense to use the CPU for other work while awaiting the result. External services are also often unreliable by nature, which means that they might fail. So it is common that many asynchronous APIs need to be able to report those failures.

A common asynchronous operation is a network request. On Apple platforms, we typically use URLSession's task-based APIs such as:

extension URLSession {
func dataTask(with url: URL, completionHandler: @escaping (Data?, URLResponse?, Error?) -> Void) -> URLSessionDataTask
}

The signature of the method hints that it is asynchronous as the expected Data result is not immediately returned. Instead, a completionHandler closure is passed to the API. This closure will be called as soon as a result is available or when the operation failed for some reason.

So let us focus in on the completionHandler. We will ignore the less used URLResponse to simplify the API a bit. What does it tell us?

completionHandler: (Data?, Error?) -> Void

We would expect to get some data back or if something went wrong an error. But as a consumer of this API we actually have to handle four cases:

switch (data, error) {
case (data?, nil): // Received data
case (nil, error?): // Received error
case (data?, error?): // Received both data and an error?
case (nil, nil): // Received neither data nor an error?
}

By reading the documentation for dataTask() we can see that the last two cases should never occur, and it should be safe to ignore them. But it would be great if we did not have to rely on documentation to express this.

Introducing the Result enum

enum Result {
case success(Data)
case failure(Error)
}

Further, by using Swift generics, Result can be generalized to work with any success type Value instead of just Data:

enum Result<Value> {
case success(Value)
case failure(Error)
}

With Result<T> at our disposal, we can now simplify our network API. When at it, we will also make the API more focused on the data result than the data task itself:

func data(at url: URL, completion: @escaping (Result<Data>) -> Void) {
dataTask(with: url) { data, _, error in
if let error = error {
completion(.failure(error))
} else {
completion(.success(data!))
}
}.resume()
}

Transforming results

For this article, we will use the framework Lift to work with JSON. Lift wraps JSON using the Jar container type. It comes with an initializer accepting raw JSON data:

extension Jar {
init(json data: Data) throws
}

So how do we go about transforming a Result<Data> into a Result<Jar>. Firstly, we need a way to extract the success value:

extension Result {
func getValue() throws -> Value {
switch self {
case .success(let value): return value
case .failure(let error): throw error
}
}
}

And given a value or a thrown error, we need a way to put them back into a Result<T>. This is like running getValue() in reverse:

extension Result {
init(getValue: () throws -> Value) {
do {
self = .success(try getValue())
} catch {
self = .failure(error)
}
}
}

Combining the two, allow us to first extract the data from a Result<Data>, then transform it to JSON using Jar(json:), and then finally construct a Result<Jar> with the resulting JSON:

let jarResult = Result<Jar> {
try Jar(json: dataResult.getValue())
}

This is called a map(), the operation of transforming a value inside a box into another one:

extension Result {
func map<O>(_ transform: (Value) throws -> O) -> Result<O> {
return Result<O> { try transform(getValue()) }
}
}

The implementation of map() is a great example of how we can compose new methods from more general ones. The map() function is a very useful function that you will find in other box like types as well, such as Swift's optional and container types. Using map(), our new JSON method now becomes:

func json(at url: URL, completion: @escaping (Result<Jar>) -> Void) {
return data(at: url) { result in
completion(result.map { try Jar(json: $0) })
}
}

Here we can see how we chained two operations, one asynchronous and one synchronous, both of which might fail. Thanks to Result<T> and map(), we could focus on the "happy cases" and avoid writing explicit error handling.

Introducing the Future type

struct Future<Value> {
let onResult: (@escaping (Result<Value>) -> Void) -> Void
}

This would let us rewrite data(at:completion:) as:

func data(at url: URL) -> Future<Data> {
return Future { completion in
self.data(at: url, completion: completion)
}
}

To listen on a future’s result, we would just call onResult() and pass a completion handler:

let dataAtURL = session.data(at: url)
...
dataAtURL.onResult { result in
...
}

This is really powerful. We have abstracted the idea of an asynchronous operation and encapsulated it into a type. Instances of this type could be passed around, this without any need to know the origin of the operation nor having to drag along any of its dependencies.

Having a type representing asynchronous operations also means we can extend it with useful functionality. One of those is map(), as Future<T> similar to Result<T> also acts as a box wrapping a value:

func map<O>(_ transform: @escaping (Value) throws -> O) -> Future<O> {
return Future<O> { completion in
self.onResult { result in
completion(result.map(transform))
}
}
}

Our JSON method can now be simplified to:

func json(at url: URL) -> Future<Jar> {
return data(at: url).map { try Jar(json: $0) }
}

It is now so trivial that it might not even be worth adding a convenience method for it.

Improving the implementation

A related problem is that nothing stops us from calling onResult() more than once. This would currently result in a new asynchronous operation being kicked off for each call to onResult(). This problem did not exist when we passed a completion handler, where at most one could be passed.

Given the above, it makes sense to model Future<T> to always execute its work, even though there potentially are no ones interested in the result. We might attempt to restrict the number of listeners to at most one. Potentially, we could assert or complete with an error if calling onResult() a second time. But in the context of the user, there might be no way of knowing if somebody is already listening on the future or not. And even if it was, it is not obvious what the user should do with that information. So it also makes sense to allow multiple listeners, but still only execute the future once.

So let us update Future<T>'s internals by adding an initializer that immediately calls back the provided onResult closure. The previous onResult member now becomes a function instead where we add received completions to a completions array. As the future now has internal state, we also update it to be a class instead of a struct:

final class Future<Value> {
var completions: [(Result<Value>) -> Void] = []

init(onResult: @escaping (@escaping (Result<Value>) -> Void) -> Void) {
onResult { result in
self.completions.forEach { $0(result) }
}
}

func onResult(_ completion: @escaping (Result<Value>) -> Void) {
completions.append(completion)
}
}

This solves our two original issues but introduces a new one. What if the future has already been completed at the time onResult() is called? That would mean that its completion callback will never be called. This will be an issue even if we call onResult() immediately after creating the future, as the future could potentially complete even before leaving the initializer:

Future<Int> { c in c(.success(1)) }.onResult { result in
// Will never be called as `c` was called before `onResult()` was called.
}

The solution is to keep the result around in the case someone calls onResult() after the future has completed. And once we have a result we do not need to keep the completions around any longer. So the state of a future is now either pending with an array of completions or completed with a result:

final class Future<Value> {
enum State {
case completed(Result<Value>)
case pending([(Result<Value>) -> Void])
}

var state: State
}

The initializer now becomes:

init(onResult: @escaping (@escaping (Result<Value>) -> Void) -> Void) {
state = .pending([])
onResult { result in
guard case .pending(let completions) = self.state else { return }
completions.forEach { $0(result) }
self.state = .completed(result)
}
}

And onResult() is updated to immediately call the completion back if the future has already been completed:

func onResult(completion: @escaping (Result<Value>) -> Void) {
switch state {
case .completed(let result):
completion(result)
case .pending(var completions):
completions.append(completion)
state = .pending(completions)
}
}

Chaining futures

To better see how map() and flatMap() are basically the same for Optional, Array, Result and Future, we could look at the signatures of their transformation functions:

| type      | map()    | flatMap()        |
|-----------|----------|------------------|
| A? | (A) -> B | (A) -> B? |
| [A] | (A) -> B | (A) -> [B] |
| Result<A> | (A) -> B | (A) -> Result<B> |
| Future<A> | (A) -> B | (A) -> Future<B> |

Given the function signature of flatMap(), even with the addition of letting the transform to throw errors, the implementation for Future<T> is still quite straight-forward:

func flatMap<O>(_ transform: @escaping (Value) throws -> Future<O>) -> Future<O> {
return Future<O> { completion in
self.onResult { result in
do {
try transform(result.getValue()).onResult(completion)
} catch {
completion(.failure(error))
}
}
}
}

Focus on the “happy flow”

Where onResult() requires us to handle both failures and successes, it is often more convenient to focus on one or the other. This can easily be improved by adding an onValue() helper:

func onValue(_ callback: @escaping (Value) throws -> Void) -> Future {
return map { value in
try callback(value)
return value
}
}

Similarly, we can add an onError() helper.

Bringing it all together we can now build quite complex compositions, such as fetching a user’s friends:

json(at: userURL).flatMap { jar in
let user: User = try jar^
return json(at: user.friendsURL)
}.map { jar in
let friends: [Friend] = try jar^
return friends
}.onValue { friends in
// Everything was successful
}.onError { error in
// Something failed
}

If you wonder what the try jar^ is all about, it is just how Lift converts the JSON contained inside a Jar into model values.

In this example, you can see how futures makes it easy to chain several transformations after each other. If you would try to do this with the original APIs, using completion callbacks and without Result<T>, the code would soon be really messy and hard to read and maintain.

In the above example, it might not be obvious at first how much could fail. Both requests could fail, transforming data to JSON could fail, and constructing our model values could fail as well. If all goes well, we end up in onValue()'s callback. If something fails, we will end up in onError()'s callback.

Summary

We discovered that futures carry a state of being completed or not. This state resulted in the implementation being a bit more tricky then perhaps first anticipated. Finally, we introduced the map() and flatMap() transforms and showed how we could chain futures together and mainly focus on the "happy flow".

If you want to learn more about futures and try them out yourself, download the open sourced framework Flow. Here, you can also see how a production implementation might look like. There is also a playground available, where you can see the code introduced in this article all come together.

In an upcoming article, we will see how we can make futures even more powerful and convenient to work with.

iZettle Engineering

We build tools to help business grow — this is how we do it.

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch

Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore

Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store