API Design — Deriving Signal

In this article, we will explore three abstractions that are useful building blocks in asynchronous event handling. As a use case, we will build an API for registering callbacks to get notified of events. Working with callbacks might sound trivial at first. After all, it is just the passing of a closure to be called back on event updates. But as it turns out, we will run into several challenges to come up with an API design that is both robust and flexible.

It all starts out with trying to find a solution how to deregister our callbacks, a lifetime management problem which leads us to the Disposable protocol. After that, we address the repetitive work of callbacks book-keeping, leading us to the reusable Callbacker type. Finally, we investigate how to make our callbacks even more composable and stand-alone by bringing it all together into a basic implementation of a Signal type. This Signal type is a core concept found in many reactive frameworks, such as Flow

Callback registration

A recurring problem in API design is to be able to notify clients of events. If the language supports closures, a natural design is to let the client register a callback that will be called when something happens. It typically starts out like:

func addObserver(onEvent: @escaping (Event) -> Void)

And at the call-site we will have:

addObserver(onEvent: { event in
   ...
})

One major omission in this API is that there is no way to deregister your interest in event callbacks. It is unlikely you would always want to receive events for the lifetime of the application. Removing an observer would also free up the resources captured by the callback closure. A first attempt to add support might look like:

func removeObserver()

It should be obvious that this design will not scale beyond one observer. What should removeObserver do if we have registered more than one callback? What we need is a way to identify one callback from another. One approach would be to add some kind of key:

func addObserver(forKey: String, onEvent: @escaping (Event) -> Void)
func removeObserver(forKey: String)

This is better. But to make the user of the API responsible for creating the keys opens up for several problems. As the keys have to be unique and we might use the API from different places, we have opened up for involuntarily reusing of the same key for different callbacks. This could result in callbacks being replaced by new registrations or existing callbacks being removed when deregistering others. So it is better to move the complexity of key generation to the API:

func addObserver(onEvent: @escaping (Event) -> Void) -> String
func removeObserver(forKey: String)

Using String as our key type comes with one disadvantage though. New strings can be constructed freely. This allows removeObserver to be accidentally called with an arbitrarily key. An improvement would be to add a custom key type with a private initializer. This would restrict the creation of keys to the implementation of the API:

struct ObserverKey { private init() { ... } }
func addObserver(onEvent: @escaping (Event) -> Void) -> ObserverKey
func removeObserver(forKey: ObserverKey)

We could still call removeObserver more than once for the same key, but that could potentially be preconditioned to cause a trap (at least in debug builds). This API design is very similar to what Foundation is using for notifications:

func addObserver(forName: /*...*/, using block: @escaping (Notification) -> Void) -> NSObjectProtocol
func removeObserver(_ observer: Any)

As can be seen, this API has a similar problem with its keys as some of our earlier attempts. It is even worse, as the user could pass a key of any value and type to removeObserver. This is a problem we nicely avoided when introducing our custom key type.

Another advantage of introducing custom key types is that we could have different types of keys for different but similar APIs. This would stop us from accidentally mixing up keys from different APIs. A disadvantage is that it requires us to define a new custom key type every time we add a similar API. It could also be an administrative burden for the consumer of these APIs to have to keep track of the different kind of keys.

It would be great if the API could abstract away what underlying key is being used and help us calling removeObserver correctly. One, at first perhaps not obvious way to solve this, is to let addObserver return something that can be called when we want to deregister. The most straightforward would be to return a function:

func onEvent(callback: @escaping (Event) -> Void) -> () -> Void

That can be implemented using our existing observer APIs as:

func onEvent(callback: @escaping (Event) -> Void) -> () -> Void {
  let key = addObserver(onEvent: callback)
  return { removeObserver(forKey: key) }
}

And voilà, in one sweep we got rid of the key and the removeObserver function all at once. Another great benefit of this change is that all deregister functions are now of the same type and we do not necessarily have to keep track of separate lists for different observer APIs. The method name can be simplified to onEvent, and by introducing a type alias, intention and readability can be further improved:

typealias Disposable = () -> Void
func onEvent(callback: @escaping (Event) -> Void) -> Disposable

Applying this to Foundation’s notification would look something like:

extension NotificationCenter {
  func onNotifications(named: /* ... */, callback: @escaping (Notification) -> Void) -> Disposable {
    let observer = addObserver(forName: /* ... */, using: callback)
    return { self.removeObserver(observer) }
  }
}

Disposable

Even though Disposable describes everything we need, there are some limits of using functions directly such as they cannot be extended. Hence it makes a lot of sense to replace our function type alias with a protocol:

protocol Disposable {
  func dispose()
}

And the most basic implementation of Disposable would be the Disposer type:

final class Disposer: Disposable {
  private var disposer: (() -> Void)?

  init(_ disposer: @escaping () -> Void) {
    self.disposer = disposer
  }

  func dispose() {
    disposer?()
    disposer = nil // Stop from being called more than once
  }

  deinit { dispose() }
}

And now when all our different registration APIs are all returning the same Disposable type, we can add a convenience type for collecting them:

final class DisposeBag: Disposable {
  fileprivate var disposables: [Disposable] = []

  func dispose() {
    disposables.forEach { $0.dispose() }
    disposables = []
  }

  deinit { dispose() }
}

And by adding an operation +=:

func +=(bag: DisposeBag, disposable: Disposable) {
  bag.disposables.append(disposable)
}

We can now collect disposables and return a bag as another Disposable:

func setupObservers() -> Disposable {
  let bag = DisposeBag()
  bag += onSomething { .. }
  bag += onSomethingElse { .. }
  return bag
}

Implementing callbacks

The introduction of Disposable made our API more robust. But how would an implementation of the observe method look like, supporting multiple listeners? Looking at e.g. NotificationManager, one could imagine that they internally would use a dictionary of some sort to keep track of the callbacks. If we apply this to our example, we would get something like:

private typealias Key = ...
private var callbacks: [Key: (Event) -> Void] = [:]
private func generateKey() -> Key { ... }

func onEvent(callback: @escaping (Event) -> Void) -> Disposable {
  let key = generateKey()
  callbacks[key] = callback
  return Disposer {
    callbacks[key] = nil
  }
}

private func notify(event: Event) {
  callbacks.forEach { $1(event) }
}

At a closer look, not much of this code seems unique for this specific API. If we have to repeat this code for several observer APIs we risk introducing subtle differences and bugs. This is especially true if taking thread safety and efficient key generation into consideration. It makes sense to introduce a Callbacker helper:

final class Callbacker<Value> {
  private var callbacks: [Key : (Value) -> Void] = [:]

  func addCallback(_ callback: @escaping (Value) -> Void) -> Disposable {
    let key = generateKey()
    callbacks[key] = callback
    return Disposer {
      self.callbacks[key] = nil
    }
  }

  public func callAll(with value: Value) {
    callbacks.forEach { $1(value) }
  }
}

Using Callbacker would reduce our onEvent implementation to:

private let callbacker = Callbacker<Event>()

func onEvent(callback: @escaping (Event) -> Void) -> Disposable {
  return callbacker.addCallback(callback)
}

private func notify(event: Event) {
  callbacker.callAll(with: event)
}

Even though the onEvent API and its implementation have improved a lot from what we started out with, there are still problems with it. For one, it is hard to pass the onEvent functionality around. It is likely onEvent would be a member of some type and perhaps we could pass an instance of that type around instead. But that would introduce unnecessary dependencies on this type. It would be great if the code that is interested in observing changes does not need to have knowledge about who provides the API and how it is implemented.

One attempt to improve this would be to pass around functions:

typealias Signal<T> = (_ callback: @escaping (T) -> Void) -> Disposable

let onEvent: Signal<Event> = { callback in
  return callbacker.addCallback(callback)
}

bag += onEvent { event in ... }

But perhaps the consumer does not even need to know about the details of the original event but rather a transformation of it. This could be a boolean value expressing the enabled state of a button, or a string value to be displayed in a label. To allow this we need to add wrappers such as:

let shouldEnableButton: Signal<Bool> = { callback in
  onEvent { event in
    callback(event.shouldEnableButton)
  }
}

As those transforms might become common, we could improve this further by adding a map function:

func map<T, O>(_ observer: Signal<T>, transform: @escaping (T) -> O) -> Signal<O> {
  return { callback in
    observer { value in
      callback(transform(value))
    }
  }
}

The shouldEnableButton can now be rewritten to:

let shouldEnableButton = map(onEvent) { $0.shouldEnableButton }

But using free functions such as map gives us flashbacks to days before Swift 2 and protocol extensions. The more Swifty way to express this would be:

let shouldEnableButton = onEvent.map { $0.shouldEnableButton }

Yet again we have run into the limit of working directly with functions or type aliases of thereof. The solution for Disposable was to move the function into a type or protocol. We will do the same here by replacing the Signal<T> type alias with a Signal<T> type holding the closure in an onValue property:

final class Signal<Value> {
  let onValue: (_ callback: @escaping (Value) -> Void) -> Disposable
}

Now we can pass the onEvent function around as a Signal instead:

let event: Signal<Event> = Signal { callback in
  return callbacker.addCallback(callback)
}

And as event is no longer a function we now call it indirectly via Signal's onValue:

bag += event.onValue { event in
  ...
}

We can now move the map function to an extension of Signal:

extension Signal {
  func map<T>(transform: @escaping (Value) -> T) -> Signal<T> {
    return Signal<T> { callback in
      self.onValue { event in
        callback(transform(event))
      }
    }
  }
}

And hence our shouldEnableButton can now be written in our preferred way:

let shouldEnableButton = event.map { $0.shouldEnableButton }

Of course, map is just one of many transformations you could use on signals, but that is for another article to talk about. Until then you could always have a sneak peek at our open source framework Flow.

Summary

We started out trying to design an API to register callbacks to receive event updates. By identifying different issues with our different API attempts, we discovered several powerful abstractions. The last of those, Signal, is the basis of many popular reactive frameworks.

To see how the concepts introduced in this article all come together, there is a playground for you to play with. You can also learn more about Disposable, Callbacker and Signal by downloading our open sourced framework Flow.

In the follow-up article Expanding on Signals we will further explore the utility of signals and how we can make them even more convenient to work with.