Dynamic Encodable with type erasure

Swift 4 brought us new Codable APIs aiming to simplify serializing and deserializing data models into various formats, most popular of which is json. Swift 4 compiler has special support for these APIs in form of code autogeneration. This post is about a small yet annoying problem that you might face while adopting Encodable protocol for dynamic structure models.

The problem

Suppose we want to serialize some data, which can be structured in a dynamic way. Let me explain what I mean by that. We will be serializing model Packet which contains some statically defined properties, like date, as well as one dynamically defined property called payload. The reason why I call it dynamic is that several different types should be able to fit into that payload. The only common thing between those types is that they should be Encodable as well.

struct SomeDataModel: Encodable {}
struct SomeOtherDataModel: Encodable {}
struct Packet: Encodable {
  var date: Date
  var payload: Encodable
}

If you try to put that into playground it won’t compile

cannot automatically synthesize ‘Encodable’ because ‘Encodable’ does not conform to ‘Encodable’

That reads odd, doesn’t it? Well, let’s try to figure out what’s wrong. Encodable requires to have only one method

public func encode(to encoder: Encoder) throws

so maybe we can implement that method ourselves? For that we will also have to define CodingKeys enum. Let’s see

struct Packet: Encodable {
  var date: Date
  var payload: Encodable
  enum CodingKeys: String, CodingKey {
    case date, payload
  }
  public func encode(to encoder: Encoder) throws {
    var container = encoder.container(keyedBy: CodingKeys.self)
    try container.encode(date, forKey: .date)
    try container.encode(payload, forKey: .payload)
  }
}

That doesn’t compile either:

error: ambiguous reference to member ‘encode(_:forKey:)’

and it only complains about the line where we try to encode payload. And the reason why it complains is that encode for custom types is actually generic method parameterized with that custom type. It’s static, meaning that it requires to know exact type of what we’re trying to encode before invoking encode(forKey:). Since we don’t have type at this point, we only know that this type conforms to Encodable. To me — that should be enough, after all, that’s all encode(forKey:) cares about. But it’s just the way encodable APIs were designed.

There are ways to make Packet conform to Encodable with dynamic payload. Most straightforward (and I would argue — the intended) way would be to make Packet generic:

struct Packet<T: Encodable>: Encodable {
  var date: Date
  var payload: T
}

and it works! Even without explicit Encodable implementation — compiler can autogenerate all that boilerplate for us.

There is downside to this approach though. Since Packet is now generic — you’d have to make every method taking Packet as a parameter to be generic over payload type. That may require a lot of changes in existing code, not to mention, also leaks of implementation details. And if you happen to have method returning Packet — you’re going to have even more headache.

Good old Type Erasure

There is a way to simplify Packet using the Type Erasure pattern. Consider we have helping internal type

struct AnyEncodable: Encodable {
  var _encodeFunc: (Encoder) throws -> Void
  
  init(_ encodable: Encodable) {
    func _encode(to encoder: Encoder) throws {  
      try encodable.encode(to: encoder)
    }
    self._encodeFunc = _encode
  }
  func encode(to encoder: Encoder) throws {
    try _encodeFunc(encoder)
  }
}

As you can see, this is a classic example of type erasure. It can turn any dynamic type conforming to Encodable into statically known type (since AnyEncodable is a struct) that also conforms to Encodable. We can use AnyEncodable as substitute for Encodable:

struct Packet: Encodable {
  var date: Date
  var payload: AnyEncodable
}

Compiler is perfectly fine with such definition: AnyEncodable is a statically known type (it’s a struct after all) and can happily autogenerate all the code required for Encodable for us. With small handy addition to our Packet type:

init(date: Date, payload: Encodable) {
  self.date = date
  self.payload = AnyEncodable(payload)
}

we can use Packet in the same way we wanted to use it from the beginning when trying to annotate payload as Encodable.

As a conclusion I wanted to emphasize again how type erasure isn’t the only way to approach the problem but how it allows us to have minimum integration changes and keep APIs simple.