Extensibility in Elixir Using Protocols

Published in

everydayhero engineering

4 min readJun 26, 2017

The last post took a little look into the concept of atoms in Elixir and how they are used. This post will cover one of options that Elixir provides for extensibility, protocols.

Protocols

To quote the guide:

Protocols are a mechanism to achieve polymorphism in Elixir. Dispatching on a protocol is available to any data type as long as it implements the protocol.

In other words you can write a function that behaves differently depending on the type of the first argument to it’s functions. Protocol implementations can be supplied for one of the built in supported aliases Atom, BitString, Float, Function, Integer, List, Map, PID, Port, Reference, Tuple, and Any; or a user defined struct. Lets make a Countable protocol, a contrived example that counts items:

The first step is to define a protocol:

iex> defprotocol Countable do
...>  def count_items(term)
...> end

Notice that we are defining a function in the Countable protocol without a body. Protocols only support definition headers, in fact, they redefine the def macro. Definition headers are a little more restrictive, for example a definition with a guard clause will not compile:

iex> defprotocol Countable do
...>   def count_items(term) when is_binary(term)
...> end
** (CompileError) iex:4: missing do keyword in def
    iex:4: (module)

Nor will a function that matches on a literal:

iex> defprotocol Countable do
...>   def count_items("")
...> end
** (CompileError) iex:4: can use only variables and \\ as arguments in definition header
    iex:4: (module)

A function header in a protocol must have at least one argument, this is what is used to dispatch to the correct protocol implementation:

iex> defprotocol Countable do
...>   def count_items()
...> end
** (ArgumentError) protocol functions expect at least one argument
    (elixir) expanding macro: Protocol.def/1
             iex:19: Countable (module)

Lets define protocol implementations that count the items in a List a Map and a user defined Order struct.

List implementation:

iex> defimpl Countable, for: List do
...>   def count_items(list), do: length(list)
...> end

Map implementati0n

iex> defimpl Countable, for: Map do
...>   def count_items(map), do: map_size(map)
...> end

Order implementation:

iex> defmodule Order do
...>   defstruct number: 0, items: []
...>
...>   defimpl Countable do
...>     def count_items(%Order{items: items}) do
...>       Countable.count_items(items)
...>     end
...>   end
...> end

Notice that the for argument can be omitted when defining an implementation inside a module that also defines a struct.

Although there are restrictions when defining the protocol when it comes to matching, there are no such restrictions when implementing the protocol, we can use pattern matching and guards freely. This also means multiple function bodies for the same arity can be defined.

It is now possible to invoke the protocol function with an argument that matches the types above:

iex> Countable.count_items([:one, :two])
2
iex> Countable.count_items(%{one: 1, two: 2})
2
iex> Countable.count_items(%Order{number: 1, items: [:apple]})
1
iex> Countable.count_items({:one, :two})
** (Protocol.UndefinedError) protocol Countable not implemented for {:one, :two}
    iex:1: Countable.impl_for!/1
    iex:2: Countable.count_items/1

Interestingly each implementation will become a sub-module of the protocol module, which means module attributes are supported in a Protocol implementation. We can prove this by looking at some module info:

iex> Countable.Order.__info__(:functions)
[__impl__: 1, count_items: 1]
iex> Countable.impl_for(%Order{})
Countable.Order

The @for module attribute

Protocol implementations have access to the @for module attribute that is the alias of the current target, and the @protocol module attribute that is the alias of the protocol being implemented. This can come in handy when implementing a protocol for multiple aliases:

iex> defprotocol ToList do
...>   def to_list(term)
...> end
iex> defimpl ToList, for: [Map, Tuple] do
...>   def to_list(term) do
...>     @for.to_list(term)
...>   end
...> end
iex> ToList.to_list({:ok, "Hurray"})
[:ok, "Hurray"]

Both the Map and Tuple modules define a to_list function, the implementation above will invoke the correct function depending one the value of the @for attribute.

Fallback implementation with Any

There is a special alias Any that can be used as a fallback/default implementation of a protocol, it is necessary to opt in to this when declaring the protocol however:

iex> defprotocol Countable do
...>   @fallback_to_any true
...>   def count_items(term)
...> end
iex> defimpl Countable, for: Any do
...>   def count_items(_), do: :unknown
...> end
iex> Countable.count_items({:one, :two})
:unknown

Note: It is possible to derive the Any protocol implementation if it is necessary to whitelist the modules that can fallback.

Built-in protocols

There are a number of built-in protocols that may prove useful to implement in your own code. At the time of writing these are Collectable, Enumerable, Inspect, List.Chars and String.Chars. I think it is telling that all of the built-in protocols only have one function with the exception of Enumerable which only has three. Where possible it is best to keep the "surface area" of a protocol small.

It is worth mentioning the Access module, that was moved from a protocol to a behaviour.

When to use them

The great advantage of protocols is that the implementation of a protocol can be outside of the library / application that defines it. This is particularly handy for library authors to create extension points for the consumers own types. Some examples from packages out there might help illustrate this:

Plug.Exception: A protocol that extends exceptions to be status-code aware
Phoenix.Param: A protocol that converts data structures into URL parameters
Scrivener.Paginater: A protocol that paginates types
Poison.Encoder: A protocol that JSON encodes types.
Joken.Claims: Handles turning data into claims.

Performance

There is a potential performance hit when using protocols; from the guide:

Because a protocol can dispatch to any data type, the protocol must check on every call if an implementation for the given type exists. This may be expensive.

This is why, as part of compilation, protocol consolidation is on by default. Protocol consolidation is the process of optimising dispatches by looking at all of the implementations in a project / application. To check if the option is on in your project you can run:

iex> Mix.Project.config[:consolidate_protocols]
true

Wrapping Up

Protocols provide a great way to add extension points to your code, particularly for library authors. It is important not to go protocol crazy though, you shouldn’t use protocols when some functions on a module with pattern matching will do. The fact that there are only six built-in to the language is a good indicator of this.