Another Look Through Optics

Optics are usually presented as data accessors for immutable data structures, and although this is very useful in functional programming it’s hardly a compelling story if you can opt for mutable structures. I present a different approach to optics as proxies for external APIs, which may help with the understanding of optics and their bigger menu of use-cases.

Lenses and other optics have been championed by functional programmers as one of the golden assets in our toolbox, but almost every time we try to convince others about the virtues of optics we face this look in their eyes like saying: “Are you serious? Are you telling me that it’s easier and better to write getters and setters instead of simply using my dot-syntax?”

// When using dot-syntax:
// reading values
val x = a0.b.c.d;
// writing values
a0.b.c.d = 1;
// When using automagically generated lenses in Haskell:makeLenses ''A
makeLenses ''B
makeLenses ''C
// reading values (getter)
let x = view (a . b. c. d) a0
// writing values (setter)
over (a . b. c. d) (cons 1) a0

Although these code snippets are missing a lot of context, they are enough to show how terse the imperative style appears to be compared to the immutable functional style. We know immutability is worth this extra complexity, but this conviction is the same as saying exercise is good for your health, you need to look ahead for benefits over time instead of focusing on immediate hassles as the time it takes or how exhausted you feel after going to the gym.

I have to admit that even for me the justification for using lenses as data accessors is kind of artificial and it feels like a high tax on immutability. Don’t get me wrong, I love lenses and immutability, it’s just that it’s hard to sell them without giving more context.

Besides the apparent complexity they present, there is another ingredient to their lack of popularity among the non functional world, the type signatures for lenses and other optics are kind of esoteric: Lenses have four type arguments: Lens s t a b! Really? And we explain: “well, yes, but let’s pretend for now that the type is more like Lens s a, the other two are just a generalization we can ignore at the beginning”. And then we hope nobody asks for the generalization use case.

Optics as Proxies

Forget for now about lenses and complex nested data types and let’s think in a more mundane example (although in its simplified version, just like when in high school the teacher says we may ignore friction when looking at Newton laws): Suppose you have a REST API that returns a pie recipe if you provide a pie name:

http://kitchen.library/pie?name=[some name for a delicious pie]

One way of thinking about your API is that it’s some kind of machine that has an entry for pie name requests and yields in return a response with the recipe.

Simplified type declarations for our system would be:

-- Nothing more than a tagged string
type ByNameQuery = PieName string
type Ingredient = {
 quantity: Double
 unit: String
 name: String
}
-- Another tagged string for the step description
type Step = Step String
type PieRecipe = {
 servings: Int
 ingredients: List Ingredient
 steps: List Step
}

I’m using PureScript’s modified syntax for code snippets with the hope they can be easily adapted either to functional languages like Haskell and Scala or to more imperative ones like TypeScript.

If our API was a function, it would have a type similar to:

recipesAPI :: ByNameQuery → PieRecipe

But even if it’s not a real function (at least in the sense of pure total functions) it shares some characteristics with functions, among them (the one we are interested in now) the type arguments: Functions can be seen as type constructors with two type arguments, one for its argument and one for its response: A → B ~ Function A B.

We can say that APIs are like that too: type API a b = API a b where a is the type of the request and b the type of the response. The new signature then would be:

recipesAPI :: API ByNameQuery PieRecipe

In this post we will represent an API with request type A and response type B drawing:

Isos or Adapters

Suppose an important client uses this API and they need to use a numeric identifier instead of the pie name because they restrict options to a catalog of pies they want to support and they don’t want their users getting into the creative mode of asking for the “X-AG23” pie (in a more real example you may need to convert from JSON to Protobuf or add OAuth protection or something like that). This client also wants the response formatted as Markdown instead of JSON and because they are a very important customer, you want to support their use case if you want to pay your bills.

As you may still need to support other clients using the old API, you decide to offer an alternative endpoint which uses a proxy that translates requests with numeric ids to the ones you already support with strings, and JSON to Markdown responses.

This proxy layer is just a pair of functions: toStringQuery… that performs the request transformation and toMarkdown… that does its part with responses.

We can regard this proxy as a function that takes an API (the original API) and yields a new API:

proxy :: API a b → API s t

In this drawing fis the function transforming the request objects and g is the one transforming the responses:

f :: S → A
g :: B → T

It ends up that there is already something that performs this kind of transformation and it’s part of the optics toolset: It’s called “iso” (or “adapter”, depending what library or article you refer to) which means we just found our first example of optics usage!

from :: S → A
to   :: B → T

Prisms

What about the case when the user provides a number that is not associated with any pie? We don’t want to call the underlying API and then fail and throw and exception and who knows what. We want instead to fail with a common error response that users will understand: a 404 response. And we want to yield this response as soon as we can even before reaching our recipes library service, that is, we want a kind of short circuit:

Which means that the response type now needs to consider this new possibility:

f :: S → A ⊕ T
g :: B → T

Wait a moment, what does it mean A⊕T?

The type A⊕T represents what is called a sum type and it means: a value a_t of type A⊕T is either of type A or of type T. In languages that support sum types like Typescript or Haskell you can write it like:

f :: S → A | T

Although it’s more commonly represented as an Either a b data structure because the class Either has a lot of prebuilt functionality (Haskell Lens package). To summarize, we’ll use the A ⊕ B notation to express Either A B

Going back to f declaration. What the Either presence conveys is that if f s is an A, then we send it to our original API, but if it is a T then we return it immediately to the client (we short circuit).

The signatures of f and g match those of another optic: the prism.

As a data accessor a prism works as a validator for values, which is pretty much what we are doing here:

match :: S → A ⊕ T
build :: B → T

Lenses

Time to pretend more use-cases: what if a client wants to allow its users to personalize recipes based on the number of servings, which they plan to enable through an extra URL parameter:

http://kitchen.library/pie
        ?name=[some name for a delicious pie]
        &servings=[how big is your party?]

The idea here is that you will query your original API using the pie name and once you get the recipe, you modify the list of ingredients based on how many servings were originally set in the recipe and how many servings the user wants the recipe for. This means that for building the response, you need both the request of the user (new servings) and the response from your original API (servings and list of ingredients in the recipe). Our drawing will be now:

Giving us the new type signatures:

f :: S → A
g :: B ⊗ S → T

Here we go again… what does it mean B⊗S?

The type B⊗S represents a product type of types B and S which translates as a pair (A, B) or any other isomorphic value, for example, the Typescript type:

class Pair<A, B> { a : A, b : B }

Which means that gcould also be written as:

g :: (B, S) → T

Do you recognize the signatures? You are right, this is a Lens s t a b!

view   :: S → A      -- also called get
update :: B ⊗ S → T  --also called set

Profunctors

Functional programmers and mathematicians have a name for the kind of objects like our APIs that are parametrized by two types and that can be transformed into a new thing of the same shape although with different type parameters:

The technical name for these things is “profunctor”.

You can think about profunctors as a generalization of our regular functions (functions are, after all, the canonical example for profunctors), and although they don’t need to behave exactly as functions they do need to follow certain rules when being transformed by the translation functions (this rule is said to governs dimap):

Suppose you have a type constructor with two type parameters, just as our API a b, and four functions:

f  :: S → A
g  :: B → T
f’ :: M → S
g’ :: T → N

Then if we dimap over f and g and then over f’ and g’ is the same as if we dimap over f ◃ f’ and g ▹ g’ (the ◃ symbol represents regular function composition while g ▹ g’ translates as g’ . g).

Optics as Profunctor Transformations

The last thing we can observe if we look into the drawings is the common patterns between the three optics we described: Iso, Prism and Lens. They all transform one profunctor (our API) into another profunctor (the new API) and they are regular functions between profunctors, which means they can be composed as well.

In API terms, it means that if you have a proxy that adds the “servings” feature to the API and other that converts the output to Markdown, if you compose them both you get a Markdown output personalized on servings! Of course you can also compose these with proxies that add OAuth protection, spike arrest quotas, etc.

There are some caveats though, not all profunctors can be used with Lens or Prism optics, they need to follow certain rules in order to be compatible with each optic but that’s another story and shall be told another time.

Final Thoughts

Looking at the proxy use case for optics provides a scenario where the full type signature optic s t a b comes naturally and where optics can deliver powerful abstractions adding to the already rich menu of use cases of accessors for nested data structures. I hope that this presentation will help others to see how useful and interesting Lenses and optics are.

References:

The most complete and mature reference for optics IMO is the webpage of the Haskell Lens library by Edward Kmett. There is also a video where E. Kmett presents these concepts: Lenses, Folds, and Traversals.

[2018], Guillaume Boisseau, Jeremy Gibbons, What You Needa Know about Yoneda. An elegant proof using the Yoneda lemma that traditional optics definitions are equivalent to their profunctor representation. They also include additional material to understand the importance and applications of the famous lemma.

[2016] Phil Freeman, Fun with Profunctors. A presentation by the creator of PureScript about the optics library implementation in PureScript.

[2017] Bartosz Milewski, Profunctor Optics: The Categorical View. Also presents a proof of equivalence between classical and profunctorial representations of optics, providing also a generalization of special optics dependent on Ends (Tambara optics) and a general pattern for optics generation.

[2013] Simon Peyton Jones, Lenses: Compositional Data Access and Manipulation. It’s always fun and educational to watch S. Peyton Jones explain whichever subject he picks. In this video he gives a presentation of optics as data accessors and shows that data translation is just another example of these patterns.

[2017], Matthew Pickering, Jeremy Gibbons, and Nicolas Wu, Profunctor Optics. A great reading to understand more about the profunctor representation of optics.

[2007] Kent Beck, Implementation Patterns. For those looking for immutability advocates among object oriented patterns.