Handling Errors with Profunctor Optics

Using Optical Proxies to Short-circuit and Retry APIs

Profunctor optics remove the need for configuration objects for error handling and improve testing and maintainability; these proxies are regular functions written in the same language as the rest of the code and can be composed as any other function.

One familiar pattern when dealing with errors from an external API is to retry the operation. Some client libraries offer a way to configure the number of attempts or under which error conditions they will retry. This works most of the times, but as applications grow and corner cases become frequent, developers need granular configuration for clients. In extreme situations, configuration parameters may develop into a DSL on their own and their complexity into a liability. Can we leave the retry operation to the client but keep the details on when to do it in a proxy in front of it?

In a previous post, I explored API proxies as profunctor optics. Continuing on that line of thinking, in this text I want to review error handling through optics.

Errors are regular values

What is an error? We can argue at this point as much as you want, but for now I will consider an error any unexpected value over the happy paths of a program.

I believe a key to understand errors is to stop treating them as if they were something different from regular values. Errors have types and hence inhabit sets in our programming language. If a function may throw an error, the function signature should mention it in the return type. If you happen to be in a world with sum types, or its more powerful equivalent: Either, adding errors to type signatures is probably a common practice for you. For Java developers this is also not a novelty, they are used to including the exception signature in their functions — although not in the return type but in a special syntax, which together with catch statements can be used similar to a pattern-match mechanism in the output.

Errors by position

Classifying errors can be useful, not all errors are created the same, and they do not represent the same idea in a computation. There are multiple ways to classify errors, for this post I’ll look at their position.

In pure functions there are two type positions: one is some kind of input and the other one some kind of output. Strictly speaking functions are a special case of relationships, they don’t produce outputs from inputs, they just match them. Although we can relax our language and pretend they are like small machines that create output values from input values (as they often do in a computer context).

A distinctive characteristic between these two positions is how you transform them. To convert a function f :: a → b into a function f’ :: c → d you need two transformations:

c2a :: c → a
b2d :: b → d

Drawing only the arrows: c → a → b → c, we can see that input values are converted in a backwards fashion while output values are converted in a forward way. We say that in f, a is in a contravariant (input like) position and b is in a covariant (output like) position.

Back to errors, some errors occur when the input fails a validation (for example, if you ask for the 20th element in a 10 elements array you get an index out-of-bounds error). We can see these errors as a type mismatch: I were expecting an integer less than 10 and bigger or equal to zero but got 20. Remember that types are like sets (in the sense of set theory) and that the set {a | a is an Integer and a >= 0 and a < 10} is a legit set and in theory should also be a legit type. Unfortunately a type signature with such restrictions is difficult to write in common programming languages (unless you are lucky to use Agda, Idris, Coq or a similar one).

One way to solve this limitation of type systems is adding assertions executed at run time. If the input fails the validation we do an early return (throw an error) and avoid the rest of the computation. Because the error happens on an input validation, we say its contravariant and it can be short circuited.

If the error is not an input type mismatch, it is a computation or environment error: the disk is full, the system ran out of memory, there is a bug in the code, etc. In such cases we say the error is covariant or in the output position. When a covariant error happens, users have no way to solve the situation, either they retry the operation or they just go away; it was not their fault and nothing they can do about it. We can build a middle layer that retries the computation, fingers crossed hoping that the world would change in between, and we’ll have some extra memory to finish the job.

It’s worth noticing, as we are talking about API proxies, that there is a domain where this classification is the status quo pattern. HTTP Response codes dedicate the segment 400 -> 499 to client errors (contravariant/input) and the segment 500 -> 599 to server errors (covariant/output).

Classifying errors by their position gives us not only a way to talk about them but also a way to deal with them. How do we do that with proxies (AKA optics)?

Either is an Error or not

In functional programming errors are typically encoded as sum (or Either) values working as carriers of either a success or an error. In optics this translates to prisms (for contravariant errors) and coprisms (for covariant errors).

Imagine a service that performs arithmetic addition on integers. The backend of our service is a squirrel which sometimes goes away to get lunch leaving an auto response machine in charge of the calculator service. When the squirrel is in office, the service is pretty much what we would expect:

λ> calc 3 4
7

When the squirrel is not there we get:

λ> calc 3 4
("I’m not in office, try again later", (3, 4))
-- We are very lucky and the answer machine
-- gives us back the arguments we provided.

In this case, tuples values of the form String ⊗ (Int ⊗ Int) represent an error on the service. Remember that the type a ⊕ b translates to Either a b and that the type a ⊗ b translates to (a, b). Using a sum type we can have an idea of the service type:

calc :: Int → Int → (String ⊗ (Int ⊗ Int)) ⊕ Int

To make our life a bit easier, let’s add some type aliases:

type Error = String ⊗ (Int ⊗ Int)
type SquirrelRequest = Int ⊗ Int
type SquirrelResponse = Error ⊕ Int
-- making the calc signature a bit more expressive:
calc :: SquirrelRequest → SquirrelResponse

There is another case to consider: when someone asks for a sum where one or two of the arguments is greater than 1000, the squirrel eventually fails with another message:

λ> calc 3000 4
"Sorry, ran out of paper, try again with smaller numbers"

There is this new customer that wants to use the squirrel service. We want to isolate them from the curious detail that sometimes the squirrel is not available. We also don’t want to bother the rodent trying to sum numbers greater than 1000 just to discover that the paper was not enough and having to fail.

Short circuit

A proxy that short circuits when each of the numbers is greater than 1000 can be created using a prism (we also use this opportunity to reword the message):

type ValidationResponse = Int
type ValidationRequest  = Int
buildValidationResp :: SquirrelResponse → ValidationResponse
buildValidationResp n = n -- ...yes, this is the identity function,
                          -- but let me keep the verbose version
                          -- so the example is clearer.
matchValidationReq :: ValidationRequest → String ⊕ SquirrelRequest
matchValidationReq (n, m)
   | n > 1000 || m > 1000 =
       Left “Invalid arguments, use numbers less or equal to 1000”
   | otherwise = Right (n, m)
validationProxy = prism buildValidationResp matchValidationReq

We can produce a new validated API:

validatedCalc = validationProxy calc

Great! This new API handles the validation and isolates the squirrel from difficult questions. As you can see from the code, we don’t expect the prism function to know how to distinguish offending requests, we provide an explicit way to know what is a valid request and what is not in the match function. What prism does know is what to do if there is an error: return the left value.

Retry

For the next trick we’ll need an optic that is not present in any of the Optics libraries I know and that I’ll call “coprism” until Edward Kmett comes with a better name for it (following the tradition started by O’Connor [2015]).

The coprism optic is similar to a prism and its drawing is like the one for prism but rotated 180 degrees:

type RetryResponse = Int
type RetryRequest = Int
buildRetryReq :: RetryRquest → SquirrelRequest
buildRetryReq n = n
matchRetryRes :: SquirrelResponse → SquirrelRequest ⊕ RetryResponse
matchRetryRes Right n = Right n
matchRetryRes Left (s, (n, m)) = Right (n, m)
   -- we may want to add some delay to let the squirrel finish
   -- some nuts.
retryProxy = coprism buildRetryReq matchRetryRes

And the new API with retry capabilities:

retryCalc = retryProxy calc

As before, we use Either to tell about errors which keeps the responsibility of knowing what is an error and what not is in the function match and not in coprism.

Note: We were very lucky that the squirrel answer machine returned the original values in its response. We could use other optics in other cases, but looking into them would make this post longer.

Composition

A neat thing about optics as profunctor transformations is that they are regular functions and because of that they compose as functions do.

What is the signature of an optic?

type Optic p s t a b = forall p. Profunctor p ⇒ p a b → p s t

Both, prism and coprism functions build optics:

type Prism s t a b = forall p. Choice p ⇒ Optic p s t a b
prism :: forall s t a b. (b → t) → (s → t ⊕ a) → Prism s t a b
type Coprism s t a b = forall p. Cochoice p ⇒ Optic p s t a b
coprism :: forall s t a b. (s → a) → (b → a ⊕ t) → Coprism s t a b

Hence we can compose them:

newCalc = (validationProxy . retryProxy) calc

And get an API with validation and retry mechanisms in place.

Because they are functions, testing them in isolation is easy. Besides being small, the type of objects they take and return includes regular functions (in other words, functions are profunctors and can be transformed with optics as well); we can test proxies with synchronous dummy functions and then use them in real life with more complicated profunctors (as long as they respect the restrictions of the optics we use).

And this pretty much the end of this story.

The next part contains the details about coprisms, my brand new optic for retry handling, and is not really needed to understand the main ideas on this post. You are welcome to read it but I’ll not be enormously offended if you don’t.

Coprism

Before we jump into coprisms let’s do a quick review on the ingredients we’ll need: profunctors which are the things we are transforming; optics, which is the name for functions that transform profunctors; choice profunctors and prisms, the profunctor type with its optic that we’ll use as model; and cochoice and coprisms, the profunctor type and the optic we need for retry operations.

What is a profunctor again?

A profunctor can be seen as a generalization of a function, it’s not quite a function but it’s close to it, and sometimes is the closest we’ll have to a function.

For what it’s relevant to us, if p is a profunctor it has two type parameters p a b; p should be co-functorial (also knowns as contravariant) in the first parameter and functorial in the second one. There is a function dimap, similar to fmap or map for regular functors which takes two functions c → a and b → d that converts a p a b into a p c d (as they are functorial on each side they also need to follow certain rules, see Pickering et al [2017] for a detailed and better explanation):

class Profunctor p where
    dimap :: (a → b) → (c → d) → p b c → p a d
    dimap f g = imap f . omap g
    imap :: (a → b) → p b c → p a c
    imap f = dimap f id
    omap :: (c → d) → p a c → p a d
    omap g = dimap id g

In the code we also include the regular map as omap and the contravariant map as imap. We’ll use these short versions in the following snippets.

Profunctor Transformations AKA Optics AKA Proxies

Well, the title says it all, an optic is what we’ve been calling proxy and is the name for special functions that transform a profunctor p a b into one p s t:

type Optic p s t a b = forall p. Profunctor p ⇒ p a b → p s t

The name and order of the type parameters has been the same for a while in the optics libraries.

And then Prisms are?

Prisms in terms of profunctors require an additional restriction: the profunctor should be a choice (also known as co-cartesian) profunctor. Choice profunctors can be lifted to sum types:

class Profunctor p ⇒ Choice p where
    left :: p a b → p (a ⊕ c) (b ⊕ c)
    left = dimap (either Right Left) (either Right Left) . right
    right :: p a b → p (c ⊕ a) (c ⊕ b)
    right = dimap (either Right Left) (either Right Left) . left

The type of a prism is an alias for an optic restricted to choice profunctors:

type Prism s t a b = forall p. Choice p ⇒ Optic p s t a b

Because they are choice profunctors we can transform one into another using the build and match functions (observe we use right to perform the transformations):

prism :: forall s t a b. (b → t) → (s → t ⊕ a) → Prism s t a b
prism build match pab = 
    dimap match (either id id) (right (omap build pab))
                                ^^^^^

Don’t forget about Coprism

Coprisms are based on cochoice profunctors, almost the same as choice except that they un-lift from a sum type:

class Profunctor p ⇒ Cochoice p where
    unleft :: p (a ⊕ d) (b ⊕ d) → p a b
    unleft = unright . dimap (either Right Left) (either Right Left)
    unright :: p (d ⊕ a) (d ⊕ b) → p a b
    unright = unleft . dimap (either Right Left) (either Right Left)

Armed with this special case of profunctors we can write the new alias with the cochoice restriction:

type Coprism s t a b = forall p. Cochoice p ⇒ Optical p s t a b

Coprism can be created with the same type of function arguments that Prisms require, which is why I keep the names of build and match (although you may argue that view and match are better options):

coprism :: forall s t a b. (s → a) → (b → a ⊕ t) → Coprism s t a b
coprism build match pab =
    imap build (unright (dimap (either id id) match pab))
                ^^^^^^^

Again, observe that we use unright to perform the transformation.

In terms of data accessors, coprisms are similar to both lenses and prisms, they provide a way to extract a value build (as view in Lens) but it may fail on the update operation match (as prisms fail to read).

A digression on the co- notation: it is important to not confuse the sense of co- in the name for co-choice, although choice is also known as co-cartesian and a co-co-cartesian should be a cartesian, we can end up thinking that a co-choice is nothing more than a cartesian or strong profunctor which is not. The reason is that we are abusing the co- notation. The co- in co-cartesian name acts on the Set/Hask category and the co- in the co-choice alludes to the profunctor category.

References:

• [2017], Matthew Pickering, Jeremy Gibbons, and Nicolas Wu, Profunctor Optics
• [2018], Oscar Ponce, Another Look Through Optics
• [2015], Russell O’Connor, Grate: A new kind of Optic
• [2014], Russell O’Connor, Mainline Profunctor Heirarchy for Optics