Carry On! … Continuation over binding pipelines for functional web

6 min readJul 13, 2017

Update 2018: This is now core composition design of Giraffe Library.

Functionally composing pipelines provides a very nice developer experience allowing the developer to compose many smaller reusable functions into a full pipeline of functionality. In particular, these are used in Suave and Giraffe, f# web server frameworks that make web server programming functional. I will focus on Giraffe as the two have similar formats and Giraffe is the one I have been playing around with.

I will presume you have some level of familiarity with Suave/Giraffe already, so if not please become a little familiar before reading on.

Binding

As per below, you can see how binding & composition work in Giraffe, every composabale HttpHandler function, the building blocks of the pipeline, require 2 functions to compose together, bind & compose:

The typical format of the handlers look like the following and show how an async option of Some HttpContext is returned from a function to communicate success to the binder, or anasync None is passed to indicate a failed route.

Async Wrapping (of sync)

The one major problem is that although many of the operations of HttpHandlers will by async, (writing to the response, fetching external data, IO), many are also sync (route tests, reading from headers, Verbs etc), so as can be seen from the examples, we are forced to wrap sync results in async just to “make the pipeline work”. I will refer to async wrapper as Tasks interchangeably , inline with c# as this is what Asp.net Core uses throughout. The issue is creating tasks are expensive as you are not only creating the object but it needs to be added to scheduler to be picked back up again to be treated as a normal task in the task pipeline.

ValueTask….

A work around for this issue that asp.net core have been developing is aValueTask<T> struct type that has cells for a task and value reference so that if it is constructed with a value, no Task needs to be created. Although this helps, its not very compatible in f#, and is still a crutch to work around an issue that can be solved in better ways.

Continuations

I was trying to think of ways to remove this need to wastefully wrap sync functions in async when I realised there was an elegant way to not only remove the wasteful creation, but also improve the stack pressure and performance if we could instead use continuations, rather then HttpHandlers evaluating a path (Some/None) back to the bind function every time, instead a function could be provided that represents the next HttpHandler in the pipeline and the async return of the current HttpHandler would instead, just be the return of that next subsequent function… Carry on, don’t pass backwards and forwards.

Tail Call Optimisation

The benefits of this approach begin to (ironically) stack up as not only are we not creating Tasks where they are not needed but we are also benefiting from tail call optimisation, as there is no additional stake frames required for the subsequently called functions, while conversely with binding, there is a stack frame required all the way (while each bind awaits its result). TCO is most commonly used in f# with tail recursion, and although the stack wont quite grow as big as many recursive functions would, for long pipelines there is little stack frame pressure with this tail call optimisation.

Replicated branch (if/else) assessment

Another issue with bind, although small, is the replication of branch path logic, and the creation of option objects in the process. What I mean by this is, we have our handler code that eventually comes to some logic branch of if true then Some ctx else None and then returns the result out to the bind function that then has to evaluate which branch we’re taking depending on the option value, duplicated conditional statement with interim temporary heap allocated option, small but adds up and is needlessly wasteful.

An (initial) alternate continuation format

I re-wrote the Handlers to use the different continuation format and quickly realised that for Handler cases like choose we needed some way to fail the path that would not just be cancelling a task, but could progress to an alternate route of execution, so initially I tried out a three parameter function of succ(ess) continuation, fail continuation, and thirdly, as before, the ctx of HttpContext.

Although this format worked fine, fixed the sync wrapping of async, and benefited from tail call optimization, it was a bit clunky for development for people who where not too confident with function currying and partial application. Using this format improved total server performance by approx 9% over current bind system in rough benchmarks.

Reinventing the wheel…

I was reading through some of the early design blogs on asp.net core and found one mentioning that a breaking change was the introduction of the next requestDelegate, as returning results and binding testing were slowing things down. I realised then that I clearly was not the first person to reach this conclusion, and in reading the requestDelegate name ( next ), I remembered, it was the same in node/express also, the handler functions provide a next continuation function to allow flow to continue rather then returning truthy values for assessment by some binder. Although I was a little disheartened to realise I was bit slow to come to this obvious conclusion, I now knew what I needed was a next continuation.

Our composition is now 1 function, 5 lines of code vs 2 functions of 10 lines.

Some -> Next

The interesting thing about just adding a singlenext function (continuation) to the HttpHandler input parameters was that the HttpHandlerResult Could be the same format of an async HttpContext option . I know it may seem like this format has some holes in it but it works almost the exact same as before only instead of using the function Some (Option.Some: ctx -> ctx option) that create the option value, we use a different function that instead of creating an option, is a continuation of the next function in the pipeline.

We can even name next function value Some' to make it look almost the exact same (only for the extra input parameter, and the missing need to wrap Some in an async), but next avoids confusion.

To get the whole pipleline running we would need an additional parameter in the middleware, this new next function, the function that would be passed down the pipeline to be the final function executed … and given the format we are already using it is simply a function that returns our Some ctx

So still binding options?

‘So, if we are using option of HttpContext still are we still not back to the binding issue?’, the answer is mostly no , with most of the pipeline using next instead of Some , it is able to tail call all the way without needing to create async wraps on sync (although obviously needed when async functionality in the Handler) so that the only stack frames that build up are on functions like choose where we need to bind the option to change branch, but moving down a branch is all continuation benefiting with TCO. Importantly the developer can still think in options so less of a drastic change in format.

Short-circuit both ways!

Another benefit of the continuation format is that it allows you to short-circuit the pipeline for both Success and failure, if handler and branch fails, return async None if a result/conclusion is arrived at before processing the full pipeline, a handler can return an async Some ctx (like original return) but now it explicitly signifies the success of the operation, not the moving to the next handler.

For your consideration…

Having outlined the pros, with little cons (I can see) other then breaking changes and the “extra complexity” of one additional function parameter, hopefully someday soon we can see a new web framework like Giraffe adopt this more efficient format to prevent wrapping sync in async as well as adding to performance. Would love to hear any feedback or suggestions of a better approach so feel free to comment.

In Summary:

for vast majority of handlers (linear none “choose”)

Avoid having to wrap sync results in expensive async on every handler
Tail Call Optimised to minimise stack pressure
No logic branch duplication (if/else, create option object/match)
Type pattern mostly unchanged, just adding next continuation parameter on HttpHander to be used instead of Some option ctor