NB: this post was updated 8th Jan 2018 to account for changed image names and invocation being “hot” by default.
FnProject is out now. Chad Arimura explained the motivation and structure of the project in a good amount of detail in his recent post, with one of the major components being Fn Flow. Flow allows developers to build high-level workflows of functions with some notable features:
- Flexible model of function composition. Sequencing, fan out/in, retries, error-handling and more.
- Code-driven. Flow does not rely on massive yaml or json descriptors or visual graph-designers. Workflows are defined in code, expressed as a function, naturally.
- Inspectable. Flow shows you the current state of your workflow, allows you to drill into each stage to see logs, stack traces and more.
- Language agnostic. The initial Flow implementation which this post will use is in Java but support for other langauges has already started including Python, Go and JS.
What is a Flow?
A Flow is a way of linking together functions, and incidentally provides a way to define those functions inline if you need to. It’s a FaaS-friendly way of saying stuff like
Start with this, then do that, then take the result and do these things in parallel then when they’ve all finished do this one last thing, and if there’s any errors then do this to recover
Where all the this and that are FaaS functions.
A simple Flow function looks like this:
Anyway it’s easy to tell the stages of what’s going to happen:
- Start with a value provided by the user
- Apply some transformation
i -> i+1
- Pass that to an external function called
- Then return get the result and return it
Flow class submits each stage in this workflow to an Fn service we call the "Flow Server". You'll meet it soon. The Flow Server will then orchestrate each stage as an individual call to Fn. The Flow Server is responsible for working out which stages are ready to be called, calling them, handling the results and triggering any following stages until you reach the point where there's no more work to do.
This example could easily be written without Flow but it’s good to start simple.
Running your first Flow
Currently FnProject is available to download, to experiment with, and to run on your private cloud. A managed service by Oracle is in the works. To play with Flow at the moment you will need to run everything locally, but it’s not hard. We need
fn, the Fn server, the Flow Server and not necessary but nice-to-have is the Flow UI
fn CLI tool:
Then start the Fn server:
The Flow Server needs to know how to call the Fn server, so ask Docker which IP address to use.
Start the Flow Server:
Then start the Flow UI:
Now, everything’s set so lets crack on!
A simple Flow function
Create a new function:
Flow has a comprehensive test framework, but lets concentrate on playing with the code for the time being:
Make peace with yourself after that, then let’s get the code in shape. Change
HelloFunction.java to look like this:
Then deploy this to an app which we call
flow101 on the local Fn server, and configure the function to talk to the Flow Server
You can now invoke the function using
or equivalently with
Exploring the UI
Browsing to http://localhost:3002 you should see something like this:
Which is showing us 3 function invocations:
- The main flow function, in blue
.thenApplyfor the code
i -> i*2
.thenApplyfor the code
i -> "Your number is " + i
Click on any of these and see the detail for each one expanded at the bottom of the screen.
The blue function is shown as running for the whole time that the
thenApply stages are. Why? Because we are calling
.get() at the end, so this is synchronously waiting for the final result of the chain. Exercise: Try removing the
.get() from the code (you'll need to return a different String, and don't forget to re-deploy). Now it will look like:
This shows that Flow is well-suited for asynchronous functions which result in a side-effect (posting to slack, for example).
Why we made Flow
Consider how else you could have achieved what we did in 4 lines of Java above. First of all there are actually 3 different functions at play, each of which would need its own codebase and entrypoint. Then, consider that Flow chains values through them while preserving type information. We could have written an overarching “Call function A using an HTTP client, call function B the same” type orchestrator function (ugh), or even hard-coded each function to call the next one in the chain (ugh ugh). Perhaps we could have used an Enterprise Service Bus to glue it all together (multiple ughs).
We think Flow hits a very sweet spot of allowing sophisticated stateful apps defined in code, and maintaining the promised benefits of FaaS a la Serverless Manifesto. We think you’ll like it too.
So, congratulations — we’ve covered a lot! You’ve got a Flow function running, seen how to use the API to compose simple transformations and run things in parallel. Head to the Flow 102 post to take your Flows to the next level.
Originally published at mjg123.github.io.