Getting started with Durable Functions
Azure Durable Functions are an extension of Azure Functions, and therefore, has additional benefits over a standard function, such as state management, retry activities and being able to easily cancel workflows that are still in progress. However, for someone that was not used to working with them, there were a few scenarios that I was caught out by.
In this post, I aim to cover the important basics about Durable Functions and how I personally got used to working with them and started to write efficient Durable Functions. This includes covering how state management works, as well as how to deal with long running functions and large data sets.
An Orchestration Client binding allows you to write functions that can interact with Orchestrator functions. The Orchestration Client is used to start the orchestrator function, and can also be used to check the status of the function, as well as terminate them.
The below example shows how an Orchestration Client is used to start the Orchestrator function.
However, for long running functions, you can use the Orchestration Client to ensure that the heavyweight Orchestrator function is run once. If the HTTP trigger is called whilst the function is still running, a Conflict response will be returned until the previous function call has been marked as completed, failed or terminated.
This is important because you could accidentally trigger multiple instances of the same function. Using the second example means that the current running instance must either be completed, failed or terminated before a new instance can be started.
The Difference between Orchestrator and Activity Functions
When a large process is split up, each step should be implemented as an Activity Function. An Orchestrator function usually calls one or more activity functions, it can be used to decide the order in which these functions are executed.
The following shows a basic example of an Orchestrator function calling one Activity function, which returns “Hello World!”. However, you can call as many Activity functions as you like within the Orchestrator function.
How State Management works in Durable Functions
State Management is an important factor when working with Durable Functions.
When an orchestration function is given more work to do (for example, a response message is received or a durable timer expires), the orchestrator wakes up and re-executes the entire function from the start to rebuild the local state.
As an Orchestrator can re-execute the entire function, you have to be careful how you set up your Orchestrator and Activity Functions. Setting these up incorrectly could mean you repeat long running calls, such as database calls and therefore increase the time that your function takes to complete.
If you are calling a service from your activity that does a lot of work, this can cause issues. A bad example could look like the following:
Doing something similar to this example means that when the Orchestrator wakes up and re-executes the entire function, it has no knowledge that this Activity has run in the local state and will make that same service call again.
This can be improved by breaking the work done in the service into multiple Activity functions such as:
Note: This example uses the Function Chaining pattern, which executes the sequence of functions in a specific order:
This will be stored in local state, so that when the Orchestrator re-executes the entire function, it remembers what has been called and does not re-run any long running calls. The other option is batching, which is discussed as part of ‘Dealing with large data sets’.
In the example above, it is important to note the use of
await. The Durable Functions framework makes use of
await to yield the result of each activity. Therefore, the function knows not to re-run that particular activity as the result has been stored in local state.
Orchestrator Code Constraints
Due to the orchestrator waking up and re-executing the entire function from the start to rebuild the local state as per the quote above, there are some orchestrator code constraints. These are usually where the generated value can differ on each execution, such as DateTime.UtcNow and Guid.NewGuid and therefore is nondeterministic.
See Durable Functions Code Constraints for a full list.
Dealing with large data sets
Finally, dealing with large data sets is not the easiest within Durable Functions and something I was not previously familiar with, but is achievable. Using large data sets in Durable Functions can give you exceptions such as:
Exception: “Exception while executing function: #######” — Source: “System.Private.CoreLib”
Inner exception: “Error while handling parameter $return after function returned.” — Source: “Microsoft.Azure.WebJobs.Host”
Inner / Inner exception: “Exception of type ‘System.OutOfMemoryException’ was thrown.” — Source: “System.Private.CoreLib”
These exceptions occur when attempting to return large data sets back from the Activity function to the Orchestrator function. To overcome this, activity functions can be set up to run in batches.
The example uses the MoreLinq nuget package to deal with the batching.
In conclusion, it takes a little while to get used to writing Durable Functions, even if you have experience writing Azure Functions. However, having an understanding of the Orchestration Client and knowing the difference between Orchestrator and Activity functions is very useful for a basic Durable Function. Once you start implementing more complex Durable Functions, this is when you need to start factoring in State Management and Orchestrator Code Constraints. Getting used to writing Durable Functions considering these two factors will help you organise your Activity functions so that your Durable Function runs efficiently.
Durable Orchestrations - Azure Functions
Introduction to the orchestration feature for Azure Durable Functions.
Function types in Azure Durable Functions
Durable Functions is an extension of Azure Functions. You can use Durable Functions for stateful orchestration of…
About the Author
Stephanie is a Senior Software Engineer in the External Product Feeds team at ASOS. When she’s not coding, she enjoys RPM and BodyAttack classes at the gym and spending as much time as possible with her two house rabbits.