Stateful Serverless workflows with Azure Durable Functions

Within the Notifications team at ASOS, we are responsible for sending out service emails and push notifications to customers. All the important updates related to your order, your return or your account go through us in the form of events published by other teams across the business. We then use that data to build custom notifications at scale. The volumes we deal with represent an average of 20 notifications sent per second on a normal day which can go up to 100 per second during peak periods.

One of the technical challenges we have had to tackle over the past year was building a service able to aggregate different types of events together in order to inform ASOS customers about their returns. These events are all owned by different teams and are published independently. Each one of them holds valuable information, such as which items were returned, as well as the refund transaction details.

Each day, more than 100k email notifications are sent to ASOS customers, informing them about their return, thanks to Durable Functions. 🚀

As you may know, Azure Functions provides a simple and efficient way to run small pieces of code that can be triggered in various ways (Timer trigger, Http trigger, Blob trigger, etc.) and achieve a single task. They are serverless, meaning that you will not have to worry too much about the infrastructure behind it. However they are stateless, so you cannot link multiple functions together easily and out-of-the-box.

So how to solve this problem? What about the infrastructure, scalability, concurrency, cost and availability? Surely Azure must have something up its sleeve…

Azure Durable Functions to the rescue

What if we were able to combine the power and simplicity of Azure Functions with the concept of long-running workflows? To be able to wait indefinitely for asynchronous triggers, and act accordingly.

The Durable Task framework provides that capability.

Durable Functions is an extension of Azure Functions that lets you write stateful functions in a serverless compute environment. The extension allows you to define stateful workflows by writing orchestrator functions and stateful entities by writing entity functions using the Azure Functions programming model.

Multiple patterns can be implemented, such as function chaining, Fan-out / Fan-in, Monitor, Aggregator, etc. You can see the full overview using the link below.

Durable Functions Overview - Azure

To achieve our goal, the Aggregator pattern quickly appeared to be the best candidate. It relies on an another concept of the framework: Durable Entities.

The “Aggregator” pattern

This allows us to aggregate event data from multiple sources over indefinite periods of time.

The first thing we need to do is to define the following pieces:

• Some triggers, or client functions (input)
• An entity (our state)
• An orchestrator and some activities (output)

Then, we can start building. FYI: each of the elements listed above will become an actual function living within your Function App in Azure.

For this, we will use Azure Functions v3 running on .NET Core 3.1 along with the following packages:

Microsoft.NET.Sdk.Functions v3.0.11
Microsoft.Azure.Functions.Extensions v1.1.0
Microsoft.Azure.WebJobs.Extensions.DurableTask v2.4.3
Microsoft.Azure.WebJobs.Extensions.ServiceBus v4.3.0
Microsoft.Azure.ServiceBus v5.1.3
Newtonsoft.Json v13.0.1

Please note that the following code snippets have been simplified and stripped of any non-essential lines for readability.

In this example, our triggers will be using the ServiceBusTrigger type and each one of them will be defined as below, with MyEventV1 being the event type we’re expecting to receive:

As you can see, after successfully deserializing the message, we’re instructing the code to ‘signal’ our entity, meaning that we send it an operation message but don’t expect any response. An entity in the context of durable functions is a class that has a unique id and some properties that will define a state. The EntityId is then quite important as it will be the unique reference that will tie the different events together. Following with our example, let’s have a look at what the durable entity would look like:

Pretty simple, right? Now, we also want to start a specific action whenever the current state is ‘complete’ (which is for us to define). In order to do that, let’s use the StartNewOrchestration method. It allows us to launch another function from within the entity. In addition to the name of the function we wish to signal, an input object will be passed on. In our case, this will be the aggregated result of all the events we were expecting. Let’s have a look at the updated code:

And finally, let’s see what the ‘orchestrator’ function looks like. We will use it to process the new event that we have consolidated. In this example, we want to publish it to one of our own service bus topic:

And.. that’s it! Our first Durable Function! 👏 Of course this is just one of many examples of what you can achieve with Durable Functions. Please find below a few extra tips on other aspects of the project that I think are important. 🔽

> Dependency injection

You can easily use dependency injection within your Durable Function. The first step is to make the functions non-static. Then, create a Startup.cs class as you would do for a traditional .NET Core application. Except this one will inherit from FunctionsStartup, which is available as part of the Microsoft.Azure.Functions.Extensionspackage:

> Cleaning up storage

With Durable Functions, because a state is stored, you will notice that once the function is running, the associated storage account will start being populated with two tables: History and Instances. Out of the box, they will just keep growing as you go, as there is no built-in mechanism to clean them automatically. That is why the authors of the framework recently introduced a method called PurgeInstanceHistoryAsync which you can use from an IDurableOrchestrationClient and execute on a regular basis.

Here is an example where all instances that have been completed for more than 90 days will be purged every day at midnight:

You could also include additional statuses to clean up instances that have been running for too long. Another good practice would be to set up automated alerts in Azure based on custom Application Insights queries, to stay on top of any irregular behaviour of your app.

There are a few other topics that I wish I could cover in more detail, such as logging and testing, but that will be for another article, so watch this space.

Hi, my name is Théo Rémy, I am a Senior Software Engineer at ASOS. When I’m not busy writing code you will probably find me playing guitar, reading comic books or watching a Star Wars movie.

ASOS is hiring across a range of roles. If you love Azure and are excited by things like Durable Functions, we would love to hear from you! See our open positions here.