Stateful Serverless workflows with Azure Durable Functions

Théo Rémy
Jun 2 · 5 min read

Within the Notifications team at ASOS, we are responsible for sending out service emails and push notifications to customers. All the important updates related to your order, your return or your account go through us in the form of events published by other teams across the business. We then use that data to build custom notifications at scale. The volumes we deal with represent an average of 20 notifications sent per second on a normal day which can go up to 100 per second during peak periods.

One of the technical challenges we have had to tackle over the past year was building a service able to aggregate different types of events together in order to inform ASOS customers about their returns. These events are all owned by different teams and are published independently. Each one of them holds valuable information, such as which items were returned, as well as the refund transaction details.

Each day, more than 100k email notifications are sent to ASOS customers, informing them about their return, thanks to Durable Functions. 🚀

As you may know, Azure Functions provides a simple and efficient way to run small pieces of code that can be triggered in various ways (Timer trigger, Http trigger, Blob trigger, etc.) and achieve a single task. They are serverless, meaning that you will not have to worry too much about the infrastructure behind it. However they are stateless, so you cannot link multiple functions together easily and out-of-the-box.

So how to solve this problem? What about the infrastructure, scalability, concurrency, cost and availability? Surely Azure must have something up its sleeve…

Azure Durable Functions to the rescue

The Durable Task framework provides that capability.

Durable Functions is an extension of Azure Functions that lets you write stateful functions in a serverless compute environment. The extension allows you to define stateful workflows by writing orchestrator functions and stateful entities by writing entity functions using the Azure Functions programming model.

Multiple patterns can be implemented, such as function chaining, Fan-out / Fan-in, Monitor, Aggregator, etc. You can see the full overview using the link below.

To achieve our goal, the Aggregator pattern quickly appeared to be the best candidate. It relies on an another concept of the framework: Durable Entities.

The “Aggregator” pattern

This allows us to aggregate event data from multiple sources over indefinite periods of time.

The first thing we need to do is to define the following pieces:

  • Some triggers, or client functions (input)
  • An entity (our state)
  • An orchestrator and some activities (output)

Then, we can start building. FYI: each of the elements listed above will become an actual function living within your Function App in Azure.

For this, we will use Azure Functions v3 running on .NET Core 3.1 along with the following packages:

Microsoft.NET.Sdk.Functions v3.0.11
Microsoft.Azure.Functions.Extensions v1.1.0
Microsoft.Azure.WebJobs.Extensions.DurableTask v2.4.3
Microsoft.Azure.WebJobs.Extensions.ServiceBus v4.3.0
Microsoft.Azure.ServiceBus v5.1.3
Newtonsoft.Json v13.0.1

Please note that the following code snippets have been simplified and stripped of any non-essential lines for readability.

In this example, our triggers will be using the ServiceBusTrigger type and each one of them will be defined as below, with MyEventV1 being the event type we’re expecting to receive:

As you can see, after successfully deserializing the message, we’re instructing the code to ‘signal’ our entity, meaning that we send it an operation message but don’t expect any response. An entity in the context of durable functions is a class that has a unique id and some properties that will define a state. The EntityId is then quite important as it will be the unique reference that will tie the different events together. Following with our example, let’s have a look at what the durable entity would look like:

Pretty simple, right? Now, we also want to start a specific action whenever the current state is ‘complete’ (which is for us to define). In order to do that, let’s use the StartNewOrchestration method. It allows us to launch another function from within the entity. In addition to the name of the function we wish to signal, an input object will be passed on. In our case, this will be the aggregated result of all the events we were expecting. Let’s have a look at the updated code:

And finally, let’s see what the ‘orchestrator’ function looks like. We will use it to process the new event that we have consolidated. In this example, we want to publish it to one of our own service bus topic:

And.. that’s it! Our first Durable Function! 👏 Of course this is just one of many examples of what you can achieve with Durable Functions. Please find below a few extra tips on other aspects of the project that I think are important. 🔽

> Dependency injection

> Cleaning up storage

Here is an example where all instances that have been completed for more than 90 days will be purged every day at midnight:

You could also include additional statuses to clean up instances that have been running for too long. Another good practice would be to set up automated alerts in Azure based on custom Application Insights queries, to stay on top of any irregular behaviour of your app.

There are a few other topics that I wish I could cover in more detail, such as logging and testing, but that will be for another article, so watch this space.

Hi, my name is Théo Rémy, I am a Senior Software Engineer at ASOS. When I’m not busy writing code you will probably find me playing guitar, reading comic books or watching a Star Wars movie.

ASOS is hiring across a range of roles. If you love Azure and are excited by things like Durable Functions, we would love to hear from you! See our open positions here.

The ASOS Tech Blog

A collective effort from ASOS's Tech Team, driven and…

Medium is an open platform where 170 million readers come to find insightful and dynamic thinking. Here, expert and undiscovered voices alike dive into the heart of any topic and bring new ideas to the surface. Learn more

Follow the writers, publications, and topics that matter to you, and you’ll see them on your homepage and in your inbox. Explore

If you have a story to tell, knowledge to share, or a perspective to offer — welcome home. It’s easy and free to post your thinking on any topic. Write on Medium

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store