Azure functions. Limiting throughput and scalability of a serverless app.

Or how to comply with internal and external constraints.

Azure functions are a great thing, cheap, easy to develop and use, the ideal companion for event-driven application, but there is a specific use case, that I want to talk about.

Sometimes serverless solutions are not event-based and have to communicate with different endpoint types, which have certain restrictions - connection pool limits for PostgreSQL and Azure SQL, request per second limit for external API, and so on.

TL;DR. You will get info on how to limit incoming incoming HTTP traffic to Serverless API and messages stream from Azure IoT Hub. And how to enforce restriction to Azure Functions scale-out and why singleton attribute might be needed.

Disclaimer, these are very specific cases, that can be considered as antipatterns and should be used when there are no other options😈.

Anti-pattern: limit scale-out and concurrency Jeff Hollan

How to limit incoming traffic.

In order to limit inbound HTTP traffic or a number of incoming messages from EventHub, there is a need to edit the host.json configuration file of Azure Function. Bear in mind that online edit in SCM or Portal will restart an application.

App Insights API status (usual)

Use case. Usually, there is a nice situation with a predictable and stable load of project serverless API, and it can handle usage spikes effortlessly. But if there are calls to external API from functions, things can go went haywire during the spike pressure.

To limit the number of HTTP requests, the parameter MaxConcurrentRequests have to be in the range from 1 to any number link, the default value for the Consumption plan is 100. Setting maxOutstandingRequests to anything less than 200 is probably a bad idea unless a client application is implemented correctly retry pattern and can handle 429 errors.

The HTTP 429 status code indicates that the user has sent too many requests in a given amount of time (“rate limiting”).

So, for HttpTrigger based function, the extensions section will look like this.

"extensions": {
"http": {
"routePrefix": "api",
"maxOutstandingRequests": 200,
"maxConcurrentRequests": 1,
"dynamicThrottlesEnabled": true

In the case of Event Hub or IoT Hub Events endpoint (essentially it’s an Event Hub) and need to limit incoming messages stream, configuration is below. Just set maxBatchSize to 1 and enjoy the endless process :).

"extensions": {
"eventHubs": {
"batchCheckpointFrequency": 1,
"eventProcessorOptions": {
"maxBatchSize": 16,
"prefetchCount": 512

Beware though, that 230 seconds is the maximum amount of time that an HTTP triggered function can take to respond to a request.

How to limit instance scale-out.

The cheapest and optimal way to use functions is to host them via the App Service Consumption plan, but if there is a need to control a number of function instances or use it as a singleton, there are few options.

The first(easiest) option is to add the configuration parameter
to settings.local.json file or you can also set it from Azure Portal UI viaApp Service configuration.

"IsEncrypted": true,
"Values": {
"AzureWebJobsStorage": "-----------------------------",
"APPINSIGHTS_INSTRUMENTATIONKEY": "--------------------------",

The second is to use the App Service Premium plan(still in preview) with the D compute instances and the ability to set both the number of pre-warmed instances and the upper scale limit.

The third is to use the App Service Production plan (like S1, P1V2) with the pre-allocated VM’s and VMSS scale. Always running applications is a significant downside here, but it is a more robust solution.

Use case. There was a need to stream data to SQL and avoid “connection pool” exhaustion. So the changes in incoming requests and scale-out limit result in the picture below. Latency is significant, but you might know that worker allocated in the Consumption tier is about (100 ACU 1.5 Gb RAM ). Interestingly, these instances are slower than cheap B1 ones with 100 ACU and 1.75 Gb RAM.

Ideal projection of actual execution via App Insights

However, sometimes the scale-out limit doesn't work.

parameter only reliable if set to a value <= 5. But sometimes it might not work :), like on the screenshot below.

Live metrics stream with 3 instances
Live metrics stream with 3 instances
App Insights live metrics stream with 3 instances live instead of an expected 1

It’s not a bulletproof solution, unfortunately, because with heavy CPU load you will get spin-up of instances, even if the parameter set as follows:


Singleton attribute to help.

This will result in the creation of additional instances, but they will sleep and not be used for your function execution. They will start and then wait for the lock to be released, but one major downside is that they are emitting billing meters while waiting, so you will pay extra money.

Singleton attribute
Singleton lock
Singleton lock
Singleton lock can be observed in App Insights.

PS. There is one more useful limit that can be helpful. App Service parameter DailyMemoryTimeQuota will limit memory throughput for your functions, to avoid excessive billing shock. Which can be the result of load testing 😱 by QA engineers or DDoS attacks. Just calculate it according to your daily load and document it properly :).

az functionapp update --resource-group $functionsGroupName --name $functionAppName --set dailyMemoryTimeQuota=20000

Use anti-patterns wisely, thanks for reading. Cheers!

Useful links.

  • Functions Singleton issue #912 link.
  • Mikhail Shilkov article on scaling link.
  • Azure functions triggers and bindings documentation link.

Stas(Stanislav) Lebedenko is a developer, architect, mentor at the non-profit community IT2School and helping with Microsoft .NET User Group Odesa. Follow him on Twitter at @angry_stas and Odesa Azure/.NET community via #msugodua and @alextumanoff.

Stas(Stanislav) Lebedenko

Written by

Positive geek, serverless fan, senior software dev @ Sigma, an active member of Ukraine .NET/Azure communities.

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade