Async Await in akka.net ReceiveAsync v.s. PipeTo

When you using async code with akka.net, you will come across this blog post. (https://petabridge.com/blog/akkadotnet-async-actors-using-pipeto/)

And recently i ran into a problem, later turned out to be a really interesting one. So here is this blog.

A simple overview of the problem

In sample code I have created:

1. A caller actor, which make (fake) API calls.
2. AsyncCallEmulator is used to emulate API call with a delayed response.
   After response, if returned, the caller will pass response to a child actor (receiver).
3. when receiver received the message from the caller, print the response.

simple!

see source code: https://github.com/zdjohn/asyncpipetosample

An extremely simplified version of the problem.

PipeTo is simple and efficient

Instead of being blocked and waiting for the response coming back. PipeTo enables threads to continue processing messages from inbox while letting async code running in a separate thread.

What can go wrong?

There is a catch:

When there are thousands of requests, using PipeTo means you are pushing huge of API requests to your downstream API without controlling request rate.
In this case, your requesting service would either bail or throttle the requests.

When that happens the actors system will start falling apart. you could also kill your APIs with a DDOS style execution.

There could be more problems adding to the above. As requests lining up, your connection is opened for every request while waiting for the response coming back, your connections pool would exhaust quickly.

Semaphore as a capacity locker

if you google “async” “throttle” or “rate limit”. You will come to see a lot of posts mentioning “semaphore”.

The idea is simple. Creating a `Semaphore` in your API handler instance. Then you define how many threads you want it to be run in parallel.

so every time, before API request is made. it will check the semaphore. if the maximum count is reached, code will wait for the semaphore to be released.

The code goes like this: (Checkout branch `semaphore` for this solution)

https://github.com/zdjohn/asyncpipetosample/tree/semaphore

public async Task<int> CallWithDelayedResponse()
{
await _semaphore.WaitAsync();
var delay = random.Next(_min, _max);
Console.WriteLine($"call open with expected delay of {delay}");
await Task.Delay(delay).ConfigureAwait(false);
Console.WriteLine($"call closed with delay {delay}");
_semaphore.Release();
return delay;
}

by doing this, it means you will need to add your semaphore at your API handler level. and every API service you call will need to have its own semaphore, in order to control the throughput accordingly.

we see here even a lot of message is received, but there is only 5 connection opened at the maximum

new connection opens after previous connection closed

let's use AsycReceive

We still strongly recommend PipeTo over asyc / await for performance and cohesion reasons but there are scenarios where the latter makes life easier.

When i was reading the quote above, i had questions in my head. When is the time of “makes life easier”? as the mailbox will be suspended on actors when using async and await until the last continuation is executed.

But turns out this is perfect in my case. if we want to control the parallelism of our HTTP requests. instead of having SemaphoreSlim and fiddling with threads. So, instead of controlling the threads number to gain control of the throughput of requests. here we control the throughput of messages. from actor model perspective, i find it more “pattern matching”

Plus AsycReceive is much easier, by using the async support + routing

1. we use RoundRobinePool to set up a fix sized actors pool

2. Use async/await as you used to know, inside actor ReceiveAsync

This setup resulted in much better control of concurrent request with much less code.

This is all the code changes!

see code in branch `receive-async` https://github.com/zdjohn/asyncpipetosample/tree/receive-async

Now you can see, no more than 5 messages are processed until the previous async call is closed

Unsolved problem

To be honest, i tried a few rounds to find the right pool size for actors i want to set a cap on. (even though hocon config did make it really easy without requiring code change).

However, due to the service capacity difference, and response time difference. i have to have different settings for my staging and prod environments.

if there is a smart way of working out a suitable size automatically, I’d love to know.