Effective limited concurrent execution in JavaScript
Let’s say you need to save 10000 books into a books service. The service’s API only allows us to save 100 books per request. My first idea would be to split everything into chunks of 100 books and fire everything concurrently.
The downside is that it will create a spike in the load of the books service. It usually means that the service will either struggle to process such a spike or it will be over-provisioned to handle loads like that. Over-provisioning usually means paying for computing power that isn’t used most of the time.
So we want to flatten the spike a bit. To do that we will use what’s called a limited concurrent execution. Basically, we will be sending batches of books concurrently but not more than 10 at a time.
You probably already know that this code has one issue. On each iteration, we have to wait for the slowest operation to complete.
Let’s solve that as well with a promise pool. A promise pool allows you to limit the maximum number of functions running concurrently by waiting for the promises to settle. I’m going to use the @supercharge/promise-pool package, but there are many alternatives.
As you can see we no longer wait for the slowest function to complete and start a new execution as soon as one finished execution.
In this particular example, the total execution time was reduced by a third.
Of course, there there is no limit to perfection. What if the books service has a rate limit? What if there is more than one producer? Your next step then would be setting up a queue like RabbitMQ, Apache Kafka, or Amazon SQS. But that’s another topic.
