Synchronize cache updates in Node.js with a Mutex

Photo by Rob King on Unsplash

The problem

We have been working a lot with Node.js microservices lately. Some endpoints require data to be cached to improve performance and save network requests. Cached data must be shared between instances, so we use some kind of remote store (a Memcached or a Redis server).

The client libraries we use provide us with async methods to both read from and write into the cache. Using those methods we can implement a fetch operation that either returns the cache entry for a given key or, if missing, lazy evals a function to get and store the new value.

Seems easy enough. However, the harsh reality of the asynchronous nature of node.js and the event loop has a nasty surprise for us. If we send two parallel requests this is what happens:

Req 1: await get("aKey")
Req 2: await get("aKey")
Req 1: cache miss
Req 1: await for slowAsyncOperation()
Req 2: cache miss
Req 2: await for slowAsyncOperation()
Req 1: await set("aKey", value)
Req 2: await set("aKey", value)

Damn! Each parallel request will try to fetch the value and store it. This is bad. We need to synchronize the cache miss branch and prevent multiple requests to execute the code at the “same” time. What we need is mutual exclusion.

Mutex to the rescue

mutual exclusion is a property of concurrency control, which is instituted for the purpose of preventing race conditions; it is the requirement that one thread of execution never enters its critical section at the same time that another concurrent thread of execution enters its own critical section.

As far as I’m concerned, Node.js does not have native mutex support. The programming model of Node.js enforces to embrace asynchronicity and usually you do not have to worry about mutexes. Looking for an existing solution, I came across an article by Valeri Karpo, “Mutual Exlusion Patterns with Node.js Promises”, that implemented a mutex using promises and an EventEmitter.

There are some npm libraries that implement some kind of mutex support (lock, mutex, async-lock) but none of them feel as simple and elegant as Valeri’s code. His solution (pasted below with original comments):

• Uses a boolean to keep the locked/unlocked state of the mutex.
• Uses promises to wait for the release of the lock.
• Relies on an event emitter to notify the code that is waiting when the lock is released.

Below is a first working version of fetch that uses this “lock” to ensure that only one set is performed.

Improving the code

The code still presents some issues due to the Lock general purpose:

• All writes are synchronized. It would be best to lock by cache key to maximize safe parallel writes.
• It needs an additional get call after entering the mutex to avoid the repeated set calls. We would like to avoid another network trip if possible.

We decided to modify the original lock code to make a specialized “fetch” lock, tailored to our problem:

• Instead of a boolean flag we use a map to allow flagging by cache key.
• The mutex promise now resolves to the new fetched value to avoid the second get operation (we emit the new value through the EventEmmiter).

Below is the new code. We’ve also made two additional changes: we’ve converted the original class into a factory function to make the lock state private using closures, and we’ve removed the max listeners limit in the EventEmitter.

With this new “fetch lock” in place, the final fetch code looks like this:

Summing up

Node.js is by nature asynchronous. It’s great for performant I/O bound stuff and you rarely have to concern yourself with code synchronization. When the need arrives you can use several synchronization patterns. In our case using a mutex got us through the day. We could have used some existing library but by relying on a custom code snippet we avoided an external dependency.