Solving the Readers-Writers problem in a multithreaded environment.

No BS. Straight to the use case.

At work, in one of our services, we were using an OAuth enabled API. For every request made to this API, the authentication token had to be passed in the request header.

What we were doing till now to make any API call:

1. Make an API call to get a fresh authentication token.
2. Use this token to make any other API call to fulfil our request.

Something like this:

What we were doing till now…

PROBLEM!

Every time callApi() is called, getting a fresh authentication token is totally unecessary. We could use the same authentication token for multiple API calls. Imagine doing this at scale.

The solution: Get an authentication token as the app starts and save it. Use this authentication token for every other call to this API.

Maybe something like this?

Initializing the oAuthToken in the constructor and re-using it

Changes done:

1. Introduced a class variable oAuthToken.
2. Introduced a constructor for APICaller class.
3. oAuthToken is initialized in the constructor.

Result: With this, we no more have to get a fresh authentication token in callApi(). We could simply instantiate APICaller and it initializes the oAuthToken during object initialisation.

ANOTHER PROBLEM!

Out of La-la land, in real life, we cannot keep using the same authentication token all our lives. It will expire after some time. And hence, we need to update the oAuthToken variable before the authentication token expires. Else our callApi() method will start failing.

Solution: Run the getOAuthToken()method periodically. In our use case, we chose to run this every 45 minutes arbitrarily.

Have a look at this.

Running getOAuthToken() every 45 minutes

Changes done:

1. Removed the constructor.
2. Changed the method name of getOAuthToken() to refreshAuthToken() , changed its return type to void. This method updates the class variable oAuthToken.
3. Added a Spring boot annotation @Scheduled with a fixedDelay parameter. This just runs the refreshAuthToken() every 45 minutes.

Result: This would refresh the authentication token and update the value of oAuthToken every 45 minutes. Hence, we eliminate the case of using the expired authentication token.

MORE PROBLEMS!

What happens when hundreds of threads are using the same oAuthToken while the scheduler thread tries to update the oAuthToken?

Well, nothing Major. Or is it?

Readers-Writers Problem

This is the famous Readers-Writers problem. oAuthToken is a piece of data that is shared across multiple threads. It is a shared resource. It may very well be possible that some threads use the expired authentication token even after the scheduler thread has refreshed it. Hence, we know our data won’t be consistent across multiple threads.

To put things simply, lets say we have 2 threads. Now if both threads perform read or write operations simultaneously, our happy case would only be when both threads are performing read operations. Below table describes it better:

Table describing happy and problem cases

In our case, we may have N number of readers and just 1 writer.

How do we ensure data consistency in this situation?

2 things to ensure here as a part of our solution:

1. Do not let any thread read or update oAuthToken when the scheduler thread is refreshing it.
2. Do not let the scheduler thread update the oAuthToken when there are threads reading it.

We could achieve this with a pair of read/write locks. Essentially,

1. Read lock can be acquired by multiple threads simultaneously only when the write lock is not acquired by any thread.
2. Write lock can be acquired by 1 thread at a time. And can be acquired only when no thread is holding on to read lock.

Java provides a ReadWriteLock interface in the java.util.concurrent.locks package. We could simply use to achieve data consistency.

And finally, our prod ready code 😃:

Makes use of a ReadWriteLock to achieve data consistency

Changes made:

1. Made use of ReadWriteLock. Initialized the lock in the constructor.
2. Instead of directly reading the oAuthToken in callApi() method, we now call a getOAuthToken() method.
3. IngetOAuthToken(), all we do is acquire the read lock, return the oAuthToken, and finally release the read lock.
4. In refreshAuthToken(), we acquire the write lock, update the oAuthToken, and release the write lock.

We do not have to handle the complexity of ensuring if no thread has acquired the read or write lock while we acquire the write lock, or to ensure if no thread has acquired the write lock while we acquire the read lock. Java handles the complexity for us.

HOLD ON! NOT JUST YET!

We still have a problem. Our writer thread that updates oAuthToken every 45 minutes might just starve for a very long time, possibly forever. For a write lock to be acquired, no thread should be acquiring the read lock or the write lock. What happens when our service gets too busy and read lock is never released? Lets say our scheduler thread is waiting to acquire the write lock while read requests keep coming again and again?

How do we ensure threads do not starve?

Solution: Well, not a very complicated task. All we need to do is set the fairness policy to true in the ReentrantReadWriteLock constructor. This would guarantee some fairness to the threads that they will acquire the lock.

Just like this:

Fairness policy set

Result: We minimized the possibility of our threads to starve. You can read more about fairness policy here.

Conclusion: We manage to achieve consistency of data (oAuthToken) across multiple threads.

I have whole heartedly written this to educate you and give back to the programming community. When I teach, I learn more. And when I learn more from you, I can teach more to others. Please feel free to comment if you spot any mistakes, or any misinformation I might have unintentionally produced here, your criticisms and your reviews. 😄