[System Design] Message Queue — In a Nutshell(with OOP code example)
Imagine, you go to the bank to deposit some money. The bank is having a lot of rush and due to which, you need to stand in a line, and wait till your chance comes.
Time passed and you are about to get the counter. Suddenly your mobile rings. It’s an urgent call and you can’t ignore the call.
In order to take the call, you get out of the line, by informing the person, next to you, that you'll be right back. After the call ends, you go back and find, that the line is still there, and counter is still busy.
you reach the counter, and try to access it. But the person standing at the counter stops you as, it’s his chance now. You try to get into the middle of line/queue , but everyone resists and tells you to join at the end. So, you are left with only one option, i.e. join at the end and wait again for your chance.
The Main Problem
The problem is not standing in the line, or waiting for the chance. The issue comes, when you leave the line due to some unavoidable circumstances and then expects to get the counter again within the same time.
For example, in above scenario, if you didn’t get the call, you would have got the counter in 2 or 3 minutes. since you took the call and came back, even if your call took less than a minute, you won’t be able to get the counter again, as people next to you would have moved ahead and now they are in the same position, as you were few minutes ago. So why will the give you the place?
The problem can solved, if every member in the line/queue has a number assigned and you can’t get the counter till the numbers before your number get called.
The Token System [Real World Solution]
Majority of the banks have this system already implemented. It states that, In order to perform any action/service, you take a token from the token machine and wait for your chance.
A token is nothing but a unique number, which is valid for 24 hours. There will a screen on which the token number flashes, and respective guy can go to the counter for service. Since, every person has a token, he or she doesn’t have to stand in a line/queue. They can do whatever they want to do to pass the time, and wherever too.
But…How does this system work?
As soon as the token gets assigned to the customer, the token system sends it to a queue. The queue is nothing but a virtual line, in which your token on behalf of you, is standing and waiting for the chance.
The Token queue is nothing but a Message Queue.
The end system takes token from the queue one by one, and starts calling respective customers for service. In this way, you don't have to worry about anything.
Since the token number assures that you will get the chance after fixed number of people, it’s a kind of acknowledgement from the bank to you that you will surely be addressed despite knowing, who you are and without letting you worry about the bank load, server issues, etc.
Message Queue [Store, Acknowledge & Wait]
A message queue provides an asynchronous communications protocol, which is a system that puts a message onto a message queue and does not require an immediate response to continuing processing.
Since it’s asynchronous, it keeps both the end points mutually exclusive. The producer adding items to queue, doesn't need to care about the process on the other side. The Consumer can also handle it's work independently, taking the messages from the queue without worrying about producers’ state.
Don't worry, if you don’t get the definition, Let’s try to implement it, for more clarity.
Implementation with OOP —Let’s begin!!
At a very high level, we see the design closely, there are 3 entities,
(let’s take the bank’s example)
- Token System — which updates the token queue by adding a token.
- Token Queue — saves the tokens from Token System, and removes token from Bank system.
- Bank Internal System — which also updates the token queue by removing a token.
So, the design looks quite simple…!!!
But…there is a problem?
Since, the token Queue is going to be a shared resource between the bank system and token system, the token Queue needs to take care of following checks :
- It should not add the same token twice.
- It should not allow the bank system to add any token.
- It should not allow the token system to remove any token.
Point 1 can be taken care internally, but for Point 2 and 3, we need to divide the design in such a way, that both the systems, should be able to access only the intended functionality required.
So, How can we solve this?
Divide the functionalities — add Interfaces
Let’s create 2 interfaces,
- Producer Interface — adding to queue
2. Consumer Interface— removing from queue
Let the Token Queue/Message Queue implement these 2 interfaces and add the required functionalities.
3. Token Queue/Message Queue
Now, we just need to give respective APIs to Bank and Token Systems.
The beauty of this design is,
- We can have all the functionalities packaged in a single class, but in order to access each functionality, the caller needs to have the respective API access.
- The token system will use MessageQueueProducerInterface and only it’s methods will be visible to token system. So, it won't be able to know, any other methods of Message queue class.
- Same as Point 2, Bank system uses MessageQueueConsumerInterface and only it’s methods will be visible to them, nothing else.
4. Token system — will use producer API.
5. Bank System — will use consumer API.
Obviously, the code snippets are just for reference and just a high level representation of the actual system, but still you can get an idea, how does message queue work?
and finally, we reached the END.
You made it. Grab a coffee, sit back and relax. Also, do comment, if you find anything interesting or missing.
