Today I will talk about messaging queues, its advantages, its limitations, some regular use cases, and how they integrate into PHP language.
Using Queues allows delegating the task that not needs to be processed immediately, sending it to execute as a background job.
In a sequential programming language like PHP, this means that we can process tasks asynchronously in parallel to our master process or HTTP request.
Having this feature implemented can be a significant performance improvement in conventional web applications. Also, it is a very used and robust solution to communicate between systems in more complex or enterprise applications.
What are message queues?
Based on the Technopedia’s definition: A message queue is a component used to communicate between processes or between threads within a process, providing a communication protocol to allow involved systems to don’t need to interact at the same time, allocating the sent messages until the receiver retrieves and processes them.
Queues can be sequential or parallel depending on the use case, architecture, available resources, and so on.
Now, I will explain some advantages and limitations of using queues in the context of a medium-large size web application. Like every tool, it has its trade-offs to consider.
Pros:
- Delegating: It allows delegating the most resource-consuming tasks to background processes. An HTTP request doesn’t need to care about any more of; sending emails, generating CSV or XML files, maybe processing traffic data, everything you can imagine that didn’t need to respond to the current user request.
- Decoupling: It facilitates decoupling the complex business logic from the parts of the application that must need high availability and performance.
- Flow control: It gives the ability to manage massive concurrency tasks linearly. Or the opposite, It allows managing small low consuming processes in a parallel way.
Cons:
- Un-expected replies: In some architectures, a job can fall in different systems at practically the same time, you may need some dedupe layer to avoid this behavior.
- Hard to debug: When applying in legacy applications, it may hide some logic and can become much more complex to debug compared to a linear LAMP web application,
- Resource consumption: You have to be aware of your system resources, adding many queue workers will end in a saturated system.
Status in PHP ecosystem:
Thre are many different mature solutions already adapted to use with PHP. Among the more popular technologies, we have Redis for its simplicity, AMQP for its features to communicate between services, Amazon’s SQS, SNS, or Apache Kafka for its performance, WAMP for real-time bidirectional client-server communication, and so on. You have to decide which of these technologies fits your requirements.
There are libraries like PHP Enqueue to provide an abstracted layer to allow us to choose between many different queueing technologies with the same implementation. We can use it in any PHP project. Also, it has adapters for the most popular frameworks.
Laravel users have out of the box queue system and the Horizon dashboard to avoid debugging and monitoring issues.
Conclusion:
In my opinion, message queueing is a powerful technology, stable and battle-tested. Using it in the right way, we take a lot of advantages in the short term.
Thinking in a more long term, it allows us to develop more decoupled, efficient, and resilient software, separating the immediate actions from all the other business logic, third party communications, or high-resource requiring operations.
We have to take care of applying it, but sure that with a little practice, we can take its benefits.
Personally, it has been a success story in more than one work. It helps a company managing high traffic commercial campaigns withing an Event Sourced architecture, and more recently, reducing five times the time consumed by a long time running daily console script.
As you can see, I like a lot to using queues in my projects, wanna try?
Happy coding;-D
