Goodbye RabbitMQ Bundle
Symfony’s brand new Messenger component is the real deal
I must confess that since I discovered the power of command buses I can’t stop using them in my applications, even if they are really simple ones. I also must confess that, since I started using Symfony’s brand new Messenger component, I decided not to ever again use another command bus. And I probably also must confess that this is not a sales pitch.
Symfony’s Messenger component is a command bus on steroids. It has been carefully crafted mainly by Samuel Roze and many other Symfony core contributors. The documentation is scarce, and lacking of many practical examples since the component is just a few months old and is marked as experimental (so api changes are likely to be found). But with some digging into the source code, I’ve been able to make good use of it for a couple of applications I’m currently working on.
In this post, I would like to introduce you to this component, how it works, and what are the ways in which you can benefit from it in your applications.
Command Bus 101
Command is a well know pattern. Was featured in the good ’ol patterns explained by the Gang of Four. It’s workings are a sublime object-oriented metaphor of how mailing service works in the real world. A command (or a “message”) is created containing all the necessary information to execute a certain action. Then that message is piped to a series of pre-configured middleware that perform side actions depending of the message type or payload (wrap in a transaction, log the message, etcetera) until it reaches the end of the road in a handler: a (sometimes invokable) class that will execute the action described in the command, and only for that command (and maybe give back a response). Something along these lines:
For the client code using this, there’s a ridiculously simple api:
$commandBus->dispatch(new SomeAppAction($param1, $param2, $etc))
Now, as everything in software, where there are patterns, there are implementations (and many!). We have really lightweight buses that honor their name, like Noback’s Simple Bus. We have the cool and loved Tactician, and now, last but not least, Symfony Messenger (SM, for the friends). The code I posted above will work in any of them effortlessly, and that’s what is so cool of the pattern: except for the middleware, it’s very standard. All of the aforementioned are quality libraries. But I’m not here to praise all of them equally. I fell in love, folks.
“Baby, you’re so special”
That’s what I say to my screen when I’m working with the Messenger component (and when my wife is not around). But, “what makes SM so special?” you ask. Well, apart of having all the traditional features of a command bus, it also comes with a full fledge routing system, capable of working with multiple transports (a thing that defines the way a message is sent, received, encoded and decoded). What that means, in short words, is that you can send a message with that command bus pretty much wherever you want: Rabbit, Amazon SQS, Redis, Google Pub/Sub, you name it.
Now, here’s the best part. Let’s say that you are working in a DDD application. So you have your commands that affect your aggregate root, and in the process some domain events are fired. But you want to handle those events asynchronously, as a good DDD follower. Well, one thing you could do is that, right in the repository implementation, when the aggregate root is persisted, you can fetch its events and pass them to the injected “EventBus” (you can define multiple buses with the messenger component) and handle them with a new set of middleware (persist the event, send it to elastic, log it, etcetera). But, at the end of the pipe, the message will be serialized and sent with the amqp transport to Rabbit. Achieving that, is as simple as adding a yaml config that will send all the messages implementing certain interface to a specific transport.
Then when you consume your messages with the provided console command, they go straight into your default command bus after being decoded into objects again, and are handled as if they were regular application commands. That is just powerful.
Even if you are not doing DDD, you can just use it for executing async actions in your application with no major hassle. By means of a simple DTO and a corresponding handler.
Why I’m saying goodbye
I couldn’t avoid thinking of my days with my good Rabbit MQ Bundle. I just remember the verbose and complicated config files, and all the parameters to fill. It was just not nice. Then Enqueue bundle came, and it was much simpler, but still had all that Producer, Consumer jargon, and those interfaces that polluted my business logic.
But the Messenger component is elegant. It works hard to not get in your way and make the broker part of aysnc processing as transparent as it can be. So much, that your handlers do not contain any piece of code relative to message queuing, and the configuration you have to do to achieve that, is minimal.
So I get two of the things that I frequently use in my apps, spectacularly combined into one: a command bus and a producer/consumer. And this consumer knows my handlers and can find them. I feel like it’s Christmas already.
Avoiding Pitfalls
Of course, like any new piece of tech, you not only need to know how to use it, but also how not to use it.
First rule of thumb is: design your messages in a serializer friendly way. Specially those destined to the transport. Just keep them simple. Do not use named constructors and if you make them immutable (that you should) be consistent with the argument names in the constructor and in the properties, so you can denormalize your message using Symfony’s PropertyNormalizer. Remember that those are serialized into json; they are not serialized the PHP way. This is so they can be used by workers in other languages if so you prefer it.
UPDATE @ 2019–05–09: Starting at Symfony 4.3, messages will not be serialized to JSON anymore, but to PHP’s native serialization string format. Symfony team decided it was too cumbersome to serialize messages to json for devs, because in most cases it forced specific methods or property names to be present in the message class, breaking encapsulation and making them mutable. Serializing using the native PHP serializer is simpler and much more effective. This is the PR describing the proposal, discussion and implementation of the change. https://github.com/symfony/symfony/pull/29958. Thanks to Tac Tacelosky for the heads up!
Also, keep in mind eventual consistency. Anything that you sent to the queue is theoretically guaranteed to happen, but we don’t know when exactly. Most operations are executed extremely fast, specially because all the kernel bootstrapping is already in place in the worker’s loop. Keep an eye to your queues, and beware of operations that can make the message fail and requeue ad infinitum. When a irrecoverable error occurs, throw an exception implementing the RejectMessageExceptionInterface, so the message is discarded. You would do well to look a the amqp reciever loop here to have an idea on how your message is handled.
UPDATE @ 2019–05–09: As Max Kotliar rightly points, if you follow my poorly thought recommendation of throwing a RejectMessageExceptionInterface, you will be bounding your domain logic with the specific queue implementation. Here’s a better alternative (1) create an special exception in your domain for irrecoverable errors, not implementing RejectMessageExceptionInterface, and (2) create a middleware in the message bus that catches that kind of exception and throws instead an anonymous class exception implementing the RejectMessageException. This is a much more cleaner, loosely-coupled approach.
Lastly, consume your messages with a time limit or message limit, and keep it running with tools like supervisor. This is because PHP is not really that good handling long processes, so your queue is going to crash at some point. Control that crash by using those means. Or, try to make your handlers memory efficient.
