Dealing with big website projects is cumbersome when we’re using monolith architecture. As a Laravel developer, it’s quite challenging, because basically, Laravel offers us a full-stack development where you can work with Front-end as well as Back-end in the same project, in other words, monolithic application. The question is, can we develop microservices using Laravel (or maybe Lumen) for a bigger project? Of course, we can.
Before we go on, what are the bad things about developing apps using monolithic architecture? Well, because all the functionalities are inside in one project, means that every time we make any changes (even a small one) we’ll re-deploy our entire app, which may affect the whole app. With the big-scale project, it’ll much harder to maintain. So, microservices architecture comes to help, where we can separate its functionality into a standalone component.
A message broker acts as a middleman for the microservices, receiving messages from one application (producers) and handing them over to others (consumers) to do the job. — https://www.cloudamqp.com
When we’re dealing with microservices, we need to prepare something for our microservice internal communications. Based on this article, at least there are 3 ways to communicate:
1) REST APIs
2) Remote Procedure Calls (RPC)
3) Brokers (RabbitMQ, Apache Kafka, etc)
This article will show you how to use RabbitMQ for our microservice communication using Laravel applications. The good thing about using RabbitMQ is that it’ll queue our messages. Let’s say, the receiver is busy or disconnected right now, then the message will be stored inside the temporary RabbitMQ Storage and will push the message again whenever the consumer/receiver is ready.
- Our Goals
- Cloud AMQP
- Laravel Project Setup
- Dispatching PingJob
- Dispatching UserCreated Job
- Several Improvements
#1 Our Goals
Let’s start with the result of what we’re going to build in this article. We’ll need to prepare 2 services. One of them will act as a consumer/receiver, where it’ll listen to any incoming messages. The other one will act as a producer/publisher that pushes some messages.
As you can see, the right terminal is acting as a producer/publisher. First, it published the very standard ping job, and the second try published the user job where it’ll pass the user data into our receiver. As you might guess, when the receiver received some data (or user data in this case) we can do anything with the data inside the receiver, right?
If you notice, there’s a delay between sending a message and receiving it. That’s okay because in this demo I was using Cloud AMQP where the server is far from my country right now. But at the end of this article, in the improvements section, I’ll show you to set up your own RabbitMQ in your local with docker, and of course, it’ll be much faster.
#2 Cloud AMQP
There are two ways to use RabbitMQ. The first one, we can install RabbitMQ into our local machine, or the second one is using Cloud AMQP as a service, so we don’t need to install anything inside our local. For this article, the second option is pretty much simple to do. So let’s stick with that and do the registration first on this website https://www.cloudamqp.com.
After registration succeeds, then create a new instance and see the instance details. The information we need is the hosts, user & vhost, and the password. After we get what we need, let’s move on.
#3 Laravel Project Setup
I assume you’ve installed two Laravel projects in your local. For using the RabbitMQ, we’ll use this package https://github.com/vyuldashev/laravel-queue-rabbitmq. So let’s install the package in both of our projects.
composer require vladimir-yuldashev/laravel-queue-rabbitmq
Then, open your config/queue.php file, then add a new rabbit connection.
The last thing to do, we need to add some new environment variables. You can get the .env information from step #1, and you can use port 5672 as a default port. And don’t forget to change your default QUEUE_CONNECTION to rabbitmq. Of course, we can change it back programmatically inside our project if needed.
Just remember, you must do all the setups in both of your projects. Let’s say, in this project, I’ll use Service 1 that will act as a producer/publisher that publishes the message, and Service 2 as a consumer/receiver.
That’s all we need to do. Let’s see how it works in the next step.
#4 Dispatching PingJob
First, we’re gonna build our first ping job. This job does nothing, we need this to test the connection between the publisher and receiver. Let’s create a PingJob on both projects. Wait… what? Why are we creating the PingJob in both places? I’ll explain it below, you’ll more understand how this package works after some trials.
php artisan make:job PingJob
Open the App\Jobs\PingJob.php on Service 1 (Publisher) and inside it, there’s nothing. Move on.
Open the App\Jobs\PingJob on Service 2 (Receiver) and type this simple command.
What the PingJob on Service 2 does is simply echo it out if there’s a PingJob class being sent/dispatched. Of course, we’ll dispatch it from Service 1.
Let’s Test it. Open both of your projects, and because Service 2 is the receiver, we can type this in the terminal to make the project listen to any incoming messages/events.
php artisan rabbitmq:consume
After Service 2 is listening, then from Service 1 (Publisher), you can publish the message by dispatching the job.
Because we cannot dispatch the PingJob from the command line/terminal, we need to create a custom command that will dispatch the PingJob. If you already know how to use tinker, you can use that.
Then for dispatching the PingJob, we can use the command that we just built.
php artisan ping:job
#5 Dispatching UserCreated Job
Dispatching PingJob is simple, we don’t even need to pass any data. Now, we’re going to dispatch a job with the data inside. So, we can do something with the data inside the receiver. Let’s create a new UserCreated job on both of our projects. Let’s say we have a scenario when a new user is being created, we’ll dispatch this job.
php artisan make:job UserCreated
On Service 1 (Publisher), we can accept the data inside the constructor.
Inside Service 2 (Receiver), we’ll do something with that data. For now, let’s just echo it out.
For dispatching the job from Service 1, we need to pass the data in the form of an array. I assume you have a User record in your database.
The question is, why is it in an array? Because if we pass in the form of an object, let’s say the User class where the User ID is 4, but the receiver doesn’t have that records inside the database, then it’ll throw an error because the data can’t be unserialized (receiver doesn’t have the record). It’s the same thing if we pass an object that the receiver doesn’t have that class/object, it can’t be unserialized either. It’s safe to pass data in the form of an array.
I’ll create another command to easily dispatch the UserCreated event.
So far so good. Right now, we can send some data between our microservices.
#6 Several Improvements
1. Custom Receiver Handler
You may have been noticed, that when we create a job, we must create in both places, the publisher and the receiver. The problem is, when we’re dispatching a job, that may not exists in the receiver, then the receiver can throw an error. The reason? As you might guess because the receiver doesn’t have that object/class.
Fortunately, there’s a solution for it. If you see again in the config/queue.php on the options rabbitmq section, there you can specify your own job class. So, any messages/events that come to your receiver, it’ll be handled by your own custom class.
See more explanations in the documentation here on how to create your custom RabbitMQ Job, or you can see CustomHandleJob in my Github below.
2. Custom Queue
You may be thinking, what if we’ve 5 microservices that listen to the same queue, won’t it be chaos? We want to publish a message for a specific queue or a single microservice, not all the microservices. We got you cover.
First, make sure you have created a custom queue on your Cloud AMQP.
If you want to specify for a specific microservice to target a single queue, for example, a custom queue, then add this new environment variable.
Then your project/receiver will only listen to the message that is being pushed into the custom queue. Then, how we send the message from the publisher, for the specific custom queue? Simple, you can use the onQueue(‘queuename’) method.
3. Dockerize RabbitMQ
In my example above, there’s a delay between the publisher and the receiver. In a production environment, we want to use our own server for the RabbitMQ service. The easiest thing to do is using docker. I assume you have docker installed on your local machine.
docker run --rm -it --hostname my-rabbit -p 15672:15672 -p 5672:5672 rabbitmq:3-management
Then after it’s running, open your localhost http://localhost:15672
The username and password will be the same, guest. Don’t forget to create a new queue in the RabbitMQ panel. By default, the package we’re using uses the default queue name.
Because it’s running in your local machine, the RabbitMQ environment will change become something like this.
That’s it! So we can have our internal communication between microservices using RabbitMQ. Of course, there are several things to improve to be able to use it in production, especially for the security process, to make sure that everything is secure. I hope this article can help you to start with RabbitMQ in your Laravel project. Thanks for reading, and see you next time.
Microservices - why use RabbitMQ? - CloudAMQP
Today's monolithic systems are being replaced at a rapid pace by microservice architecture. To understand why this is…
Laravel is a web application framework with expressive, elegant syntax. We believe development must be an enjoyable and…