Laravel Microservice Communication using RabbitMQ Message Broker

Cerwyn Cahyono
May 13 · 8 min read
Laravel + Rabbit MQ

Dealing with big website projects is cumbersome when we’re using monolith architecture. As a Laravel developer, it’s quite challenging, because basically, Laravel offers us a full-stack development where you can work with Front-end as well as Back-end in the same project, in other words, monolithic application. The question is, can we develop microservices using Laravel (or maybe Lumen) for a bigger project? Of course, we can.

Before we go on, what are the bad things about developing apps using monolithic architecture? Well, because all the functionalities are inside in one project, means that every time we make any changes (even a small one) we’ll re-deploy our entire app, which may affect the whole app. With the big-scale project, it’ll much harder to maintain. So, microservices architecture comes to help, where we can separate its functionality into a standalone component.

A message broker acts as a middleman for the microservices, receiving messages from one application (producers) and handing them over to others (consumers) to do the job. — https://www.cloudamqp.com

When we’re dealing with microservices, we need to prepare something for our microservice internal communications. Based on this article, at least there are 3 ways to communicate:
1) REST APIs
2) Remote Procedure Calls (RPC)
3) Brokers (RabbitMQ, Apache Kafka, etc)

This article will show you how to use RabbitMQ for our microservice communication using Laravel applications. The good thing about using RabbitMQ is that it’ll queue our messages. Let’s say, the receiver is busy or disconnected right now, then the message will be stored inside the temporary RabbitMQ Storage and will push the message again whenever the consumer/receiver is ready.

Content Overview

  • Our Goals
  • Cloud AMQP
  • Laravel Project Setup
  • Dispatching PingJob
  • Dispatching UserCreated Job
  • Several Improvements

#1 Our Goals

Let’s start with the result of what we’re going to build in this article. We’ll need to prepare 2 services. One of them will act as a consumer/receiver, where it’ll listen to any incoming messages. The other one will act as a producer/publisher that pushes some messages.

Image 1. Goals

As you can see, the right terminal is acting as a producer/publisher. First, it published the very standard ping job, and the second try published the user job where it’ll pass the user data into our receiver. As you might guess, when the receiver received some data (or user data in this case) we can do anything with the data inside the receiver, right?

If you notice, there’s a delay between sending a message and receiving it. That’s okay because in this demo I was using Cloud AMQP where the server is far from my country right now. But at the end of this article, in the improvements section, I’ll show you to set up your own RabbitMQ in your local with docker, and of course, it’ll be much faster.

#2 Cloud AMQP

There are two ways to use RabbitMQ. The first one, we can install RabbitMQ into our local machine, or the second one is using Cloud AMQP as a service, so we don’t need to install anything inside our local. For this article, the second option is pretty much simple to do. So let’s stick with that and do the registration first on this website https://www.cloudamqp.com.

After registration succeeds, then create a new instance and see the instance details. The information we need is the hosts, user & vhost, and the password. After we get what we need, let’s move on.

Image 2. RabbitMQ Credentials

#3 Laravel Project Setup

I assume you’ve installed two Laravel projects in your local. For using the RabbitMQ, we’ll use this package https://github.com/vyuldashev/laravel-queue-rabbitmq. So let’s install the package in both of our projects.

composer require vladimir-yuldashev/laravel-queue-rabbitmq

Then, open your config/queue.php file, then add a new rabbit connection.

config/queue.php

The last thing to do, we need to add some new environment variables. You can get the .env information from step #1, and you can use port 5672 as a default port. And don’t forget to change your default QUEUE_CONNECTION to rabbitmq. Of course, we can change it back programmatically inside our project if needed.

QUEUE_CONNECTION=rabbitmq
RABBITMQ_HOST=
RABBITMQ_PORT=5672
RABBITMQ_USER=
RABBITMQ_PASSWORD=
RABBITMQ_VHOST=

Just remember, you must do all the setups in both of your projects. Let’s say, in this project, I’ll use Service 1 that will act as a producer/publisher that publishes the message, and Service 2 as a consumer/receiver.

That’s all we need to do. Let’s see how it works in the next step.

#4 Dispatching PingJob

First, we’re gonna build our first ping job. This job does nothing, we need this to test the connection between the publisher and receiver. Let’s create a PingJob on both projects. Wait… what? Why are we creating the PingJob in both places? I’ll explain it below, you’ll more understand how this package works after some trials.

php artisan make:job PingJob

Open the App\Jobs\PingJob.php on Service 1 (Publisher) and inside it, there’s nothing. Move on.

App\Jobs\PingJob.php (Service 1 — Publisher)

Open the App\Jobs\PingJob on Service 2 (Receiver) and type this simple command.

App\Jobs\PingJob.php (Service 2— Receiver)

What the PingJob on Service 2 does is simply echo it out if there’s a PingJob class being sent/dispatched. Of course, we’ll dispatch it from Service 1.

Let’s Test it. Open both of your projects, and because Service 2 is the receiver, we can type this in the terminal to make the project listen to any incoming messages/events.

php artisan rabbitmq:consume

After Service 2 is listening, then from Service 1 (Publisher), you can publish the message by dispatching the job.

PingJob::dispatch();

Because we cannot dispatch the PingJob from the command line/terminal, we need to create a custom command that will dispatch the PingJob. If you already know how to use tinker, you can use that.

App\Commands\PingJobCommand.php

Then for dispatching the PingJob, we can use the command that we just built.

php artisan ping:job
Image 3. PingJob Test

#5 Dispatching UserCreated Job

Dispatching PingJob is simple, we don’t even need to pass any data. Now, we’re going to dispatch a job with the data inside. So, we can do something with the data inside the receiver. Let’s create a new UserCreated job on both of our projects. Let’s say we have a scenario when a new user is being created, we’ll dispatch this job.

php artisan make:job UserCreated 

On Service 1 (Publisher), we can accept the data inside the constructor.

App\Jobs\UserCreated.php (Service 1 Publisher)

Inside Service 2 (Receiver), we’ll do something with that data. For now, let’s just echo it out.

App\Jobs\UserCreated.php (Service 2 Receiver)

For dispatching the job from Service 1, we need to pass the data in the form of an array. I assume you have a User record in your database.

UserCreated::dispatch(User::inRandomOrder()->first()->toArray());

The question is, why is it in an array? Because if we pass in the form of an object, let’s say the User class where the User ID is 4, but the receiver doesn’t have that records inside the database, then it’ll throw an error because the data can’t be unserialized (receiver doesn’t have the record). It’s the same thing if we pass an object that the receiver doesn’t have that class/object, it can’t be unserialized either. It’s safe to pass data in the form of an array.

I’ll create another command to easily dispatch the UserCreated event.

App\Console\Commands\UserJobCommand.php (Service 1 Publishser)
Image 4. UserCreated Job Test

So far so good. Right now, we can send some data between our microservices.

#6 Several Improvements

1. Custom Receiver Handler

You may have been noticed, that when we create a job, we must create in both places, the publisher and the receiver. The problem is, when we’re dispatching a job, that may not exists in the receiver, then the receiver can throw an error. The reason? As you might guess because the receiver doesn’t have that object/class.

Fortunately, there’s a solution for it. If you see again in the config/queue.php on the options rabbitmq section, there you can specify your own job class. So, any messages/events that come to your receiver, it’ll be handled by your own custom class.

See more explanations in the documentation here on how to create your custom RabbitMQ Job, or you can see CustomHandleJob in my Github below.

2. Custom Queue

You may be thinking, what if we’ve 5 microservices that listen to the same queue, won’t it be chaos? We want to publish a message for a specific queue or a single microservice, not all the microservices. We got you cover.

First, make sure you have created a custom queue on your Cloud AMQP.

Image 5. Create a Custom Queue

If you want to specify for a specific microservice to target a single queue, for example, a custom queue, then add this new environment variable.

RABBITMQ_QUEUE=custom

Then your project/receiver will only listen to the message that is being pushed into the custom queue. Then, how we send the message from the publisher, for the specific custom queue? Simple, you can use the onQueue(‘queuename’) method.

PingJob::dispatch()->onQueue('custom');

3. Dockerize RabbitMQ

In my example above, there’s a delay between the publisher and the receiver. In a production environment, we want to use our own server for the RabbitMQ service. The easiest thing to do is using docker. I assume you have docker installed on your local machine.

docker run --rm -it --hostname my-rabbit -p 15672:15672 -p 5672:5672 rabbitmq:3-management

Then after it’s running, open your localhost http://localhost:15672

The username and password will be the same, guest. Don’t forget to create a new queue in the RabbitMQ panel. By default, the package we’re using uses the default queue name.

Because it’s running in your local machine, the RabbitMQ environment will change become something like this.

RABBITMQ_HOST=localhost
RABBITMQ_PORT=5672
RABBITMQ_USER=guest
RABBITMQ_PASSWORD=guest
RABBITMQ_VHOST=/

That’s it! So we can have our internal communication between microservices using RabbitMQ. Of course, there are several things to improve to be able to use it in production, especially for the security process, to make sure that everything is secure. I hope this article can help you to start with RabbitMQ in your Laravel project. Thanks for reading, and see you next time.

References

Geek Culture

Proud to geek out. Follow to join our +500K monthly readers.

Medium is an open platform where 170 million readers come to find insightful and dynamic thinking. Here, expert and undiscovered voices alike dive into the heart of any topic and bring new ideas to the surface. Learn more

Follow the writers, publications, and topics that matter to you, and you’ll see them on your homepage and in your inbox. Explore

If you have a story to tell, knowledge to share, or a perspective to offer — welcome home. It’s easy and free to post your thinking on any topic. Write on Medium

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store