A simple guide to properly queuing mail in Laravel

There aren’t many useful guides out there that provide a simple, straight-forward answer how to use Laravel queues to send e-mails in batches. And if you’re in the same boat, here’s the lowdown.

What’s a queue?

A queue allows you to defer the processing of tasks like sending e-mails or firing API requests until you’re ready to deal with them. In practice, you can queue anything you like; but it’s primarily designed for anything that would either bring your server to a crawl if you were to do all processing at once or where rate-limiting would otherwise take effect.

What confuses developers about queues is just how intrinsically simple they are: it’s a queue and nothing more. Whether or not a task is run straight away is not the job of the queue to know (no pun intended), but rather the task (or job) you are dispatching to a queue.

Creating jobs

A job is the item of logic you want dispatched to a given queue (it’s the task you want to run — like sending e-mails). When you dispatch a job, you can define whether there is a delay between the time a job is dispatched and the time on which it is executed.

To create a new job, run:

$ php artisan make:job <name>

By default, job classes are generated and stored in app/Jobs. If the directory doesn’t exist, it will be created for you.

If the job needs data to be passed into it when it’s dispatched, specify that as required arguments within the constructor of the job class like this:

use App\User;
use Illuminate\Mail\Mailable;
/**
*
@var User
*/
protected $user;

/**
*
@var Mailable
*/
protected $mail;

/**
* Create a new job instance.
*
*
@param User $user
*
@param Mailable $mail
*/
public function __construct(User $user, Mailable $mail)
{
$this->user = $user;
$this->mail = $mail;
}

In my case, I’m creating a SendEmail job to throttle how many e-mails can be sent at any one time; hence why I require a Mailable object to be passed in.

Dispatching jobs

Now that we’ve defined the job that we want to run, we need to dispatch it.

In my case, I might dispatch the SendEmail job within a hypothetical RegisterController where I need to send a verification e-mail to a newly registered user. Here’s how I’d dispatch it:

use App\Http\Request;
use App\Jobs\SendEmail;
use App\Mail\VerifyEmail;
/**
* Store a newly created resource in storage.
*
*
@param Request $request
*
@return \Illuminate\Http\RedirectResponse
*
@throws \Symfony\Component\HttpKernel\Exception\HttpException
*/
public function store(Request $request)
{
SendEmail::dispatch($user, new VerifyEmail($user));
}

As you can see, the required constructor arguments from the job class are passed into the dispatch() static method. This method is actually a variadic function which passes the arguments into the constructor of the SendEmail job class using the func_get_args() function.

Delaying the job execution

Of course, this entire exercise is pointless if I can’t delay when the e-mail is sent. The problem here is that if I were to simply set a delay of now()->addSeconds(10) and queue multiple jobs at once, all of these queued tasks will all run at once in roughly 10 seconds because each job would have been queued at the same time, give or take a few milliseconds.

So how do we address that? Well, we need to know the time on which the last queued e-mail is scheduled for execution. We can then use that timestamp to generate a Carbon instance and add ten seconds onto it for the subsequent job we want to queue.

Within the class or controller where the job is being dispatched, let’s cache the delay time and use that for each subsequent queue entry:

use App\Http\Request;
use App\Jobs\SendEmail;
use App\Mail\VerifyEmail;
use Carbon\Carbon;

/**
* Store a newly created resource in storage.
*
*
@param Request $request
*
@return \Illuminate\Http\RedirectResponse
*
@throws \Symfony\Component\HttpKernel\Exception\HttpException
*/
public function store(Request $request)
{
$baseDelay = json_encode(now());

$getDelay = json_encode(
cache('jobs.' . SendEmail::class, $baseDelay)
);

$setDelay = Carbon::parse(
$getDelay->date
)->addSeconds(10);

cache([
'jobs.' . SendEmail::class => json_encode($setDelay)
], 5);
    SendEmail::dispatch($user, new VerifyEmail($user))
->delay($setDelayTime);
}

When retrieving a value from the cache() function, the second argument allows a default to be returned if nothing is found.

Passing an array into the cache() function sets the value for the given key for the specified period of time, in my case five minutes.

Using the Mail facade

I should point out that you can also pass the delay time to the Mail facade directly, which circumvents the need for the SendEmail job class entirely:

Mail::to($user)->later($setDelayTime);

This is ideal if you don’t need the job class to perform any other logic before sending the e-mail. Just bear in mind you won’t be able to use Rate Limiting as an alternative to delayed dispatching if you choose this approach.

Tip: If your queued jobs still run immediately after being dispatched, you’re likely using the sync driver which is designed to do just that. You need to use the database or Redis queue driver to support delayed dispatching.

Alternative approach with Rate Limiting

If caching the delay timestamp isn’t your cup of tea, you can simplify this whole process if you’re happy to use Redis to store the queued tasks. You’ll then be able to take advantage of Rate Limiting. It’s so easy, it hurts.

Within the class or controller where the job is being dispatched, just dispatch the job without delaying it and then wrap the logic contained within the handle() method of the job class within a call to Redis::throttle():

use Illuminate\Support\Facades\Mail;
use Illuminate\Support\Facades\Redis;
/**
* Execute the job.
*
*
@return void
*/
public function handle()
{
Redis::throttle('SendEmail')
->allow(1)
->every(10)
->then(function () {
Mail::to($this->user)->send($this->mail);
}, function () {
return $this->release(10);
});
}

Here, we are allowing one email to be sent every ten seconds. The string SendEmail passed to the throttle() method is a name that uniquely identifies the type of job being rate-limited. You can set this to whatever you want.

The release() method is an inherited member of your job class and specifies the maximum number of times a failed job can be released back onto the queue before it fails.

When the job is dispatched to the queue, Redis is instructed to only run one SendEmail job every ten seconds.

Using Redis with Laravel

Redis is a highly-efficient open source key-value store that you can not only use to queue jobs really efficiently in Laravel, but also store broadcast channels if your app has ‘real-time’ functionality, such as collaboration tools like Sprint Boards.

To use Redis within your Laravel app, you need to install the predis/predis Composer package. There are indeed other Redis clients out there but this one has built-in driver support in Laravel due to its popularity.

If you’re running a Homestead VM or a server configured by Laravel Forge, you’ll already have the Redis server installed. If not, follow this guide from DigitalOcean on how to install it (they have separate guides for CentOS 7, Debian 9, Ubuntu 14.04, Ubuntu 16.04 and Ubuntu 18.04).

If you’re on a Mac and using Laravel Valet, install Redis using Homebrew:

$ brew install redis
$ brew services start redis

Remember, predis/predis is just the Redis PHP client. You can always quickly determine whether Redis is installed on your box by running redis-cli.

Setting the queue driver to Redis

Once you’ve got Redis up and running (bonus if it’s already installed!), you just need to set the queue driver in your .env file:

QUEUE_CONNECTION=redis

Now that was easy. Of course, if you know the host or port differs from the defaults, set those too:

QUEUE_CONNECTION=redis
REDIS_HOST=<host>
REDIS_PORT=<port>

Otherwise, get those workers running!

Running the queue worker

Now that we have our jobs queued, we need to run them. Of course, because we are delaying the execution of each job, we know that even though the queue worker will continue to run in the background on a perpetual basis, our jobs will run at the interval we expect:

$ php artisan queue:work

If you want the worker process to quit when the queue is empty, pass the flag:

$ php artisan queue:work --stop-when-empty

Automatically restarting the queue worker

So what happens if the queue worker stops running? Your jobs will start queuing up, of course. So you’ll want to make sure the queue worker can automatically be restarted if that happens.

You can use a process monitor like supervisor to accomplish this and the Laravel documentation explains the process of installing and configuring it.

However, if your server is being managed by Laravel Forge, supervisor will already be installed and you can configure it to monitor a queue worker from within the Forge front-end directly. You’ll find the Queue tab within the sidebar after navigating to your site within Forge.