Running multiple processes asynchronously with Laravel

Running multiple processes in background

RadicalLoop
Nov 30 · 3 min read

In PHP 7.x, now we have phthread extension available to create a new thread and divide your long task in multi thread to complete it faster.

But what if you don’t have phthread extension on your server?

You will find your question answered here!

You can spawn multi processes, also know as parallel processes to complete a long task. For example, you want to send bulk 100k+ emails from application, this is the best case where you can use multi processing with PHP.

There is a component(library) “The Process Component” in Symfony which you can utilised in your PHP project. In Laravel, it already comes with the Symfony component.

You may argue here that Laravel has jobs to do it but it has timeout issue and you should not insert 100k+ records in jobs table. Plus, each job connects/disconnects MySQL connection which is not cost effective.

Let’s see how can we do it!!

Creating Process

Suppose, you want to start background process from your controller, you can simply create the instance of Process class.

$process = new Process('php ' . base_path('artisan') . ' task:long-running-task &');$process->setTimeout(0);$process->run();

Note, I have used run method which is synchronous and blocking. But here we want it asynchronous. You could use start method instead of run method for asynchronous but that process might kill immediately as soon as your controller method sends response to the browser and your task could not have been completed.

Let me explain, how start method works. As soon as start() method is executed, it starts the command in background but when your main process (which is your controller) sends the response, it kills its sub-processes too (as main process considered to be finished) and that is why background process kill before your task is completed.

Importance of “&”

If you have noticed, I have written “&” behind the command.

‘php ‘ . base_path(‘artisan’) . ‘ task:long-running-task &

It plays important role here. & starts your command as a background process (as daemon). It is considered to be an individual separate process and run that command in background.

Now, when controller’s method is done and sends response to the browser, it would not affect that daemon/background process.💪

Now your command is the Main process and you can spawn multiple processes from that command to divide your task in multiple processes. Have a look below.

for ($i = 0; $i < $numberOfProcess; $i++) {     $process = new Process('php ' . base_path('artisan') . " task {$i}");
$process->setTimeout(0);
$process->disableOutput(); $process->start(); $processes[] = $process;
}

As you can see, I have used start() method here to create sub-processes which are asynchronous and make sure this script (main process) not get exited until sub-processes are done with their tasks. Main process must wait for sub-processes to complete their tasks.

How you can wait? Check below code.

for ($i = 0; $i < $numberOfProcess; $i++) { 
... // above code
}
// wait for above processes to completewhile (count($processes)) { foreach ($processes as $i => $runningProcess) { // specific process is finished, so we remove it if (! $runningProcess->isRunning()) { unset($processes[$i]); } sleep(1);
}
}
//Executes after all processes are done with their task

By the way, we have sent 100k+ emails in less than a minute by SendGrid API using the above technique. We have spawned only 3 sub-processes. 🚀

Conclusion
In short, your Main process must not get completed or killed in order to keep alive your child processes otherwise your child processes gets killed too.

Let us know if you have any query. We would love to help. Hope you enjoy reading this post!! 😊

If you enjoyed reading this blog post, please click on the clap button below! ❤️👇

Follow RadicalLoop on Twitter, Facebook. 🚀

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade