Event Loops

Multi-Process PHP

The post is going to be a bit of a mixed bag. I’ve been working on a parallel-execution library, and I want to look at how it can be used with an event loop. Before that though, I want to look at a few parallel-execution libraries intended to be used with my two favourite reactive component libraries: Icicle and ReactPHP.

Doorman, Icicle and ReactPHP are all rather unstable (at least the parts you’re going to see today). The subject is new to PHP as a whole. I don’t want to talk about using other languages right now. Save that discussion for when it matters.

Icicle

Aaron Piotrowski and Stephen Coakley have put quite a bit of work into Icicle’s concurrency library recently. It supports two different approaches for parallel execution: forks and threads. For the purposes of our discussion, they operate in a very similar way.

I’m not going to cover the basics of Icicle or event loops here. You should have a basic understanding of what event loops do, and you should be aware that Icicle uses generators heavily to implement coroutines.

Forking

You can fork processes, in Icicle, with code resembling the following:

use Icicle\Concurrent\Forking\ForkContext;
use Icicle\Loop\Events\TimerInterface;
 
Icicle\Coroutine\create(function () {
    $fork = new ForkContext(function () {
        for ($i = 0; $i < 5; $i++) {
            yield $this->send($i);
            sleep(1);
        }
    });
 
    $fork->start();
 
    /** @var TimerInterface $timer */
    $timer = Loop\periodic(0.1, function () {
        print ".";
    });
 
    for ($i = 0; $i < 5; $i++) {
        print (string) (yield $fork->receive());
    }
 
    $timer->stop();
});
 
Icicle\Loop\run();

To fork processes, you need to have the PCNTL extension installed and have PHP compiled with Shared Memory support. There is no easy way to compile with Shared Memory support, using Homebrew, so you’ll have to use a VM.

The underlying code combines a coroutine implementation with methods like pcntl_fork(), to split processing off from the process in which the ForkContext was created. I like how clean it is, resembling the Doorman implementation. There are a few drawbacks to using PCNTL, but they are hidden behind a nice interface.

Threading

You can split processing over multiple threads, with almost exactly the same code. Just swap ForkContext with ThreadContext and the code will change from using PCNTL to using Pthreads.

Pthreads is an extensions which allows multi-threading in PHP. In addition to compiling PHP with Shared Memory support, you’ll need to compile it with thread safety before you can enable the Pthreads extension.

Pthreads is awesome, but it comes with some heavy requirements. Enabling thread safety will mean disabling many popular PHP extensions. You’ll also need to know what you’re doing. Threading is hard!

ReactPHP

ReactPHP has been around for quite some time. It also implements an event loop, and has a huge collection of asynchronous components to go with it.

One of these components is Child Process. It uses a similar approach to Doorman, but with an API that feels familiar to ReactPHP users:

$loop = React\EventLoop\Factory::create();
 
$process = new React\ChildProcess\Process("echo foo");
 
$process->on("exit", function ($code, $signal) {
    print "done";
});
 
$loop->addTimer(0.001, function ($timer) use ($process) {
    $process->start($timer->getLoop());
 
    $process->stdout->on("data", function ($output) {
        print "data: " . $output;
    });
});
 
$loop->run();

The event loop stuff loops similar to Icicle, but interface for deferred code is far less refined. There is no indication of how that deferred work should be executed (speaking of PHP code that you expect to execute in a worker).

You can simply shell out and react to STDOUT and STDERR events. You have to think about how you want that stuff to work. On the other hand, the requirements for installing and running ReactPHP scripts are significantly fewer. All you need is PHP 5.4. If you have Event/Libev extensions installed then ReactPHP will perform more efficiently.

Doorman

So where does that leave us? The multi-process methods we’ve looked at in previous posts have little to do with an event loop. We’ve used infinite loops and tick() methods to have some sort of cyclic, parallel execution model.

Well, it so happens that tick() methods work well with event loop timers. Let’s look at an example:

use AsyncPHP\Doorman\Manager\ProcessManager;
use AsyncPHP\Doorman\Task\ProcessCallbackTask;
 
$manager = new ProcessManager();
 
$task = new ProcessCallbackTask(function () {
    for ($i = 0; $i < 5; $i++) {
        print $i;
        sleep(1);
    }
});
 
$manager->addTask($task);
 
Icicle\Loop\periodic(0.1, function () {
    print ".";
});
 
Icicle\Loop\periodic(0.1, function () use ($manager) {
    if (!$manager->tick()) {
        Icicle\Loop\stop();
    }
});
 
Icicle\Loop\run();

So far, so good! We can use Doorman alongside Icicle, but this doesn’t give us the bi-directional communication we had with ForkContext and ThreadContext. For that, we need to add Remit:

use AsyncPHP\Doorman\Manager\ProcessManager;
use AsyncPHP\Doorman\Task\ProcessCallbackTask;
use AsyncPHP\Remit\Client\ZeroMqClient;
use AsyncPHP\Remit\Location\InMemoryLocation;
use AsyncPHP\Remit\Server\ZeroMqServer;
 
$server = new ZeroMqServer(
    new InMemoryLocation("127.0.0.1", 5555)
);
 
$server->addListener("data", function ($data) {
    print (string) $data;
});
 
$manager = new ProcessManager();
 
$task = new ProcessCallbackTask(function () {
    $client = new ZeroMqClient(
        new InMemoryLocation("127.0.0.1", 5555)
    );
 
    for ($i = 0; $i < 5; $i++) {
        $client->emit("data", [$i]);
        sleep(1);
    }
});
 
$manager->addTask($task);
 
Icicle\Loop\periodic(0.1, function () {
    print ".";
});
 
Icicle\Loop\periodic(0.1, function () use ($manager, $server) {
    $server->tick();
     
    if (!$manager->tick()) {
        Icicle\Loop\stop();
    }
});
 
Icicle\Loop\run();

Ok, so that’s loads more code for something Icicle did in less. And if you want this kind of thing and can use Icicle then you definitely should! Just wait until a stable version is tagged and have at it.

We definitely have a ways to go before components like Doorman and Remit are as easy to use as Icicle…

ReactPHP Pooling

Before you go, let me tell you about something else exciting…

Wyrihaximus has been working on a child process pooling component for ReactPHP. It uses similar concepts to Doorman, but is built specifically to look and feel like the other ReactPHP components:

use React\ChildProcess\Process;
use WyriHaximus\React\ChildProcess\Messenger\Messages\Factory;
use WyriHaximus\React\ChildProcess\Pool\FlexiblePool;
 
$loop = React\EventLoop\Factory::create();
 
$pool = new FlexiblePool(
    new Process("exec php child.php"),
    $loop,
    [
        "min_size" => 1,
        "max_size" => 100,
    ]
);
 
$pool->on("message", function ($message) {
    print (string) $message;
});
 
for ($i = 0; $i < 5; $i++) {
    $pool->rpc(MessageFactory::rpc("ping", [
        "i" => $i,
    ]))->then(function ($data) {
        print (string) $data;
    });
}
 
$pool->terminate(MessageFactory::message([
    "woeufh209h838392",
]));
 
$loop->run();

Cees-Jan has implemented a multi-process messenger which allows an RPC style of communication between parent and child processes. This, combined with the Child Process component allows the spawning and management of multiple child processes, and bi-directional communication. Child scripts resemble the following:

use React\EventLoop\LoopInterface;
use React\Promise\Deferred;
use WyriHaximus\React\ChildProcess\Messenger\Factory;
use WyriHaximus\React\ChildProcess\Messenger\Messenger;
 
$loop = React\EventLoop\Factory::create();
 
$recipient = Factory::child($loop);
 
$recipient->on("message", function ($payload, $messenger) {
    /** @var Messenger $messenger */
    $messenger->write(json_encode([
        "type"    => "message",
        "payload" => $payload,
    ]));
 
    /** @var LoopInterface $loop */
    $loop = $messenger->getLoop();
 
    $loop->addTimer(1, function () use ($loop) {
        $loop->stop();
    });
});
 
$recipient->registerRpc("ping", function ($payload, $deferred) {
    /** @var Deferred $deferred */
    $deferred->resolve([
        "result" => "server pinged with " . $payload["i"],
    ]);
});
 
$loop->run();

I’m very excited for the future of these two component libraries. They’re putting parallel execution models in the hands of PHP developers, with increasingly simple interfaces.

What’s this async PHP nonsense?! Yell at me on Twitter...