Extend Laravel Queues

One of the things I love about Laravel is how much complexity is hidden behind comparatively simple interfaces. There are some super complex components (database, queues, filesystem) which are so easy to use.

This week a friend asked how to extend queues, with custom functionality, and my answer to him was that it’s complicated. Indeed, unless you’ve done it yourself or learned from someone else who has, it can be complicated. So here’s a simple guide!

Understanding Queues

The general structure of (the more sophisticated) Laravel components include:

1. Configuration settings in app/config.
2. A facade class.
3. A manager class.
4. Several driver (sometimes called connector) classes.
5. At least one service provider, though often more.

When you’re using a queue, the facade resolves a specific connector (set in config) and runs driver specific methods, generally conforming to an interface. These interfaces, drivers and managers are bound to the IoC container in the service providers.

Let’s assume we’ll be using the beanstalkd driver. We’ll have a configuration file which includes:

return [
  "default"     => "beanstalkd",
  "connections" => [
    "beanstalkd" => [
      "driver" => "beanstalkd",
      "host"   => "localhost",
      "queue"  => "default",
      "ttr"    => 60
    ]
  ]
];

This is from app/config/queue.php

Laravel provides the Queue facade, which resolves to a manager class:

"Queue" => "Illuminate\Support\Facades\Queue"

This is from app/config/app.php

protected static function getFacadeAccessor() {
  return "queue";
}

This is from vendor/laravel/framework/src/Illuminate/Support/Facades/
Queue.php

This looks like a dead-end, but it’s not. Let’s head back to where the facades are loaded:

"providers" => [
  ...
  "Illuminate\Queue\QueueServiceProvider",
  ...
]

This is from app/config/app.php

This service provider binds queue, in the IoC container:

protected function registerManager()
{
  $this->app->bindShared("queue", function($app)
  {
    $manager = new QueueManager($app);
    $this->registerConnectors($manager);
    return $manager;
  });
}

This is from vendor/laravel/framework/src/Illuminate/Queue/
QueueServiceProvider.php

After an intermediate method (the registerConnectors() method), the Beanstalkd connector is also registered:

protected function registerBeanstalkdConnector($manager)
{
  $manager->addConnector("beanstalkd", function()
  {
    return new BeanstalkdConnector;
  });
}

This is from vendor/laravel/framework/src/Illuminate/Queue/
QueueServiceProvider.php

Finally, when a method is called against the Queue facade, the QueueManager will dispatch it to the relevant connector:

public function __call($method, $parameters)
{
 $callable = [$this->connection(), $method]; 
 return call_user_func_array($callable, $parameters);
}

This is from vendor/laravel/framework/src/Illuminate/Queue/
QueueManager.php

Changing Queues

There are two practical ways to extend queues, and the approach you use will be determined largely by what you want to do:

1. Add methods/properties on a subclass of QueueManager. This is the class the Queue facade talks to, and the class you would otherwise inject as a dependency when not using facades (http://laravel.com/docs/
   facades#facade-class-reference). You would use this approach if the functionality you were adding is not specific to a particular engine, and concerns the general operation of queues in your application.
2. Add methods/properties on a subclass of BeanstalkdQueue. This is for when you want to add functionality which is specific to Beanstalkd, or which is guaranteed to change based on the connector being used.

Extending QueueManager

To extend QueueManager, all you need to do is replace the queue IoC binding with a QueueManager subclass of your own:

$this->app->bindShared("queue", function($app)
{
  $manager = new CustomQueueManager($app);
  $this->registerConnectors($manager);
  return $manager;
});

You’ll also need to register the connectors all over again. The previous connector registration was against the original queue binding, so this new queue binding won’t be aware of them.

You can then add new features to CustomQueueManager, and everything else will continue to operate as usual (provided you don’t break any of the expected functionality of the QueueManager.

Extending BeanstalkdQueue

To extend BeanstalkdQueue; you need to override the Beanstalkd-Connector binding (in QueueManager) and override the Beanstalkd-Queue binding (in BeanstalkdConnector). It sounds more complicated than it really is:

$manager = $this->app["queue"];
 
$manager->addConnector("beanstalkd", function()
{
  return new CustomBeanstalkdConnector;
});
class CustomBeanstalkdConnector implements ConnectorInterface
{
  public function connect(array $config)
  {
    $pheanstalk = new Pheanstalk(
      $config["host"]
    );
  
    return new CustomBeanstalkdQueue(
      $pheanstalk,
      $config["queue"],
      array_get(
        $config, "ttr", Pheanstalk::DEFAULT_TTR
      )
    );
  }
}

You can then add any additional functionality to the CustomBeanstalkd-
Queue class, and it will be available (in the case of methods) in the Queue facade.

Extending Others

A similar method of deduction and replacement can be used to help you extend other complex Laravel components. Just follow the path through each class and replace bindings until the class you’re calling (facade or injected) is the one with your modifications!