PHP Xdebug proxy: when Xdebug’s standard capabilities are insufficient

Published in

Bumble Tech

11 min readNov 27, 2019

Xdebug is often used for debugging PHP programs. However, IDE and Xdebug’s standard capabilities are not always sufficient. Some, but not all, problems can be resolved using the Xdebug proxy, pydbgpproxy. This is why I have created a PHP Xdebug proxy based on the asynchronous framework, amphp.

In this article, I will be looking at what is wrong with pydbgpproxy, what it is missing and why I decided not to rework it. I will also be explaining how PHP Xdebug proxy works and using an example to show how it can be extended.

Pydbgpproxy vs PHP Xdebug proxy

Xdebug proxy is an intermediate service between IDE and Xdebug (it proxies queries from Xdebug to IDE and vice versa). Generally, it is used for multiuser debugging. This is when you have one web server and several developers.

Pydbgpproxy is usually used as a proxy. However, it has several problems:

There is no official page.
Locating where to download it from is difficult but you can do so here. Click on the link named Python (sic!) Remote Debugging Client.
I have not been able to find an official repository.
With no repository, it is unclear where to send a pull request.
The proxy, as you can see from its name, is written in Python which not all PHP programmers know, making extending it a problem.
Continuing on from the previous point, if there is PHP code which needs to be used in a proxy, then you will need to port it to Python, and duplicating code is never a really good idea.

A search for a Xdebug proxy written in PHP did not yield any results on both GitHub and the internet. So, I wrote a PHP Xdebug proxy. I have done so based on the asynchronous framework amphp.

Here are some of the main advantages of PHP Xdebug proxy over pydbgpproxy:

PHP Xdebug proxy is written in a language well-known to PHP programmers and this has the following implications:
- It is easier to resolve problems which arise
- It is easier to extend.
PHP Xdebug proxy has a public repository and that means:
- You can fork and fix it to suit your needs
- You can send a pull request with a feature that is lacking or with a solution to a problem.

How to work with PHP Xdebug proxy

Installation

PHP Xdebug proxy can be installed as a dev dependency via composer:

composer.phar require mougrim/php-xdebug-proxy --dev

If, however, you want to avoid introducing unnecessary dependencies into your project, then PHP Xdebug proxy can be installed as a project using the same composer:

composer.phar create-project mougrim/php-xdebug-proxy
cd php-xdebug-proxy

PHP Xdebug proxy is extendable but, by default, in order to work, ext-dom (the extension is enabled by default) is required for parsing XML and amphp/log for asynchronous writing to logs:

composer.phar require amphp/log '^1.0.0'

Running the proxy

The proxy is run as follows:

bin/xdebug-proxy

Proxy will run using the default settings:

Using config path /path/to/php-xdebug-proxy/config
[2019-02-14 10:46:24] xdebug-proxy.NOTICE: Use default ide: 127.0.0.1:9000 array ( ) array ( )
[2019-02-14 10:46:24] xdebug-proxy.NOTICE: Use predefined ides array (   'predefinedIdeList' => array (     'idekey' => '127.0.0.1:9000',   ), ) array ( )
[2019-02-14 10:46:24] xdebug-proxy.NOTICE: [Proxy][IdeRegistration] Listening for new connections on '127.0.0.1:9001'... array ( ) array ( )
[2019-02-14 10:46:24] xdebug-proxy.NOTICE: [Proxy][Xdebug] Listening for new connections on '127.0.0.1:9002'... array ( ) array ( )

You can see from the log that by default the proxy:

Listens to 127.0.0.1:9001 for IDE registration connections;
Listens to 127.0.0.1:9002 for Xdebug connections;
Uses 127.0.0.1:9000 as IDE by default and pre-installed IDE as idekey.

Configuration

If you want to set up listening ports etc. you can specify the path from the folder with the settings. All you need to do is copy the config folder:

cp -r /path/to/php-xdebug-proxy/config /your/custom/path

There are three files in the folder with the config:

config.php:

<?php
return [
    'xdebugServer' => [
        // host:port for listening for Xdebug connections
        'listen' => '127.0.0.1:9002',
    ],
    'ideServer' => [
        // If the proxy cannot find the IDE, then it will
        // use the default IDE, if you need to switch off
        // the default IDE, you need to pass an empty line.
        // Default IDE is useful when only one person
        // is using the proxy.
        'defaultIde' => '127.0.0.1:9000',
        // The predefined IDEs are given using the format
        // 'idekey' => 'host:port', if predefined IDEs
        // are not required, you can specify an empty array.
        // Predefined IDEs are useful when proxy users
        // don’t change often, that way they don’t have
        // to re-register each time they rerun proxy.
        'predefinedIdeList' => [
            'idekey' => '127.0.0.1:9000',
        ],
    ],
    'ideRegistrationServer' => [
        // host:port for listening for connections to IDE
        // registrations, if IDE registrations need
        // to be switched off, you need to pass an empty line.
        'listen' => '127.0.0.1:9001',
    ],
];

logger.php: the logger can be configured; the file must return an object which is a \Psr\Log\LoggerInterface instance, by default \Monolog\Logger with \Amp\Log\StreamHandler is used (for non-blocking write). Outputs logs to stdout;
factory.php: classes used in the proxy may be configured; the file must return an object which is an instance of Factory\Factory. By default Factory\DefaultFactory is used.

After copying the files, you can edit them and run the proxy:

bin/xdebug-proxy --configs=/your/custom/path/config

Debugging

Lots of articles have been written on how to debug code using Xdebug so I would just like to highlight the main points.

In php.ini the following settings should be in the [xdebug] section (correct them if they differ from the standard settings):

idekey=idekey
remote_host=127.0.0.1
remote_port=9002
remote_enable=On
remote_autostart=On
remote_connect_back=Off

You can then run the PHP code to be debugged:

php /path/to/your/script.php

If you have done everything right, debugging will start from the first breakpoint in IDE. Debugging in php-fpm mode by several developers falls outside the scope of the present article, but here is an example.

Extending of proxy functions

All that we have considered above, to a greater or lesser extent pydbgpproxy can also do.

Now let’s talk about the most interesting thing in PHP Xdebug proxy. The proxy can be extended using your factory (it’s created in factory.php in the configuration, see above). The factory must implement the Factory\Factory interface.

The most powerful are what are called ‘request preparers’. They can alter requests from Xdebug to IDE and vice versa. In order to add a request preparer, you need to redefine the Factory\DefaultFactory::createRequestPreparers() method. The method returns an array of objects which set up the RequestPreparer\RequestPreparer interface. When proxying a request from Xdebug to IDE they are performed in the original order; when proxying a request from IDE to Xdebug they are performed in reverse order.

Request preparers may, for example, be used to alter paths to the files (at breakpoints and in executable files).

Debugging rewritten files

To give you an example of a preparer, I need to go off on a slight tangent. In unit-tests we use soft-mocks. Soft-mocks allow you to replace functions, static methods, constants etc. in tests, and they are an alternative to runkit and uopz. They work by rewriting PHP files on the go. AspectMock also works in a similar way.

The standard capabilities of Xdebug and IDE, however, allow you to debug the rewritten files (which have a different path), but not the original files.

Let’s consider in more detail the problem of debugging using soft-mocks in tests. For a start, let’s take the example of when the PHP code is executed locally.

The first complications present themselves at the stage when breakpoints are set. In IDE these are set to original files, and not to rewritten files. In order to set a breakpoint via IDE, you have to find the most recent rewritten file. The problem is worsened by the fact that with each change to the original file a new rewritten file is created, that is to say that for each unique content in the file there will be a unique rewritten file.

This problem can be resolved by calling the xdebug_break() function which is analogous to setting a breakpoint. In such case it is no longer necessary to perform a search for a rewritten file.

Now let’s consider a more complicated situation: an application being run on a remote machine.

In this case the folder with the rewritten files can be mounted, for example, via SSHFS. If the local and remote paths to the folder are different from one another, then the mappings to IDE also need to be defined.

One way or another, this approach differs a little from the usual one and only allows rewritten files to be debugged, but not original files. We, however, want to edit and debug the same original files.

AspectMock gets round the problem by switching on the debugging mode with no option of switching it off:

public function init(array $options = [])
{
    if (!isset($options['excludePaths'])) {
        $options['excludePaths'] = [];
    }
    $options['debug'] = true;
    $options['excludePaths'][] = __DIR__;    parent::init($options);
}

In this simple example test, the debugging mode would operate about 20 percent slower. But without a sufficient number of tests on AspectMock I cannot give a more precise assessment of how much slower it is. If you have lots of tests on AspectMock, I would be pleased if you could share a comparison in the comments.

Using Xdebug with soft-mocks

Now that we have understood the problem, let’s see how we can solve it using PHP Xdebug proxy. The main part is to be found in the RequestPreparer\SoftMocksRequestPreparer class.

In the class constructor we define the path to the soft-mocks initialisation script and run it (it is assumed that soft-mocks has been connected as a dependency, but you can pass any path to the constructor):

public function __construct(LoggerInterface $logger, string $initScript = '')
{
    $this->logger = $logger;
    if (!$initScript) {
        $possibleInitScriptPaths = [
            // proxy is installed as a project,
            // soft-mocks as a dependency of the project
            __DIR__.'/../../vendor/badoo/soft-mocks/src/init_with_composer.php',
            // proxy and soft-mocks set up as dependencies
            __DIR__.'/../../../../badoo/soft-mocks/src/init_with_composer.php',
        ];
        foreach ($possibleInitScriptPaths as $possiblInitScriptPath) {
            if (file_exists($possiblInitScriptPath)) {
                $initScript = $possiblInitScriptPath;                break;
            }
        }
    }    if (!$initScript) {
        throw new Error("Can't find soft-mocks init script");
    }
    // initialising soft-mocks (path to the folder
    // with rewritten files etc.)
    require $initScript;
}

In order to prepare a request from Xdebug to IDE you need to replace the path to the rewritten file with the path to the original file:

public function prepareRequestToIde(XmlDocument $xmlRequest, string $rawRequest): void
{
    $context = [
        'request' => $rawRequest,
    ];
    $root = $xmlRequest->getRoot();
    if (!$root) {
        return;
    }
    foreach ($root->getChildren() as $child) {
        // path to rewritten file is to be found
        // in one of the tags:
        // - 'stack': https://xdebug.org/docs-dbgp.php#stack-get
        // - 'xdebug:message': https://xdebug.org/docs-dbgp.php#error-notification
        if (!in_array($child->getName(), ['stack', 'xdebug:message'], true)) {
            continue;
        }
        $attributes = $child->getAttributes();
        if (isset($attributes['filename'])) {
            // if the path to the rewritten file
            // is in the tag’s attributes,
            // we replace it with the original path
            $filename = $this->getOriginalFilePath($attributes['filename'], $context);
            if ($attributes['filename'] !== $filename) {
                $this->logger->info("Change '{$attributes['filename']}' to '{$filename}'", $context);
                $child->addAttribute('filename', $filename);
            }
        }
    }
}

To prepare a request from IDE to Xdebug we need to replace the path to the original file with the path to the rewritten file:

public function prepareRequestToXdebug(string $request, CommandToXdebugParser $commandToXdebugParser): string
{
    // parses request into command and arguments
    [$command, $arguments] = $commandToXdebugParser->parseCommand($request);
    $context = [
        'request' => $request,
        'arguments' => $arguments,
    ];
    if ($command === 'breakpoint_set') {
        // if there is an argument, -f, then we replace
        // the path to the original file with the path
        // to the rewritten file
        // see https://xdebug.org/docs-dbgp.php#id3
        if (isset($arguments['-f'])) {
            $file = $this->getRewrittenFilePath($arguments['-f'], $context);
            if ($file) {
                $this->logger->info("Change '{$arguments['-f']}' to '{$file}'", $context);
                $arguments['-f'] = $file;
                // assemble request back
                $request = $commandToXdebugParser->buildCommand($command, $arguments);
            }
        } else {
            $this->logger->error("Command {$command} is without argument '-f'", $context);
        }
    }    return $request;
}

In order for the request preparer to start working, you need to create your own factory class and either extend it from Factory\DefaultFactory, or implement the Factory\Factory interface. For soft-mocks the Factory\SoftMocksFactory factory looks like this:

class SoftMocksFactory extends DefaultFactory
{
    public function createConfig(array $config): Config
    {
        // here we create an object for our config class
        return new SoftMocksConfig($config);
    }    public function createRequestPreparers(LoggerInterface $logger, Config $config): array
    {
        $requestPreparers = parent::createRequestPreparers($logger, $config);
        return array_merge($requestPreparers, [$this->createSoftMocksRequestPreparer($logger, $config)]);
    }    public function createSoftMocksRequestPreparer(LoggerInterface $logger, SoftMocksConfig $config): SoftMocksRequestPreparer
    {
        // this is where we pass the path to the init script from the config
        return new SoftMocksRequestPreparer($logger, $config->getSoftMocks()->getInitScript());
    }
}

You need your own config class here in order to be able to specify the path to the soft-mocks init script. You can see what it looks like here Config\SoftMocksConfig.

There is one little thing left to do: create a new factory and specify the path to the soft-mocks init script. You can see how to do this by looking at softMocksConfig.

Non-blocking API

As I said above, PHP Xdebug proxy is based on amphp, and that means that if you want to work with I/O you have to use non-blocking API. apmphp already has quite a few components which implement this non-blocking API. If you are planning to extend PHP Xdebug proxy and use it in a multi-user mode, then make sure you use non-blocking API.

Conclusions

PHP Xdebug proxy is still quite a young project, but at Badoo we are already using it quite actively for debugging tests using soft-mocks.

PHP Xdebug proxy:

replaces pydbgpproxy for multi-user debugging
work with soft-mocks
is able to extend:
- paths to the files originating from IDE and Xdebug can be replaced
- statistics can be collected: in debugging mode the executable context, at least, is accessible when debugging (values of variables and executable line of code).

If you use Xdebug proxy for something other than multiuser debugging, then in the comments please share your example and the Xdebug proxy you use.

If you use pydbgpproxy or any other Xdebug proxy, try PHP Xdebug proxy, tell us about any problems you experience, and share pull requests. Let’s develop this project together! :)

P.S. Thanks go to my colleague, Eugene Makhrov, aka eZH, for the smdbgpproxy idea!