Logging Layers: A Guide to Structured and Consistent Logging

Last month, I had the opportunity to deliver a lightning talk at AmsterdamPHP on the topic of logging layers. The positive feedback received inspired me to share this information more widely through an article.

The Importance of Logging Structure

In smaller teams, logging structures may vary, and developers can adopt different approaches. However, as a team scales, maintaining a robust and consistent logging layer becomes crucial. This not only aids development teams but also ensures clarity for other departments that rely on log data without direct access to the codebase.

Consider the example of logging a client’s IP address. Various options exist, such as:

• src: 10.42.42.42
• client_ip: 10.42.42.42
• apache2.access.remote_ip: 10.42.42.42
• context.user.ip: 10.42.42.42
• src_ip: 10.42.42.42

There is no definitive right or wrong answer; what matters is establishing a standardized and contracted way to log data across all platforms, for instance, client.ip: 10.42.42.42.

The Challenge and Solution

If the goal is clear, why does achieving this consistency remain a challenge? The key lies in preparing a documented structure and ensuring its widespread adoption within the team. Naming variables, as humorously noted in the first programming joke we’ve heard, is indeed a challenging aspect of programming. It’s a time-consuming process that demands diverse knowledge to devise a standard approach across various architectural domains.

To address this, Elastic, a well-known company, introduced the Elastic Common Schema (ECS), an open-source specification developed with community support. ECS defines field names, datatypes, and usage examples, providing a well-documented guide here.

ECS Fields Documentation

Your Gateway to PHP

While ECS offers a solution, implementing it from scratch in PHP requires effort. The available PHP SDK for ECS covers only a limited set of fields.

ECS fields covered by the SDK

Recognizing this gap, PECS (PHP Elastic Common Schema) was created to bring full ECS support into the PHP world.

Abstraction and Expectations

Before diving into the package, let’s establish what makes a good abstraction. Drawing inspiration from “The Effective Engineer” by Edmond Lau, a good abstraction should be:

• Easy to learn
• Easy to use without extensive documentation
• Hard to misuse
• Powerful enough to meet requirements
• Easily extendable
• Appropriate for the intended audience

PECS strives to fulfill these criteria. In this package, ECS specifications are fully transformed into JSON configurations, enabling a built-in generator to create ECS PHP classes. The classes are fully type-hinted with both native types and defined type classes and enums, minimizing the risk of misuse. The package is designed to be easily extendable, allowing the instantiation of fields and rendering them in the expected ECS format.

This is an example of creating ECS fields in a collection:

(new EcsFieldsCollection([
    new Log(
        filePath: 'app/Http/Controllers/Controller.php',
        level: 'info',
        logger: 'name',
        originFileLine: 42,
        originFileName: 'Controller.php',
        originFunction: 'index',
    ),
    new Client(
        ip: '10.42.42.42',
        geo: new Geo(
            cityName: 'Amsterdam',
            continentCode: 'EU',
            continentName: 'Europe',
            countryIsoCode: 'NL',
            countryName: 'Netherlands',
            location: new GeoPoint(52.37403, 4.88969),
        ),
        user: new User(
            id: 'e125a612-899e-11ee-b9d1-0242ac120002',
            name: 'hamid',
            roles: (new ValueList())->push('admin')->push('user'),
        )
    ),
    new Os(
        kernel: '4.19.0-6-amd64',
        name: 'Arch',
        platform: 'x86_64',
        type: OsType::LINUX
    ),
]))->render();

and here is the output which illustrates the result of rendering the specified fields:

{
    "log": {
        "file": {
            "path": "app\/Http\/Controllers\/Controller.php"
        },
        "level": "info",
        "logger": "name",
        "origin": {
            "file": {
                "line": 42,
                "name": "Controller.php"
            },
            "function": "index"
        }
    },
    "client": {
        "ip": "10.42.42.42",
        "geo": {
            "city_name": "Amsterdam",
            "continent_code": "EU",
            "continent_name": "Europe",
            "country_iso_code": "NL",
            "country_name": "Netherlands",
            "location": {
                "lat": 52.37403,
                "lon": 4.88969
            }
        },
        "user": {
            "id": "e125a612-899e-11ee-b9d1-0242ac120002",
            "name": "hamid",
            "roles": [
                "admin",
                "user"
            ]
        }
    }
}

Integration

Recognizing that pure PHP is not the norm these days, PECS is listed as a third-party package under Monolog, the most popular PHP logging package.

$log->pushHandler($handler->setFormatter(new EcsFormatter()));

$log->info('message', [
    new Event(action: 'test event'),
]);

This integration allows seamless use with Symfony and Laravel, both of which utilize Monolog under the hood.

Conclusion

In conclusion, PECS provides a comprehensive solution for implementing the Elastic Common Schema in PHP, ensuring structured and consistent logging across diverse platforms. While this article provides an overview, a follow-up article will dive into customizing PECS with practical examples. In the meantime, refer to the documentation for a detailed walkthrough covering various topics.