Laravel 5.6 Custom Monolog channel to store logs in MariaDB
Recently I had to implement a custom Monolog Handler for a Laravel 5.6 API. I couldn’t find anything similar on the internet so decided to build a custom one and write about it.
The problem
Using Laravel built-in logger facade, and store the logs in a database
Why?
At least because the client requirements. If you ask me, I would not recommend to store such kind of data in a relational database.
The challenge
- Implementation of a custom Monolog Handler
- Separate
exception reporter
logger
Solution
Implementation of a custom Monolog
channel, which will write all logs in a SQL table, except Laravel exceptions (which should be stored in a .log
file)
migration
First, we need to migrate the table where, in the future, we’ll store all our custom logs. This is a basic migration, with some sort of custom fields, which matches the project needs:
Schema::create('logs', function (Blueprint $t) {
$t->increments('id');
$t->text('description')->nullable();
$t->string('origin', 200)->nullable();
$t->enum('type', ['log', 'store', 'change', 'delete']);
$t->enum('result', ['success', 'neutral', 'failure']);
$t->enum('level', ['emergency', 'alert', 'critical', 'error', 'warning', 'notice', 'info', 'debug']);
$t->string('token', 100)->nullable();
$t->ipAddress('ip');
$t->string('user_agent', 200)->nullable();
$t->string('session', 100)->nullable();
$t->timestamps();
});
channel config
By default Laravel uses the stack
channel for the logging. To change that, we should add a new channel to the configuration file config/logging.php
:
'channels' => [
'custom' => [
'driver' => 'custom',
'via' => \App\Services\Logs\LogMonolog::class,
],
],
Next step is to change the default logging channel to our custom one, this can be done in the .env
file, by adding the config cons:
LOG_CHANNEL=custom
It is important to understand that after we’ll have this custom
channel we have control over Monolog’s instantiation and configuration, and all logs triggered via the facade Illuminate\Support\Facades\Log
will go through your LogMonolog
class that will create your Monolog instance.
monolog instance
The class which was declared in the channel custom — via
property only needs a single method: __invoke
, which should return the Monolog instance:
namespace App\Services\Logs;use Monolog\Logger;class LogMonolog
{
/**
* Create a custom Monolog instance.
*
* @param array $config
* @return \Monolog\Logger
*/
public function __invoke(array $config)
{
$logger = new Logger('custom');
$logger->pushHandler(new LogHandler());
$logger->pushProcessor(new LogProcessor()); return $logger;
}
}
In the magic__invoke
function, we return a Monolog instance with with channel name custom
(passed as a string param to the Logger
constructor). As we can notice, on the class instance, we call two functions, to attach a LogHandler
and a LogProcessor
.
LogHandler
Monolog provides many built-in handlers. Handler is basically a class, which is pushed in a stack
, and whenever you add a new log (a new record to the logger eg: '\Log::info('Hey')
), it traverses the handler stack.
namespace App\Services\Logs;use App\Events\Logs\LogMonologEvent;
use Monolog\Logger;
use Monolog\Handler\AbstractProcessingHandler;class LogHandler extends AbstractProcessingHandler
{
public function __construct($level = Logger::DEBUG)
{
parent::__construct($level);
} protected function write(array $record)
{
// Simple store implementation$log = new Log();
$log->fill($record['formatted']);
$log->save();// Queue implementation// event(new LogMonologEvent($record));
} /**
* {@inheritDoc}
*/
protected function getDefaultFormatter()
{
return new LogFormatter();
}
}
Since to create a custom Handler, we have to implement the Monolog\Handler\HandlerInterface
, we’re extending the abstract class provided by the Monolog, AbstractProcessingHandler
to keep things DRY.
As you can see, we have two implemented function in this class:
getDefaultFormatter
is used by theMonolog\Handler\AbstractHandler
to retrieve the formatter for the current custom Handler, we will come back to theformatter
in few lineswrite
is called from theAbstractProcessingHandler@handler
in order to process somehow the log, usually write it in a file, but we’ll write it in the DB. Basicallywrite
function will receive all information about the log. We can right here trigger the insert operation in the DB, but we will use more complex architecture, in order to maintain this operation async in a laravel queue.
LogProcessor
This is the class where we can extend the default fields for the log entry with some extra information. The single requirement for this class, is to have the __invoke
function:
namespace App\Services\Logs;class LogProcessor
{
public function __invoke(array $record)
{
$record['extra'] = [
'user_id' => auth()->user() ? auth()->user()->id : NULL,
'origin' => request()->headers->get('origin'),
'ip' => request()->server('REMOTE_ADDR'),
'user_agent' => request()->server('HTTP_USER_AGENT')
]; return $record;
}
}
Let’s dive a bit deeper in what we have until this point in the the $record
argument:
As you can see, we can send some custom fields right in the Log::info
and will receive those in a context
field from the $records
object
LogFormatter
After we added some extra
fields to our log, the log goes through other transformations, if you remember we had a function App\Services\Logs\LogHandler@getDefaultFormatter
which returns an instance of LogFormatter
, this is the third important operator of the Monolog. It has few formatters built-in which can be extend or used as a reference for our:
namespace App\Services\Logs;use Monolog\Formatter\NormalizerFormatter;class LogFormatter extends NormalizerFormatter
{
/**
* type
*/
const LOG = 'log';
const STORE = 'store';
const CHANGE = 'change';
const DELETE = 'delete';
/**
* result
*/
const SUCCESS = 'success';
const NEUTRAL = 'neutral';
const FAILURE = 'failure'; public function __construct()
{
parent::__construct();
} /**
* {@inheritdoc}
*/
public function format(array $record)
{
$record = parent::format($record); return $this->getDocument($record);
} /**
* Convert a log message into an MariaDB Log entity
* @param array $record
* @return array
*/
protected function getDocument(array $record)
{
$fills = $record['extra'];
$fills['level'] = str()->lower($record['level_name']);
$fills['description'] = $record['message'];
$fills['token'] = str_random(30); $context = $record['context'];
if (!empty($context)) {
$fills['type'] = array_has($context, 'type') ? $context['type'] : self::LOG;
$fills['result'] = array_has($context, 'result') ? $context['result'] : self::NEUTRAL; $fills = array_merge($record['context'], $fills);
} return $fills;
}
}
format
if the function which mutate all log records. This function receives as argument $record
, inclusively includes extra data from the LogProcessor.
The getDocument
method will adapt our records and context according with table structure, and basically will return the object with a new formatted
array which can fill the table entry:
Simply save in the DB
At this step we are done with everything related to Monolog custom channel. We can simply store the formatted object like this:
//App\Services\Logs\LogHandlerprotected function write(array $record)
{
$log = new Log(); // Log is the model over table (see bellow)
$log->fill($record['formatted']);
$log->save();
}
We should to be aware about which custom fields we’re sending through the context
object, because, if those are not in the table, the SQL will throw an error like this:
This exception is easy to avoid if we will parse every field from the $formatted
array (before we call fill
), and exclude those from the $record['formatted'].
Or we can go further and implement the Event-Listener.
//App\Services\Logs\LogHandlerprotected function write(array $record)
{
event(new LogMonologEvent($record));
}
LogMonologEvent
In order to perform an action in queue, we can emit an event. The event we emit looks like bellow, it basically will contain all log records:
namespace App\Events\Logs;use Illuminate\Queue\SerializesModels;class LogMonologEvent
{
use SerializesModels; /**
* @var
*/
public $records; /**
* @param $model
*/
public function __construct(array $records)
{
$this->records = $records;
}
}
Register listener
To listen for an event, it is necessary to register listener in the EventServiceProvider:
protected $subscribe = [
\Exenzo\Listeners\LogMonologEventListener::class,
];
LogMonologEventListener
Listener has the responsibility to store the log in the database. To have an event in a laravel queue it is enough to implement the ShouldQueue
interface:
namespace App\Listeners;use App\Events\Logs\LogMonologEvent;
use App\Models\Log;
use Illuminate\Contracts\Queue\ShouldQueue;class LogMonologEventListener implements ShouldQueue
{ public $queue = 'logs';
protected $log; public function __construct(Log $log) {
$this->log = $log;
} /**
* @param $event
*/
public function onLog($event)
{
$log = new $this->log;
$log->fill($event->records['formatted']);
$log->save();
} /**
* Register the listeners for the subscriber.
*
* @param \Illuminate\Events\Dispatcher $events
*/
public function subscribe($events)
{
$events->listen(
LogMonologEvent::class,
'Exenzo\Listeners\LogMonologEventListener@onLog'
);
}
}
And model implementation:
namespace App\Models;class Log extends \Illuminate\Database\Eloquent\Model
{
/**
* @var string $table
*/
protected $table = 'logs'; /**
* @var array $guarded
*/
protected $guarded = ['id'];}
Separate reporter logs
Laravel has a report
function in App\Exceptions\Handler
used to log exceptions or send them to an external service like Bugsnag or Sentry. It will be enough to manually choose the channel
where we want to redirect exception logs (because by default those will be stored in the DB):
public function report(Exception $exception)
{
if ($this->shouldntReport($exception)) {
return;
} Log::channel('daily')->error(
$exception->getMessage(),
array_merge($this->context(), ['exception' => $exception])
);
}
Conclusions
Let’s resume what we basically have, going step by step:
- we write a log by using
\Log
facade, and we want to keep it in the DB - we add in the the
config/logging.php
channel
array, a new custom entry which manage aMonolog
instance - we modify the
.env
to say that default channel should becustom
- we create an instance of
Monolog\Logger\Logger
and passed our custom Handler and Processor - when the
\Log::info('message')
is called, the log records are passed to theProcessor
(which add some custom extra fields), after that to theFormatter
which formats fields according with table structure, after that it is passed to thewrite
method which triggers theevent
with formatted log - we created an event and a queueable listener to store formatted log in the DB
- we adjusted the
App\Exceptions\Handler@report
method, to write exceptions in another log channel (daily
)