Centralized logging in .NET Core using Graylog and Serilog

What’s centralized logging and why do we need it?

Usually, applications are started from monolithic architecture and are moving towards microservices as load increases. When this happens we are starting to experience that we no longer can easily observe what’s happening in the system. That’s happening because logs are scattered between different services and it’s becoming harder and harder to track log messages and their flow. However, to solve this problem we can leverage a centralized logging solution, which will allow us to search through all the logs in one place.

There are many tools to build centralized logging solutions like Graylog, ELK stack, Splunk, etc. In this tutorial, we are going to create a centralized logging solution using Graylog logging server and Serilog library for .NET Core applications.

1. Let’s convert our sample application to use Serilog.

First, we need to install a few NuGet packages to our sample application:
1. Serilog
2. Serilog.AspNetCore
3. Serilog.Extensions.Hosting
4. Serilog.Sinks.Console

After the successful installation of these packages, we can configure Serilog to use console logger through appsettings.json file:

{
  "Serilog": {
    "Using": [ "Serilog.Sinks.Console" ],
    "MinimumLevel": "Information",
    "WriteTo": [
      { "Name": "Console" }
    ],
    "Application": "Centralized logging application"
  },
  "AllowedHosts": "*"
}

After we have finished writing the configuration for Serilog we need to read it in our application. We could read this configuration and configure Serilog through code manually, but instead, we will take advantage of Serilog.Settings.Configuration NuGet package, which will do everything for us.

Finally, we need to clear logging providers that were added by default by using WebHost.CreateDefaultBuilder(args) static method and use Serilog ones.

public class Program
{
  public static void Main(string[] args)
  {
    CreateWebHostBuilder(args).Build().Run();
  }
  public static IWebHostBuilder CreateWebHostBuilder(string[] args)
  {
    WebHost.CreateDefaultBuilder(args)
    .UseStartup<Startup>()
    .ConfigureLogging(loggingConfiguration =>
      loggingConfiguration.ClearProviders())
    .UseSerilog((hostingContext, loggerConfiguration) =>
      loggerConfiguration.ReadFrom
      .Configuration(hostingContext.Configuration));
  }
}

Lets’ try running our sample application and hitting the default api/values endpoint to see some logs.

First few logs using Serilog console logger

2. Using Graylog as a centralized logging server

Graylog provides a variety of input options, but we are going to use GELF as logging driver and push log messages through UDP. For that, we will open global GELF input on port 12201. After finishing configuring GELF input we will see a view similar to this:

Graylog inputs configuration window

Because we are switching logging from console to centralized logging server we need to change the configuration as well. Since we are going to push logs to Graylog server we will use the Serilog.Sinks.Graylog NuGet package instead of Serilog.Sinks.Console. After all, configuration changes our appsettings.json file looks like this:

"Serilog": {
    "Using": [ "Serilog.Sinks.Graylog" ],
    "MinimumLevel": "Information",
    "WriteTo": [
      {
        "Name": "Graylog",
        "Args": {
          "hostnameOrAddress": "127.0.0.1",
          "port": "12201",
          "transportType": "Udp"
        }
      }
    ],
    "Properties": {
      "Application": "Centralized logging application"
    }
  },
  "AllowedHosts": "*"
}

Finally, lets’ try hitting our endpoint in our sample application again to start exploring logs in Graylog UI.

Initial Graylog window for searching through a bunch of logs

In the Graylog search window, we can see a preview of each log message, the columns in the table can be adjusted to meet your needs, but it’s easier to view full log message in log message view, just like on below.

Detailed view of log message in Graylog

3. Conclusion

By using a centralized logging solution we can easily search through all the collected logs and store them in one place. This can help to increase the observability of the system and even get some insights about what’s happening in the system. Furthermore, we can leverage various Graylog functionalities like dashboards to have a quick overview of the system’s state or use alerts to inform us, when something bad happens in the system.

Code used in this article can be at https://github.com/Skisas/LoggingExample