Structured Logging in .NET 8 application using Serilog

In this blog we will see how we can configure structured logging for application in a flexible and modern way using Serilog library. In our previous blog we have seen how logging is important pillar in observability of an application. For reading more on observability, please read below blog:

Unlocking Insight: Exploring the Three Pillars of Observability in Your .NET Application

Image created by Author

Before we delve into Serilog, let us understand what structured logging is, how is it different than traditional plain text logging.

What is Structured logging?

Structured logging in .NET involves the practice of logging events and messages in a format that is more organized and easily parseable than traditional plain text logging. Instead of logging messages as free-form text, structured logging formats the log data as a set of key-value pairs, often in formats like JSON or key-value pairs.

In plain text logging log messages will be logged as shown below, it does not give any information like log time or certain format so that we can search based on in future.

Before we dig deep into differences between structured logging and plain text logging, let us add Serilog and see how the log messages will be modified. For adding Serilog execute below command:

dotnet add package Serilog.AspNetCore

Add below lines of code in your program.cs

builder.Services.AddSerilog(options =>
{
    options.MinimumLevel.Information()
           .WriteTo.Console(new JsonFormatter(), LogEventLevel.Debug);
});

Then when you the application you will see below output in your console.

Let us take one message and see how Serilog added additional details and formatted the message.

{
  "Timestamp": "2024-03-16T09:21:55.1386236-05:00",
  "Level": "Information",
  "MessageTemplate": "Manager created resident with Id: 1",
  "TraceId": "7ff1a5728b5c34048caa53c0f8af56d8",
  "SpanId": "cf8de73ceb0fd319",
  "Properties": {
    "SourceContext": "ResidentApi.BusinessLogic.Manager.ResidentManager",
    "ActionId": "ce8f43d2-8a76-4ea3-ad86-ad4cffbe9cf9",
    "ActionName": "ResidentAPI.Controllers.ResidentController.CreateResident (ResidentAPI)",
    "RequestId": "0HN25OHE66P2I:00000004",
    "RequestPath": "/api/Resident",
    "ConnectionId": "0HN25OHE66P2I"
  }
}

As we discussed in observability it added TraceId and SpanId as well as Timestamp. We can further enhance the output by adding more Serilog sinks and changing message format. Now the log message is formatted we can easily send this to ELK and search them. We will cover sinks and writing to ELK and searching in separate blog but now let’s delve into differences between plain text logging and structured logging.

Plain Text Logging Advantages

• Simple Implementation — Plain text logging is straightforward to implement. You just need to format the log message as a string.
• Familiarity — Developers are generally accustomed to writing and reading plain text log messages.
• Quick Debugging — It’s easy to add ad-hoc log messages for quick debugging during development.
• Low Overhead — Generating plain text log messages usually incurs lower performance overhead compared to structured logging.

Plain Text Logging Disadvantages

• Limited Searchability — Log messages are just strings, making it difficult to search and filter based on specific criteria. You would need to parse strings to extract relevant data.
• Lack of Context — Plain text logs might lack context, especially when trying to understand the state of the application during a specific event.
• Non-standard Formats — Each log message can have a different format, making it harder to standardize log analysis.

Structure Logging Advantages

• Search and Filtering — Since each log event is structured, it becomes much easier to search through logs based on specific properties. You can filter logs based on any property within the structured data.
• Contextual Information — Structured logs provide context-rich information about the logged event. You can include objects, exceptions, and other data relevant to the event.
• Readability — Logs in a structured format are easier to read, especially for developers and tools. Log entries are self-describing, making it clear what each piece of information represents.
• Integration with Tools — Structured logs integrate well with various logging backends and log analysis tools. This includes ELK Stack (Elasticsearch, Logstash, Kibana), Seq, Splunk, etc.
• Consistency — Using structured logging encourages a consistent log format across an application or system. Team members can easily understand and work with logs from different parts of the application.
• Flexibility — You can easily add, remove, or modify properties in the structured logs. This flexibility allows developers to adapt logs to changing requirements without altering logging code extensively.
• Performance — Depending on the logging library and configuration, structured logging can be performant. Libraries like Serilog are designed to efficiently handle structured data serialization.

Structure Logging Disadvantages

• Complexity — Implementing structured logging might require a bit more setup and configuration compared to plain text logging.
• Learning Curve — Developers not familiar with structured logging formats might need some time to get used to working with them.

In this blog we have seen advantages of structured logging and simple setup using Serilog, in future blogs let us see how we can set up Serilog to write to different sources and how to enhance messages using Serilog sinks and even try to setup ELK instance and search logs.

Source code be found below:

GitHub - CheyPenmetsa/SerilogSample

🙏Thanks for taking the time to read the article. If you found it helpful and would like to show support, please consider:

1. 👏👏👏👏👏👏Clap for the story and bookmark for future reference
2. Follow me on Chaitanya (Chey) Penmetsa for more content
3. Stay connected on LinkedIn.

Wishing you a happy learning journey 📈, and I look forward to sharing new articles with you soon.