Open in app

Sign in

Write

Sign in

Spring Boot 3 logging for monitoring

Nithidol Vacharotayan
8 min readJun 21, 2024

--

Image by freepik

In an application, logging is significant for the programmer to figure out what happens to an application when an error or incorrect business logic occurs. The developer must read the log to analyse and solve the problem. Spring Boot supports logging in various ways.

When an error occurs, the developer team is responsible for finding out what happened and, why this error occurred and finding a solution to solve it. One way to find out is to view logs. The logs will give the programming team some clues about the root cause of an error, which is why writing logs is essential to the programmer.

Using Log4j2

Log4j2 covers key features for logging and monitoring an application, such as filtering, rolling file appender, and logging to multiple destinations.

Add dependency in a pom.xml file.

<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-log4j2</artifactId>
</dependency>

When an error message shows “SLF4J(W): Class path contains multiple SLF4J providers.”. the developer should exclude “spring-boot-starter-logging” all of group id “org.springframework.boot” in a pom.xml file.

<dependency>
  <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-security</artifactId>
  <exclusions>
    <exclusion>
      <groupId>org.springframework.boot</groupId>
      <artifactId>spring-boot-starter-logging</artifactId>
    </exclusion>
  </exclusions>
</dependency>

Config application.properties.

logging.config=classpath:log4j2-spring.xml

Create a log4j2-spring.xml file in the resources directory.

<?xml version="1.0" encoding="UTF-8"?>
<Configuration status="WARN">
    <Appenders>
        <!-- Console Appender -->
        <Console name="Console" target="SYSTEM_OUT">
            <PatternLayout pattern="%d{yyyy-MM-dd HH:mm:ss} [%t] %-5level %logger{36} - %msg%n"/>
        </Console>

        <!-- Rolling File Appender -->
        <RollingFile name="File" fileName="logs/app.log"
                     filePattern="logs/app-%d{yyyy-MM-dd}-%i.log.gz">
            <PatternLayout>
                <pattern>%d{yyyy-MM-dd HH:mm:ss} [%t] %-5level %logger{36} - %msg%n</pattern>
            </PatternLayout>
            <Policies>
                <TimeBasedTriggeringPolicy />
                <SizeBasedTriggeringPolicy size="10MB"/>
            </Policies>
        </RollingFile>

        <!-- Async Appender -->
        <Async name="Async">
            <AppenderRef ref="Console"/>
            <AppenderRef ref="File"/>
        </Async>
    </Appenders>

    <Loggers>
        <!-- Root Logger -->
        <Root level="info">
            <AppenderRef ref="Async"/>
        </Root>

        <!-- Specific Logger -->
        <Logger name="com.example" level="debug" additivity="false">
            <AppenderRef ref="Console"/>
            <AppenderRef ref="File"/>
        </Logger>
    </Loggers>
</Configuration>

Read more about configuration log4j2. link

Create RESTful web services with write logs using log4j2.

import org.springframework.http.HttpStatus;
import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.PathVariable;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;
import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;

import java.util.List;

@RestController
@RequestMapping("/api/products")
public class ProductsController {

    private static final Logger logger = LogManager.getLogger(ProductsController.class);


    @GetMapping(path = "/{productId}", produces = "application/json")
    public ResponseEntity<Object> getUserprofile(@PathVariable String productId) {
        logger.info("Received product ID {}", productId);
        return new ResponseEntity<>(null, HttpStatus.OK);
    }

}

Test web services to view the logs in the console and log file.

curl http://localhost:8080/api/products/1
2024-06-17 16:45:09 [http-nio-8080-exec-1] INFO  com.example.demo.controller.products.ProductsController - Received product ID 1

The log4j2-spring.xml file configures the information shown in the logs.

Using Slf4j annotation

Lombok provides this annotation, and it automatically creates a “Logger” instance named “log” in the annotated class.

The developer can configure logs in the application.properties file or application.yml.

logging.level.root=INFO
logging.level.com.example.demo=DEBUG
logging.pattern.console=%d{yyyy-MM-dd HH:mm:ss} - %msg%n
logging:
  level:
    root: INFO
    com.example.demo: DEBUG
  pattern:
    console: "%d{yyyy-MM-dd HH:mm:ss} - %msg%n"

Read more about configuration logs. link

Create RESTful web services with write logs using Slf4j annotation.

import lombok.extern.slf4j.Slf4j;
import org.springframework.http.HttpStatus;
import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.PathVariable;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;

import java.util.List;

@RestController
@RequestMapping("/api/products")
@Slf4j
public class ProductsController {


    @GetMapping(path = "/{productId}", produces = "application/json")
    public ResponseEntity<Object> getUserprofile(@PathVariable String productId) {
        log.info("Received product ID {}", productId);
        log.debug("Received product ID {}", productId);
        log.error("This is an error logs.");
        return new ResponseEntity<>(null, HttpStatus.OK);
    }

}

Test web services to view the logs in the console.

curl http://localhost:8080/api/products/1
2024-06-17 17:18:29 - Received product ID 1
2024-06-17 17:18:29 - Received product ID 1
2024-06-17 17:18:29 - This is an error logs.

Configuration log with logback-spring.xml

The developer can configure the log level and pattern. Logback provides various features to support log-ins with specific requirements, such as logging in to standard out or writing into a log file.

Create logback-spring.xml.

<?xml version="1.0" encoding="UTF-8"?>
<configuration>

    <property name="LOG_FILE" value="${LOG_FILE:-logs/app.log}"/>

    <appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
        <encoder>
            <pattern>%d{yyyy-MM-dd HH:mm:ss} [%thread] %-5level %logger{36} - %msg%n</pattern>
        </encoder>
    </appender>

    <appender name="FILE" class="ch.qos.logback.core.rolling.RollingFileAppender">
        <file>${LOG_FILE}</file>
        <rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
            <fileNamePattern>logs/app.%d{yyyy-MM-dd}.log</fileNamePattern>
            <maxHistory>30</maxHistory>
        </rollingPolicy>
        <encoder>
            <pattern>%d{yyyy-MM-dd HH:mm:ss} [%thread] %-5level %logger{36} - %msg%n</pattern>
        </encoder>
    </appender>

    <root level="INFO">
        <appender-ref ref="STDOUT"/>
        <appender-ref ref="FILE"/>
    </root>

</configuration>

Test web services to view the logs in the console.

curl http://localhost:8080/api/products/1

The console and the app.log file will display the log written in the logs directory.

2024-06-19 14:56:17 [http-nio-8080-exec-1] INFO  c.e.d.c.products.ProductsController - Received product ID 1

The developer can manage the log as follows attribute.
Rolling policy: The developer can limit the history log to manage disk space.
Log Level: The developer can filter logs by specific log level.

Configure logs to write log files based on log level.

The developer can write log files by level for easy tracing or to reduce disk space.

Example Directory Structure for logs.

project-root/
└── logs/
    ├── error/
    │   └── error.log
    ├── trace/
    │   └── trace.log

import lombok.extern.slf4j.Slf4j;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.http.HttpStatus;
import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.PathVariable;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;

@RestController
@RequestMapping("/api/products")
@Slf4j
public class ProductsController {

    @GetMapping(path = "/{productId}", produces = "application/json")
    public ResponseEntity<Object> getUserprofile(@PathVariable String productId) {
        log.info("Example  info  message -> Received product ID {}", productId);
        log.debug("Example debug message -> Received product ID {}", productId);
        log.error("Example error message -> not found product ID {}", productId);
        return new ResponseEntity<>(null, HttpStatus.OK);
    }

}
<?xml version="1.0" encoding="UTF-8"?>
<configuration>

    <!-- Define properties for log file locations -->
    <property name="INFO_LOG" value="logs/trace/trace.log" />
    <property name="ERROR_LOG" value="logs/error/error.log" />

    <appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
        <encoder>
            <pattern>%d{yyyy-MM-dd HH:mm:ss} [%thread] %-5level %logger{36} - %msg%n</pattern>
        </encoder>
    </appender>

    <appender name="TRACE_FILE" class="ch.qos.logback.core.rolling.RollingFileAppender">
        <file>${INFO_LOG}</file>
        <rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
            <fileNamePattern>logs/trace/trace.%d{yyyy-MM-dd}.log</fileNamePattern>
            <maxHistory>30</maxHistory>
        </rollingPolicy>
        <encoder>
            <pattern>%d{yyyy-MM-dd HH:mm:ss} [%thread] %-5level %logger{36} - %msg%n</pattern>
        </encoder>
        <filter class="ch.qos.logback.classic.filter.LevelFilter">
            <level>INFO</level>
            <onMatch>ACCEPT</onMatch>
            <onMismatch>DENY</onMismatch>
        </filter>
    </appender>

    <appender name="ERROR_FILE" class="ch.qos.logback.core.rolling.RollingFileAppender">
        <file>${ERROR_LOG}</file>
        <rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
            <fileNamePattern>logs/error/error.%d{yyyy-MM-dd}.log</fileNamePattern>
            <maxHistory>30</maxHistory>
        </rollingPolicy>
        <encoder>
            <pattern>%d{yyyy-MM-dd HH:mm:ss} [%thread] %-5level %logger{36} - %msg%n</pattern>
        </encoder>
        <filter class="ch.qos.logback.classic.filter.LevelFilter">
            <level>ERROR</level>
            <onMatch>ACCEPT</onMatch>
            <onMismatch>DENY</onMismatch>
        </filter>
    </appender>

    <!-- Root logger configuration -->
    <root level="DEBUG">
        <appender-ref ref="STDOUT" />
        <appender-ref ref="TRACE_FILE" />
        <appender-ref ref="ERROR_FILE" />
     </root>

 </configuration>

The log will create trace.log and error.log, filters followed by config.

trace.log
2024-06-19 15:37:05 [http-nio-8080-exec-1] INFO  c.e.d.c.products.ProductsController - Example  info  message -> Received product ID 1
error.log
2024-06-19 15:37:05 [http-nio-8080-exec-1] ERROR c.e.d.c.products.ProductsController - Example error message -> not found product ID 1

Logback provides various configs that help the developer manage the log. In the production zone, the developer can write only an error log to trace a problem, reducing the log file’s size.

Read more about Logback config. link

Implement micro meter to Logback.

Micrometer provides “traceId” and “spanId” to help the developer trace the log more easily. The Logback provide rolling files by size and date.

Directory Structure

logs/
├── error/
│   ├── error.log
│   └── archived/
│       ├── error-2024-06-19.1.log
│       └── error-2024-06-20.2.log
└── trace/
    ├── trace.log
    └── archived/
        └── trace-2024-06-19.1.log

The developer team can design a log directory structure through logback-spring.xml for easy maintenance. For example, the developer can design a monthly log by creating a folder category by month in which the developer can delete or move unused monthly logs to the history drive. The developer can categorise directory structure into various categories such as hostname, service name, app name or functionality.

Add dependency

<dependency>
  <groupId>io.micrometer</groupId>
  <artifactId>micrometer-tracing-bridge-brave</artifactId>
</dependency>
<?xml version="1.0" encoding="UTF-8"?>
<configuration>

    <!-- Define properties for log file locations -->
    <property name="INFO_LOG" value="logs/trace/trace.log" />
    <property name="ERROR_LOG" value="logs/error/error.log" />

    <appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
        <encoder>
            <pattern>%d{yyyy-MM-dd HH:mm:ss} [%thread] %-5level %logger{36} - %msg%n traceId=%X{traceId} spanId=%X{spanId}%n</pattern>
        </encoder>
    </appender>

    <appender name="TRACE_FILE" class="ch.qos.logback.core.rolling.RollingFileAppender">
        <file>${INFO_LOG}</file>
        <rollingPolicy
                class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
            <fileNamePattern>logs/trace/archived/trace-%d{yyyy-MM-dd}.%i.log
            </fileNamePattern>
            <maxHistory>90</maxHistory>
            <timeBasedFileNamingAndTriggeringPolicy
                    class="ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP">
                <maxFileSize>10MB</maxFileSize>
            </timeBasedFileNamingAndTriggeringPolicy>
        </rollingPolicy>
        <encoder>
            <pattern>%d{yyyy-MM-dd HH:mm:ss} [%thread] %-5level %logger{36} - %msg%n traceId=%X{traceId} spanId=%X{spanId}%n</pattern>
        </encoder>
        <filter class="ch.qos.logback.classic.filter.LevelFilter">
            <level>INFO</level>
            <onMatch>ACCEPT</onMatch>
            <onMismatch>DENY</onMismatch>
        </filter>
    </appender>

    <appender name="ERROR_FILE" class="ch.qos.logback.core.rolling.RollingFileAppender">
        <file>${ERROR_LOG}</file>
        <rollingPolicy
                class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
            <fileNamePattern>logs/error/archived/error-%d{yyyy-MM-dd}.%i.log
            </fileNamePattern>
            <maxHistory>90</maxHistory>
            <timeBasedFileNamingAndTriggeringPolicy
                    class="ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP">
                <maxFileSize>10MB</maxFileSize>
            </timeBasedFileNamingAndTriggeringPolicy>
        </rollingPolicy>
        <encoder>
            <pattern>%d{yyyy-MM-dd HH:mm:ss} [%thread] %-5level %logger{36} - %msg%n traceId=%X{traceId} spanId=%X{spanId}%n</pattern>
        </encoder>
        <filter class="ch.qos.logback.classic.filter.LevelFilter">
            <level>ERROR</level>
            <onMatch>ACCEPT</onMatch>
            <onMismatch>DENY</onMismatch>
        </filter>
    </appender>

    <!-- Root logger configuration -->
    <root level="DEBUG">
        <appender-ref ref="STDOUT" />
        <appender-ref ref="TRACE_FILE" />
        <appender-ref ref="ERROR_FILE" />
     </root>



 </configuration>

The config pattern for the log will show “traceId” and “spanId.” When the file reaches 10 megabytes or the date has changed, it will be moved to the archived directory and renamed, followed by the name pattern config.

curl http://localhost:8080/api/products/1
2024-06-19 16:01:05 [http-nio-8080-exec-1] INFO  c.e.d.c.products.ProductsController - Example  info  message -> Received product ID 1
 traceId=66729e51f99292cf271b5f1c0d181b45 spanId=83ce4d742a661473
2024-06-19 16:01:05 [http-nio-8080-exec-1] DEBUG c.e.d.c.products.ProductsController - Example debug message -> Received product ID 1
 traceId=66729e51f99292cf271b5f1c0d181b45 spanId=83ce4d742a661473
2024-06-19 16:01:05 [http-nio-8080-exec-1] ERROR c.e.d.c.products.ProductsController - Example error message -> not found product ID 1
 traceId=66729e51f99292cf271b5f1c0d181b45 spanId=83ce4d742a661473

When the client requests multiple requests, “traceId” and “spanId” have changed for any requests.

Request 1

2024-06-19 16:05:20 [http-nio-8080-exec-2] INFO  c.e.d.c.products.ProductsController - Example  info  message -> Received product ID 1
 traceId=66729f5022d10422bed6169e7fd55dcf spanId=528d51969de2d7e1
2024-06-19 16:05:20 [http-nio-8080-exec-2] DEBUG c.e.d.c.products.ProductsController - Example debug message -> Received product ID 1
 traceId=66729f5022d10422bed6169e7fd55dcf spanId=528d51969de2d7e1
2024-06-19 16:05:20 [http-nio-8080-exec-2] ERROR c.e.d.c.products.ProductsController - Example error message -> not found product ID 1
 traceId=66729f5022d10422bed6169e7fd55dcf spanId=528d51969de2d7e1

Request 2

2024-06-19 16:07:53 [http-nio-8080-exec-4] INFO  c.e.d.c.products.ProductsController - Example  info  message -> Received product ID 1
 traceId=66729fe9307508232033e9e4e762fb33 spanId=f54ffdfa23696b3b
2024-06-19 16:07:53 [http-nio-8080-exec-4] DEBUG c.e.d.c.products.ProductsController - Example debug message -> Received product ID 1
 traceId=66729fe9307508232033e9e4e762fb33 spanId=f54ffdfa23696b3b
2024-06-19 16:07:53 [http-nio-8080-exec-4] ERROR c.e.d.c.products.ProductsController - Example error message -> not found product ID 1
 traceId=66729fe9307508232033e9e4e762fb33 spanId=f54ffdfa23696b3b

In conclusion, “traceId” and “spanId” help the developer trace logs by “traceId” so that the developer can filter uninvolved logs, making the developer trace log easier.

Logback JSON format.

Logback provides support for writing logs in JSON format.

<appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
    <encoder class="ch.qos.logback.classic.encoder.JsonEncoder">
    </encoder>
</appender>
curl http://localhost:8080/api/products/1
{
   "sequenceNumber":0,
   "timestamp":1718789848868,
   "nanoseconds":868601700,
   "level":"INFO",
   "threadName":"http-nio-8080-exec-2",
   "loggerName":"com.example.demo.controller.products.ProductsController",
   "context":{
      "name":"default",
      "birthdate":1718789095279,
      "properties":{
         
      }
   },
   "mdc":{
      "traceId":"6672a6d8314917fcc6cfe2d22c60ff89",
      "spanId":"fee807d011e0d298"
   },
   "message":"Example  info  message -> Received product ID {}",
   "arguments":[
      "1"
   ],
   "throwable":null
}

The log is shown in JSON format.

Configure Logback with a custom JSON pattern and define additional fields.

The developer can modify pattern JSON by using Logstash Logback Encoder.

<!-- https://mvnrepository.com/artifact/net.logstash.logback/logstash-logback-encoder -->
<dependency>
  <groupId>net.logstash.logback</groupId>
  <artifactId>logstash-logback-encoder</artifactId>
  <version>7.4</version>
</dependency>
<appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
    <encoder class="net.logstash.logback.encoder.LoggingEventCompositeJsonEncoder">
        <providers>
            <timestamp>
                <fieldName>timestamp</fieldName>
                <pattern>yyyy-MM-dd HH:mm:ss</pattern>
            </timestamp>
            <pattern>
                <pattern>
                    { "level": "%level", "thread": "%thread", "logger": "%logger{36}", "message": "%msg", "traceId": "%X{traceId}", "spanId": "%X{spanId}", "appname":"MyApp", "env":"dev" }
                </pattern>
            </pattern>
        </providers>
    </encoder>
</appender>
curl http://localhost:8080/api/products/1
{
   "timestamp":"2024-06-21 15:37:12",
   "level":"INFO",
   "thread":"http-nio-8080-exec-1",
   "logger":"c.e.d.c.products.ProductsController",
   "message":"Example  info  message -> Received product ID 1",
   "traceId":"66753bb8662d68ae027a9a19ba6009b8",
   "spanId":"dad85f50c7cf56e6",
   "appname":"MyApp",
   "env":"dev"
}

The logs show a message log that provides a config pattern following the logback-spring.xml file.

Read more about Logstash Logback Encoder. link

Finally, the developer can design a log file or log directory structure following log design by config through Logback. This provides crucial features covering a wide support write log pattern that makes an application easy to trace and maintain. Logback can integrate with Elastic Search or Grafana Loki. The developer can monitor the log on the dashboard, which helps the developer trace the logs more effectively than reading the log by the log file. The developer can choose log libraries for suitable projects.

Read more about logback integrated with Granfana Loki. link

Read more about logback and Logstash integrated with Elasticsearch and Kibana. link

Thanks for reading.

Spring Boot
Logging
Log4j
Slf4j
Logback

--

--

Nithidol Vacharotayan

Written by Nithidol Vacharotayan

24 Followers

Programming enthusiast with years of experience loves sharing knowledge with others and exploring new technologies together!

Help

Status

About

Careers

Press

Blog

Privacy

Terms

Text to speech

Teams