Spring Boot Logging in GCP Stackdriver
Published in
4 min readDec 29, 2019
If you are deploying a Spring Boot App on Google Cloud Platform and trying to figure out how to effectively log in Stackdriver, this is the post for you.
Some Considerations
- Format: The way Stackdriver is structured, the default spring boot log format is not convenient. A JSON format is more ideal. GCP’s recommendation is to use LogEntry object and its associated format.
{
"logName": string,
"resource": {
object (MonitoredResource)
},
"timestamp": string,
"receiveTimestamp": string,
"severity": enum (LogSeverity),
"insertId": string,
"httpRequest": {
object (HttpRequest)
},
"labels": {
string: string,
...
},
"metadata": {
object (MonitoredResourceMetadata)
},
"operation": {
object (LogEntryOperation)
},
"trace": string,
"spanId": string,
"traceSampled": boolean,
"sourceLocation": {
object (LogEntrySourceLocation)
},
// Union field payload can be only one of the following:
"protoPayload": {
"@type": string,
field1: ...,
...
},
"textPayload": string,
"jsonPayload": {
object
}
// End of list of possible types for union field payload.
}- Tracing: Default spring boot logs capture trace and span information, this also needs to be captured the JSON format. For this, we will be using Spring Cloud Sleuth.
- Linking Stackdriver Tracing and Stackdriver Logging: Whenever you click on a Trace in Stackdriver Trace, there is a little button which looks like this
When we click View, we will be redirected to the Stackdriver Logging section which contains all the logs under that particular trace. Well, at least that is the ideal state.
In order to achieve this, we need to set the LogEntry “trace” field using the format “projects/[PROJECT-ID]/traces/[TRACE-ID]” when we write logs to Stackdriver Logging.
Example:
trace: projects/my-projectid/traces/06796866738c859f2f19b7cfb3214824
NOTE: If you use Google Kubernetes Engine or the Stackdriver Logging Agent via Fluentd, we can set the LogEntry “trace” and “span_id” fields by writing structured logs with the keys of “logging.googleapis.com/trace” and “logging.googleapis.com/span_id”.
Example:
logging.googleapis.com/trace: projects/my-projectid/traces/06796866738c859f2f19b7cfb3214824
logging.googleapis.com/spanId: 2797cfb321482485
We can achieve this by using Spring Cloud GCP Project’s spring-cloud-gcp-starter-logging.
Implementation
Create a Spring Boot App.
pom.xml:
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>2.2.2.RELEASE</version>
<relativePath/> <!-- lookup parent from repository -->
</parent>
<groupId>com.anoophp</groupId>
<artifactId>gcp-stackdriver-logging</artifactId>
<version>0.0.1-SNAPSHOT</version>
<name>gcp-stackdriver-logging</name>
<description>GCP Stackdriver Logging example</description>
<properties>
<java.version>1.8</java.version>
<spring-cloud.version>Hoxton.SR1</spring-cloud.version>
</properties>
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-webflux</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-gcp-starter-logging</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-sleuth</artifactId>
</dependency>
<dependency>
<groupId>org.projectlombok</groupId>
<artifactId>lombok</artifactId>
<optional>true</optional>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
<exclusions>
<exclusion>
<groupId>org.junit.vintage</groupId>
<artifactId>junit-vintage-engine</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>io.projectreactor</groupId>
<artifactId>reactor-test</artifactId>
<scope>test</scope>
</dependency>
</dependencies>
<dependencyManagement>
<dependencies>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-dependencies</artifactId>
<version>${spring-cloud.version}</version>
<type>pom</type>
<scope>import</scope>
</dependency>
</dependencies>
</dependencyManagement>
<build>
<plugins>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
</plugin>
</plugins>
</build>
</project>Create a file under main/resources
logback-spring.xml:
<configuration>
<property name="projectId" value="${projectId:-${GOOGLE_CLOUD_PROJECT}}"/>
<appender name="CONSOLE_JSON" class="ch.qos.logback.core.ConsoleAppender">
<encoder class="ch.qos.logback.core.encoder.LayoutWrappingEncoder">
<layout class="org.springframework.cloud.gcp.logging.StackdriverJsonLayout">
<projectId>${projectId}</projectId>
</layout>
</encoder>
</appender>
<root level="INFO">
<appender-ref ref="CONSOLE_JSON" />
</root>
</configuration>We are using the org.springframework.cloud.gcp.logging.StackdriverJsonLayout Layout provided the GCP Logging Starter project we have imported in our POM.
This ensures the project ID is automatically set when deployed in the GKE and GCE.
GcpStackdriverLoggingApplication.java:
package com.anoophp.gcpstackdriverlogging;
import lombok.extern.slf4j.Slf4j;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RestController;
@SpringBootApplication
public class GcpStackdriverLoggingApplication {
public static void main(String[] args) {
SpringApplication.run(GcpStackdriverLoggingApplication.class, args);
}
}
@RestController
@Slf4j
class Foo {
@GetMapping("/foo")
public String bar() {
log.info("Test log");
return "bar";
}
}That’s it!!
When you run this app and hit the /foo endpoint in or outside of GCP, you see this.
/Library/Java/JavaVirtualMachines/jdk1.8.0_172.jdk/Contents/Home/bin/java com.anoophp.gcpstackdriverlogging.GcpStackdriverLoggingApplication. ____ _ __ _ _
/\\ / ___'_ __ _ _(_)_ __ __ _ \ \ \ \
( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \
\\/ ___)| |_)| | | | | || (_| | ) ) ) )
' |____| .__|_| |_|_| |_\__, | / / / /
=========|_|==============|___/=/_/_/_/
:: Spring Boot :: (v2.2.2.RELEASE){"timestampSeconds":1577607033,"timestampNanos":29000000,"severity":"INFO","thread":"main","logger":"com.anoophp.gcpstackdriverlogging.GcpStackdriverLoggingApplication","message":"No active profile set, falling back to default profiles: default","context":"default"}
{"timestampSeconds":1577607033,"timestampNanos":479000000,"severity":"INFO","thread":"main","logger":"org.springframework.cloud.context.scope.GenericScope","message":"BeanFactory id\u003dc7b095b5-61ce-30c0-b7a7-fcc51940dbd3","context":"default"}{"timestampSeconds":1577607033,"timestampNanos":686000000,"severity":"INFO","thread":"main","logger":"org.springframework.context.support.PostProcessorRegistrationDelegate$BeanPostProcessorChecker","message":"Bean \u0027org.springframework.cloud.sleuth.instrument.web.client.TraceWebClientAutoConfiguration$NettyConfiguration\u0027 of type [org.springframework.cloud.sleuth.instrument.web.client.TraceWebClientAutoConfiguration$NettyConfiguration] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying)","context":"default"}{"timestampSeconds":1577607034,"timestampNanos":131000000,"severity":"INFO","thread":"main","logger":"com.google.auth.oauth2.ComputeEngineCredentials","message":"Failed to detect whether we are running on Google Compute Engine.","context":"default"}{"timestampSeconds":1577607034,"timestampNanos":131000000,"severity":"WARN","thread":"main","logger":"org.springframework.cloud.gcp.core.DefaultCredentialsProvider","message":"No core credentials are set. Service-specific credentials (e.g., spring.cloud.gcp.pubsub.credentials.*) should be used if your app uses services that require credentials.","context":"default"}{"timestampSeconds":1577607034,"timestampNanos":133000000,"severity":"INFO","thread":"main","logger":"org.springframework.cloud.gcp.autoconfigure.core.GcpContextAutoConfiguration","message":"The default project ID is gcp-webflux-logging-tracing","context":"default"}{"timestampSeconds":1577607034,"timestampNanos":325000000,"severity":"INFO","thread":"main","logger":"org.springframework.boot.web.embedded.netty.NettyWebServer","message":"Netty started on port(s): 8080","context":"default"}{"timestampSeconds":1577607034,"timestampNanos":330000000,"severity":"INFO","thread":"main","logger":"com.anoophp.gcpstackdriverlogging.GcpStackdriverLoggingApplication","message":"Started GcpStackdriverLoggingApplication in 1.998 seconds (JVM running for 2.472)","context":"default"}{"traceId":"5e8a279c5b6f16ab","spanId":"5e8a279c5b6f16ab","spanExportable":"false","timestampSeconds":1577607071,"timestampNanos":863000000,"severity":"INFO","thread":"reactor-http-nio-2","logger":"com.anoophp.gcpstackdriverlogging.Foo","message":"Test log","context":"default","logging.googleapis.com/trace":"projects/gcp-webflux-logging-tracing/traces/00000000000000005e8a279c5b6f16ab","logging.googleapis.com/spanId":"5e8a279c5b6f16ab"}
If you navigate to Stackdriver Trace -> Click on a Trace -> View to see the Stackdriver Logging and all the associated logs for that particular Trace.
You can find the full project in GitHub here.
If you have any questions, please drop in a comment.
