Docker’s health check and Spring Boot apps - how to control containers startup order in docker-compose
Hello! In this article I will describe how one can use Docker’s HEALTHCHECK
instruction to describe a dependencies between the containers inside of docker-compose.yml
file, and how it can be used to setup the order in which containers will run :)
What is a health check ?
In terms of Docker, health check is being used to verify if a container is still working. If you are familiar with Kubernetes, then probably you might have thought about Liveness and Readiness probes - and it’s a good association !
Often the fact that a container is running does not mean, that it’s ready to accept traffic. It is well described in Kubernetes documentation page regarding probes, but… what about Docker?
Don’t worry - Docker also has a built in mechanism to verify container’s state, and you can use it either in Dockerfile images, or in docker-compose.yml file.
Docker Compose and startup order
Docker Compose has a startup order mechanism built in, but as it is mentioned in the documentation:
On startup, Compose does not wait until a container is “ready”, only until it’s running.
An example you may see the most often is a simple usage of depends_on instruction in a Compose file, e.g. some service depends on a database. Bellow there is an example taken from the docs:
services:
web:
build: .
depends_on:
- db
- redis
redis:
image: redis
db:
image: postgres
It works, however, it is just a short syntax of this instruction ! In a long syntax, you can also provide a condition:
services:
web:
build: .
depends_on:
db:
condition: service_healthy
redis:
condition: service_started
redis:
image: redis
db:
image: postgres
Let’s see how it can be useful with Spring Boot applications.
Use case
Recently I have been playing around with Spring Cloud Config Server, and I wanted to run it and my example Config Client microservice using Docker Compose. I had my Config Client microservice configured to fail fast if a Config Server is not running:
spring:
cloud:
config:
fail-fast: true
And despite that I had configured dependency inside of my compose file… it failed! It turned out that the container with Config Server was running, but the application hasn’t been ready yet!
Luckily, I was able to fix it using:
- Spring Boot Actuator and it’s
/actuator/health/readiness
probe healthcheck
instruction indocker-compose.yml
filedepends_on
instruction indocker-compose.yml
file, but using long syntax
In the following tutorial I’ll show you what was my initial approach, and how I was able to fix this issue.
Demo
Code will be available in my GitHub repository - bellow I will focus only on the most important parts :)
Config Server
Let’s start with the Config Server. This project will require only 2 additional Spring Boot dependencies:
- spring-boot-starter-actuator
- spring-cloud-config-server
I’ve added @EnableConfigServer
over application’s main class, and configured application.yml
file:
server:
port: 8888
shutdown: graceful
spring:
application:
name: config-server
profiles:
active: native
cloud:
config:
server:
native:
search-locations:
- classpath:/config
management:
endpoints:
web:
exposure:
include: "health,refresh"
health:
readiness-state:
enabled: true
liveness-state:
enabled: true
endpoint:
health:
probes:
enabled: true
A few properties worth mentioning:
- spring.profiles.active - I used
native
profile, and… - spring.cloud.config.server.native.search-locations - config server will look for configuration files in
src/main/resources/config
directory - management.endpoints.** - I enabled liveness and readiness Actuator’s probes - it’s a very nice feature :) I will use one of these probes for Docker’s health check
Later, I added simple-microservice.yml
file under src/main/resources/config
directory:
server:
port: 8080
management:
endpoints:
web:
exposure:
include: "health,refresh"
health:
readiness-state:
enabled: true
liveness-state:
enabled: true
endpoint:
health:
probes:
enabled: true
example:
key: example-value
That’s it! After that, let’s build a .jar file:
mvn -DskipTests clean package
And create a Dockerfile:
FROM openjdk:17-slim as builder
ARG JAR_FILE=target/*.jar
COPY ${JAR_FILE} application.jar
RUN java -Djarmode=layertools -jar application.jar extract
FROM eclipse-temurin:17-jre-alpine
ENV JDK_JAVA_OPTIONS="-Xms256m -Xmx512m"
RUN apk --no-cache add bash curl
RUN mkdir /app
RUN addgroup -S spring && adduser -S spring -G spring
WORKDIR /app
COPY --from=builder dependencies/ /app
COPY --from=builder spring-boot-loader/ /app
COPY --from=builder snapshot-dependencies/ /app
COPY --from=builder application/ /app
RUN chown -R spring:spring .
USER spring:spring
ENTRYPOINT ["java", "-Djava.security.egd=file:/dev/./urandom", "-cp", "BOOT-INF/classes:BOOT-INF/lib/*", "pl.akolata.healthcheck.demo.configserver.ConfigServerApplication"]
And see if it’s running
docker build . \
--tag spring-boot-healthcheck-demo/config-service:latest
docker run \
--rm \
--name=spring-boot-healthcheck-demo-config-service \
-p 8888:8888 \
spring-boot-healthcheck-demo/config-service:latest
curl -X GET http://localhost:8888/actuator/health/readiness
{"status":"UP"}%
Good ! Let me sum up:
- Config Server application is configured and running using native profile
- it’s configuration enables Actuator’s
/readiness
probe - application itself may run in a container, based on the above Dockerfile
Config Client (named “simple-microservice”)
Dependencies needed:
- spring-boot-starter-actuator
- spring-cloud-starter-config
It will be much simpler that the previous app. It’s src/main/resources/application.yml
file looks like this:
spring:
application:
name: simple-microservice
config:
import: "optional:configserver:"
cloud:
config:
fail-fast: true
- spring.application.name - application will try to fetch simple-microservice.yml file from the config server (and others, profile specific, but you can find more details in the Config Server documentation)
- spring.config.import - provided value by default will try to access Config Server under http://localhost:8888 address
- spring.cloud.config.fail-fast - if a Config Server is not available, then application will fail immediately
At this point you can create .jar file, copy the previous Dockerfile and build the image in the same way as before. The only differences will be:
- the last line of Dockerfile, with the correct class and package:
ENTRYPOINT ["java", "-Djava.security.egd=file:/dev/./urandom", "-cp", "BOOT-INF/classes:BOOT-INF/lib/*", "pl.akolata.healthcheck.demo.simplemicroservice.SimpleMicroserviceApplication"]
- Docker image build&run commands
docker build . \
--tag spring-boot-healthcheck-demo/simple-microservice:latest
docker run \
--rm \
--name=spring-boot-healthcheck-demo-simple-microservice \
-p 8080:8888 \
spring-boot-healthcheck-demo/simple-microservice:latest
Surprise !
At this point, simple-microservice should fail. Why? Well, because of two things:
- It tries to reach Config Server under the default location which is http://localhost:8888 , so inside of the same container
- and because
spring.cloud.config.fail-fast=true
property has been set
Application logs should look more or less like these ones:
[main] ERROR org.springframework.boot.SpringApplication - Application run failed
org.springframework.cloud.config.client.ConfigClientFailFastException: Could not locate PropertySource and the fail fast property is set, failing
at org.springframework.cloud.config.client.ConfigServerConfigDataLoader.doLoad(ConfigServerConfigDataLoader.java:199)
at org.springframework.cloud.config.client.ConfigServerConfigDataLoader.load(ConfigServerConfigDataLoader.java:103)
at org.springframework.cloud.config.client.ConfigServerConfigDataLoader.load(ConfigServerConfigDataLoader.java:62)
at org.springframework.boot.context.config.ConfigDataLoaders.load(ConfigDataLoaders.java:96)
at org.springframework.boot.context.config.ConfigDataImporter.load(ConfigDataImporter.java:128)
at org.springframework.boot.context.config.ConfigDataImporter.resolveAndLoad(ConfigDataImporter.java:86)
at org.springframework.boot.context.config.ConfigDataEnvironmentContributors.withProcessedImports(ConfigDataEnvironmentContributors.java:115)
at org.springframework.boot.context.config.ConfigDataEnvironment.processWithProfiles(ConfigDataEnvironment.java:313)
at org.springframework.boot.context.config.ConfigDataEnvironment.processAndApply(ConfigDataEnvironment.java:234)
at org.springframework.boot.context.config.ConfigDataEnvironmentPostProcessor.postProcessEnvironment(ConfigDataEnvironmentPostProcessor.java:96)
at org.springframework.boot.context.config.ConfigDataEnvironmentPostProcessor.postProcessEnvironment(ConfigDataEnvironmentPostProcessor.java:89)
at org.springframework.boot.env.EnvironmentPostProcessorApplicationListener.onApplicationEnvironmentPreparedEvent(EnvironmentPostProcessorApplicationListener.java:109)
at org.springframework.boot.env.EnvironmentPostProcessorApplicationListener.onApplicationEvent(EnvironmentPostProcessorApplicationListener.java:94)
at org.springframework.context.event.SimpleApplicationEventMulticaster.doInvokeListener(SimpleApplicationEventMulticaster.java:176)
at org.springframework.context.event.SimpleApplicationEventMulticaster.invokeListener(SimpleApplicationEventMulticaster.java:169)
at org.springframework.context.event.SimpleApplicationEventMulticaster.multicastEvent(SimpleApplicationEventMulticaster.java:143)
at org.springframework.context.event.SimpleApplicationEventMulticaster.multicastEvent(SimpleApplicationEventMulticaster.java:131)
at org.springframework.boot.context.event.EventPublishingRunListener.multicastInitialEvent(EventPublishingRunListener.java:136)
at org.springframework.boot.context.event.EventPublishingRunListener.environmentPrepared(EventPublishingRunListener.java:81)
at org.springframework.boot.SpringApplicationRunListeners.lambda$environmentPrepared$2(SpringApplicationRunListeners.java:64)
at java.base/java.lang.Iterable.forEach(Unknown Source)
at org.springframework.boot.SpringApplicationRunListeners.doWithListeners(SpringApplicationRunListeners.java:118)
at org.springframework.boot.SpringApplicationRunListeners.doWithListeners(SpringApplicationRunListeners.java:112)
at org.springframework.boot.SpringApplicationRunListeners.environmentPrepared(SpringApplicationRunListeners.java:63)
at org.springframework.boot.SpringApplication.prepareEnvironment(SpringApplication.java:352)
at org.springframework.boot.SpringApplication.run(SpringApplication.java:303)
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1302)
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1291)
at pl.akolata.healthcheck.demo.simplemicroservice.SimpleMicroserviceApplication.main(SimpleMicroserviceApplication.java:10)
Caused by: org.springframework.web.client.ResourceAccessException: I/O error on GET request for "http://localhost:8888/simple-microservice/default": Connection refused
at org.springframework.web.client.RestTemplate.createResourceAccessException(RestTemplate.java:888)
at org.springframework.web.client.RestTemplate.doExecute(RestTemplate.java:868)
at org.springframework.web.client.RestTemplate.execute(RestTemplate.java:764)
at org.springframework.web.client.RestTemplate.exchange(RestTemplate.java:646)
at org.springframework.cloud.config.client.ConfigServerConfigDataLoader.getRemoteEnvironment(ConfigServerConfigDataLoader.java:304)
at org.springframework.cloud.config.client.ConfigServerConfigDataLoader.doLoad(ConfigServerConfigDataLoader.java:119)
... 28 common frames omitted
Caused by: java.net.ConnectException: Connection refused
at java.base/sun.nio.ch.Net.pollConnect(Native Method)
at java.base/sun.nio.ch.Net.pollConnectNow(Unknown Source)
at java.base/sun.nio.ch.NioSocketImpl.timedFinishConnect(Unknown Source)
at java.base/sun.nio.ch.NioSocketImpl.connect(Unknown Source)
at java.base/java.net.Socket.connect(Unknown Source)
at java.base/sun.net.NetworkClient.doConnect(Unknown Source)
at java.base/sun.net.www.http.HttpClient.openServer(Unknown Source)
at java.base/sun.net.www.http.HttpClient.openServer(Unknown Source)
at java.base/sun.net.www.http.HttpClient.<init>(Unknown Source)
at java.base/sun.net.www.http.HttpClient.New(Unknown Source)
at java.base/sun.net.www.http.HttpClient.New(Unknown Source)
at java.base/sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(Unknown Source)
at java.base/sun.net.www.protocol.http.HttpURLConnection.plainConnect0(Unknown Source)
at java.base/sun.net.www.protocol.http.HttpURLConnection.plainConnect(Unknown Source)
at java.base/sun.net.www.protocol.http.HttpURLConnection.connect(Unknown Source)
at org.springframework.http.client.SimpleBufferingClientHttpRequest.executeInternal(SimpleBufferingClientHttpRequest.java:75)
at org.springframework.http.client.AbstractBufferingClientHttpRequest.executeInternal(AbstractBufferingClientHttpRequest.java:48)
at org.springframework.http.client.AbstractClientHttpRequest.execute(AbstractClientHttpRequest.java:66)
at org.springframework.web.client.RestTemplate.doExecute(RestTemplate.java:862)
... 32 common frames omitted
Docker-compose (first attempt)
Let’s try to use depends_on
mechanism of Docker Compose, using short syntax:
version: '3.9'
services:
config-service:
container_name: spring-boot-healthcheck-demo-config-service
pull_policy: always
image: spring-boot-healthcheck-demo/config-service:latest
ports:
- "8888:8888"
networks:
backend:
aliases:
- "config-server"
simple-microservice:
container_name: spring-boot-healthcheck-demo-simple-microservice
image: spring-boot-healthcheck-demo/simple-microservice:latest
pull_policy: always
environment:
SERVER_PORT: 8080
SPRING_CONFIG_IMPORT: "optional:configserver:http://config-service:8888"
depends_on:
- config-service
networks:
backend:
aliases:
- "simple-microservice"
networks:
backend:
driver: bridge
Unfortunately, it will also fail. Let’s look at the logs:
docker-compose up
Starting spring-boot-healthcheck-demo-config-service ... done
Creating spring-boot-healthcheck-demo-simple-microservice ... done
Attaching to spring-boot-healthcheck-demo-config-service, spring-boot-healthcheck-demo-simple-microservice
spring-boot-healthcheck-demo-config-service | NOTE: Picked up JDK_JAVA_OPTIONS: -Xms256m -Xmx512m
spring-boot-healthcheck-demo-simple-microservice | NOTE: Picked up JDK_JAVA_OPTIONS: -Xms256m -Xmx512m
spring-boot-healthcheck-demo-config-service | NOTE: Picked up JDK_JAVA_OPTIONS: -Xms256m -Xmx512m
spring-boot-healthcheck-demo-simple-microservice | NOTE: Picked up JDK_JAVA_OPTIONS: -Xms256m -Xmx512m
spring-boot-healthcheck-demo-config-service |
spring-boot-healthcheck-demo-config-service | . ____ _ __ _ _
spring-boot-healthcheck-demo-config-service | /\\ / ___'_ __ _ _(_)_ __ __ _ \ \ \ \
spring-boot-healthcheck-demo-config-service | ( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \
spring-boot-healthcheck-demo-config-service | \\/ ___)| |_)| | | | | || (_| | ) ) ) )
spring-boot-healthcheck-demo-config-service | ' |____| .__|_| |_|_| |_\__, | / / / /
spring-boot-healthcheck-demo-config-service | =========|_|==============|___/=/_/_/_/
spring-boot-healthcheck-demo-config-service | :: Spring Boot :: (v2.7.8)
spring-boot-healthcheck-demo-config-service |
spring-boot-healthcheck-demo-config-service | 2023-01-29 22:41:37.211 INFO 1 --- [ main] p.a.h.d.c.ConfigServerApplication : Starting ConfigServerApplication using Java 17.0.6 on 7888840625df with PID 1 (/app/BOOT-INF/classes started by spring in /app)
spring-boot-healthcheck-demo-config-service | 2023-01-29 22:41:37.216 INFO 1 --- [ main] p.a.h.d.c.ConfigServerApplication : The following 1 profile is active: "native"
spring-boot-healthcheck-demo-simple-microservice | 22:41:37.801 [main] ERROR org.springframework.boot.SpringApplication - Application run failed
spring-boot-healthcheck-demo-simple-microservice | org.springframework.cloud.config.client.ConfigClientFailFastException: Could not locate PropertySource and the fail fast property is set, failing
spring-boot-healthcheck-demo-simple-microservice | at org.springframework.cloud.config.client.ConfigServerConfigDataLoader.doLoad(ConfigServerConfigDataLoader.java:199)
spring-boot-healthcheck-demo-simple-microservice | at org.springframework.cloud.config.client.ConfigServerConfigDataLoader.load(ConfigServerConfigDataLoader.java:103)
spring-boot-healthcheck-demo-simple-microservice | at org.springframework.cloud.config.client.ConfigServerConfigDataLoader.load(ConfigServerConfigDataLoader.java:62)
spring-boot-healthcheck-demo-simple-microservice | at org.springframework.boot.context.config.ConfigDataLoaders.load(ConfigDataLoaders.java:96)
spring-boot-healthcheck-demo-simple-microservice | at org.springframework.boot.context.config.ConfigDataImporter.load(ConfigDataImporter.java:128)
spring-boot-healthcheck-demo-simple-microservice | at org.springframework.boot.context.config.ConfigDataImporter.resolveAndLoad(ConfigDataImporter.java:86)
spring-boot-healthcheck-demo-simple-microservice | at org.springframework.boot.context.config.ConfigDataEnvironmentContributors.withProcessedImports(ConfigDataEnvironmentContributors.java:115)
spring-boot-healthcheck-demo-simple-microservice | at org.springframework.boot.context.config.ConfigDataEnvironment.processWithProfiles(ConfigDataEnvironment.java:313)
spring-boot-healthcheck-demo-simple-microservice | at org.springframework.boot.context.config.ConfigDataEnvironment.processAndApply(ConfigDataEnvironment.java:234)
spring-boot-healthcheck-demo-simple-microservice | at org.springframework.boot.context.config.ConfigDataEnvironmentPostProcessor.postProcessEnvironment(ConfigDataEnvironmentPostProcessor.java:96)
spring-boot-healthcheck-demo-simple-microservice | at org.springframework.boot.context.config.ConfigDataEnvironmentPostProcessor.postProcessEnvironment(ConfigDataEnvironmentPostProcessor.java:89)
spring-boot-healthcheck-demo-simple-microservice | at org.springframework.boot.env.EnvironmentPostProcessorApplicationListener.onApplicationEnvironmentPreparedEvent(EnvironmentPostProcessorApplicationListener.java:109)
spring-boot-healthcheck-demo-simple-microservice | at org.springframework.boot.env.EnvironmentPostProcessorApplicationListener.onApplicationEvent(EnvironmentPostProcessorApplicationListener.java:94)
spring-boot-healthcheck-demo-simple-microservice | at org.springframework.context.event.SimpleApplicationEventMulticaster.doInvokeListener(SimpleApplicationEventMulticaster.java:176)
spring-boot-healthcheck-demo-simple-microservice | at org.springframework.context.event.SimpleApplicationEventMulticaster.invokeListener(SimpleApplicationEventMulticaster.java:169)
spring-boot-healthcheck-demo-simple-microservice | at org.springframework.context.event.SimpleApplicationEventMulticaster.multicastEvent(SimpleApplicationEventMulticaster.java:143)
spring-boot-healthcheck-demo-simple-microservice | at org.springframework.context.event.SimpleApplicationEventMulticaster.multicastEvent(SimpleApplicationEventMulticaster.java:131)
spring-boot-healthcheck-demo-simple-microservice | at org.springframework.boot.context.event.EventPublishingRunListener.multicastInitialEvent(EventPublishingRunListener.java:136)
spring-boot-healthcheck-demo-simple-microservice | at org.springframework.boot.context.event.EventPublishingRunListener.environmentPrepared(EventPublishingRunListener.java:81)
spring-boot-healthcheck-demo-simple-microservice | at org.springframework.boot.SpringApplicationRunListeners.lambda$environmentPrepared$2(SpringApplicationRunListeners.java:64)
spring-boot-healthcheck-demo-simple-microservice | at java.base/java.lang.Iterable.forEach(Unknown Source)
spring-boot-healthcheck-demo-simple-microservice | at org.springframework.boot.SpringApplicationRunListeners.doWithListeners(SpringApplicationRunListeners.java:118)
spring-boot-healthcheck-demo-simple-microservice | at org.springframework.boot.SpringApplicationRunListeners.doWithListeners(SpringApplicationRunListeners.java:112)
spring-boot-healthcheck-demo-simple-microservice | at org.springframework.boot.SpringApplicationRunListeners.environmentPrepared(SpringApplicationRunListeners.java:63)
spring-boot-healthcheck-demo-simple-microservice | at org.springframework.boot.SpringApplication.prepareEnvironment(SpringApplication.java:352)
spring-boot-healthcheck-demo-simple-microservice | at org.springframework.boot.SpringApplication.run(SpringApplication.java:303)
spring-boot-healthcheck-demo-simple-microservice | at org.springframework.boot.SpringApplication.run(SpringApplication.java:1302)
spring-boot-healthcheck-demo-simple-microservice | at org.springframework.boot.SpringApplication.run(SpringApplication.java:1291)
spring-boot-healthcheck-demo-simple-microservice | at pl.akolata.healthcheck.demo.simplemicroservice.SimpleMicroserviceApplication.main(SimpleMicroserviceApplication.java:10)
spring-boot-healthcheck-demo-simple-microservice | Caused by: org.springframework.web.client.ResourceAccessException: I/O error on GET request for "http://config-service:8888/simple-microservice/default": Connection refused
spring-boot-healthcheck-demo-simple-microservice | at org.springframework.web.client.RestTemplate.createResourceAccessException(RestTemplate.java:888)
spring-boot-healthcheck-demo-simple-microservice | at org.springframework.web.client.RestTemplate.doExecute(RestTemplate.java:868)
spring-boot-healthcheck-demo-simple-microservice | at org.springframework.web.client.RestTemplate.execute(RestTemplate.java:764)
spring-boot-healthcheck-demo-simple-microservice | at org.springframework.web.client.RestTemplate.exchange(RestTemplate.java:646)
spring-boot-healthcheck-demo-simple-microservice | at org.springframework.cloud.config.client.ConfigServerConfigDataLoader.getRemoteEnvironment(ConfigServerConfigDataLoader.java:304)
spring-boot-healthcheck-demo-simple-microservice | at org.springframework.cloud.config.client.ConfigServerConfigDataLoader.doLoad(ConfigServerConfigDataLoader.java:119)
spring-boot-healthcheck-demo-simple-microservice | ... 28 common frames omitted
spring-boot-healthcheck-demo-simple-microservice | Caused by: java.net.ConnectException: Connection refused
spring-boot-healthcheck-demo-simple-microservice | at java.base/sun.nio.ch.Net.pollConnect(Native Method)
spring-boot-healthcheck-demo-simple-microservice | at java.base/sun.nio.ch.Net.pollConnectNow(Unknown Source)
spring-boot-healthcheck-demo-simple-microservice | at java.base/sun.nio.ch.NioSocketImpl.timedFinishConnect(Unknown Source)
spring-boot-healthcheck-demo-simple-microservice | at java.base/sun.nio.ch.NioSocketImpl.connect(Unknown Source)
spring-boot-healthcheck-demo-simple-microservice | at java.base/java.net.Socket.connect(Unknown Source)
spring-boot-healthcheck-demo-simple-microservice | at java.base/sun.net.NetworkClient.doConnect(Unknown Source)
spring-boot-healthcheck-demo-simple-microservice | at java.base/sun.net.www.http.HttpClient.openServer(Unknown Source)
spring-boot-healthcheck-demo-simple-microservice | at java.base/sun.net.www.http.HttpClient.openServer(Unknown Source)
spring-boot-healthcheck-demo-simple-microservice | at java.base/sun.net.www.http.HttpClient.<init>(Unknown Source)
spring-boot-healthcheck-demo-simple-microservice | at java.base/sun.net.www.http.HttpClient.New(Unknown Source)
spring-boot-healthcheck-demo-simple-microservice | at java.base/sun.net.www.http.HttpClient.New(Unknown Source)
spring-boot-healthcheck-demo-simple-microservice | at java.base/sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(Unknown Source)
spring-boot-healthcheck-demo-simple-microservice | at java.base/sun.net.www.protocol.http.HttpURLConnection.plainConnect0(Unknown Source)
spring-boot-healthcheck-demo-simple-microservice | at java.base/sun.net.www.protocol.http.HttpURLConnection.plainConnect(Unknown Source)
spring-boot-healthcheck-demo-simple-microservice | at java.base/sun.net.www.protocol.http.HttpURLConnection.connect(Unknown Source)
spring-boot-healthcheck-demo-simple-microservice | at org.springframework.http.client.SimpleBufferingClientHttpRequest.executeInternal(SimpleBufferingClientHttpRequest.java:75)
spring-boot-healthcheck-demo-simple-microservice | at org.springframework.http.client.AbstractBufferingClientHttpRequest.executeInternal(AbstractBufferingClientHttpRequest.java:48)
spring-boot-healthcheck-demo-simple-microservice | at org.springframework.http.client.AbstractClientHttpRequest.execute(AbstractClientHttpRequest.java:66)
spring-boot-healthcheck-demo-simple-microservice | at org.springframework.web.client.RestTemplate.doExecute(RestTemplate.java:862)
spring-boot-healthcheck-demo-simple-microservice | ... 32 common frames omitted
spring-boot-healthcheck-demo-simple-microservice exited with code 1
spring-boot-healthcheck-demo-config-service | 2023-01-29 22:41:39.003 INFO 1 --- [ main] o.s.cloud.context.scope.GenericScope : BeanFactory id=7ea18433-9846-316d-934a-7954a98ecc3c
spring-boot-healthcheck-demo-config-service | 2023-01-29 22:41:39.328 INFO 1 --- [ main] o.s.b.w.embedded.tomcat.TomcatWebServer : Tomcat initialized with port(s): 8888 (http)
spring-boot-healthcheck-demo-config-service | 2023-01-29 22:41:39.339 INFO 1 --- [ main] o.apache.catalina.core.StandardService : Starting service [Tomcat]
spring-boot-healthcheck-demo-config-service | 2023-01-29 22:41:39.339 INFO 1 --- [ main] org.apache.catalina.core.StandardEngine : Starting Servlet engine: [Apache Tomcat/9.0.71]
spring-boot-healthcheck-demo-config-service | 2023-01-29 22:41:39.450 INFO 1 --- [ main] o.a.c.c.C.[Tomcat].[localhost].[/] : Initializing Spring embedded WebApplicationContext
spring-boot-healthcheck-demo-config-service | 2023-01-29 22:41:39.450 INFO 1 --- [ main] w.s.c.ServletWebServerApplicationContext : Root WebApplicationContext: initialization completed in 1639 ms
spring-boot-healthcheck-demo-config-service | 2023-01-29 22:41:40.332 INFO 1 --- [ main] o.s.b.a.e.web.EndpointLinksResolver : Exposing 2 endpoint(s) beneath base path '/actuator'
spring-boot-healthcheck-demo-config-service | 2023-01-29 22:41:40.373 INFO 1 --- [ main] o.s.b.w.embedded.tomcat.TomcatWebServer : Tomcat started on port(s): 8888 (http) with context path ''
spring-boot-healthcheck-demo-config-service | 2023-01-29 22:41:40.395 INFO 1 --- [ main] p.a.h.d.c.ConfigServerApplication : Started ConfigServerApplication in 3.729 seconds (JVM running for 4.145)
So:
- we had
depends_on
instruction, and… - … config-server indeed started first, but…
- … it hasn’t been ready yet (take a look at the last line:
Started ConfigServerApplication in
) - … and simple-microservice had already failed
Docker-compose (working attempt)
I did the following steps:
- configured Docker Compose how to perform a health check on these images
- used Spring Boot Actuator
/readiness
probe for it - configured depends_on using long syntax this time
version: '3.9'
services:
config-service:
container_name: spring-boot-healthcheck-demo-config-service
pull_policy: always
image: spring-boot-healthcheck-demo/config-service:latest
ports:
- "8888:8888"
healthcheck:
test: "curl --fail --silent localhost:8888/actuator/health/readiness | grep UP || exit 1"
interval: 2s
timeout: 3s
retries: 5
start_period: 2s
networks:
backend:
aliases:
- "config-server"
simple-microservice:
container_name: spring-boot-healthcheck-demo-simple-microservice
image: spring-boot-healthcheck-demo/simple-microservice:latest
pull_policy: always
healthcheck:
test: "curl --fail --silent localhost:8080/actuator/health/readiness | grep UP || exit 1"
interval: 2s
timeout: 3s
retries: 5
start_period: 2s
environment:
SERVER_PORT: 8080
SPRING_CONFIG_IMPORT: "optional:configserver:http://config-service:8888"
depends_on:
config-service:
condition: service_healthy
networks:
backend:
aliases:
- "simple-microservice"
networks:
backend:
driver: bridge
Voila! Now it’s working :)
Creating network "spring-boot-docker-healthcheck-demo_backend" with driver "bridge"
Creating spring-boot-healthcheck-demo-config-service ... done
Creating spring-boot-healthcheck-demo-simple-microservice ... done
Attaching to spring-boot-healthcheck-demo-config-service, spring-boot-healthcheck-demo-simple-microservice
spring-boot-healthcheck-demo-simple-microservice | NOTE: Picked up JDK_JAVA_OPTIONS: -Xms256m -Xmx512m
spring-boot-healthcheck-demo-config-service | NOTE: Picked up JDK_JAVA_OPTIONS: -Xms256m -Xmx512m
spring-boot-healthcheck-demo-simple-microservice | NOTE: Picked up JDK_JAVA_OPTIONS: -Xms256m -Xmx512m
spring-boot-healthcheck-demo-config-service | NOTE: Picked up JDK_JAVA_OPTIONS: -Xms256m -Xmx512m
spring-boot-healthcheck-demo-config-service |
spring-boot-healthcheck-demo-config-service | . ____ _ __ _ _
spring-boot-healthcheck-demo-config-service | /\\ / ___'_ __ _ _(_)_ __ __ _ \ \ \ \
spring-boot-healthcheck-demo-config-service | ( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \
spring-boot-healthcheck-demo-config-service | \\/ ___)| |_)| | | | | || (_| | ) ) ) )
spring-boot-healthcheck-demo-config-service | ' |____| .__|_| |_|_| |_\__, | / / / /
spring-boot-healthcheck-demo-config-service | =========|_|==============|___/=/_/_/_/
spring-boot-healthcheck-demo-config-service | :: Spring Boot :: (v2.7.8)
spring-boot-healthcheck-demo-config-service |
spring-boot-healthcheck-demo-config-service | 2023-01-29 22:48:16.587 INFO 1 --- [ main] p.a.h.d.c.ConfigServerApplication : Starting ConfigServerApplication using Java 17.0.6 on f937eaa1f922 with PID 1 (/app/BOOT-INF/classes started by spring in /app)
spring-boot-healthcheck-demo-config-service | 2023-01-29 22:48:16.590 INFO 1 --- [ main] p.a.h.d.c.ConfigServerApplication : The following 1 profile is active: "native"
spring-boot-healthcheck-demo-config-service | 2023-01-29 22:48:17.776 INFO 1 --- [ main] o.s.cloud.context.scope.GenericScope : BeanFactory id=7ea18433-9846-316d-934a-7954a98ecc3c
spring-boot-healthcheck-demo-config-service | 2023-01-29 22:48:18.098 INFO 1 --- [ main] o.s.b.w.embedded.tomcat.TomcatWebServer : Tomcat initialized with port(s): 8888 (http)
spring-boot-healthcheck-demo-config-service | 2023-01-29 22:48:18.108 INFO 1 --- [ main] o.apache.catalina.core.StandardService : Starting service [Tomcat]
spring-boot-healthcheck-demo-config-service | 2023-01-29 22:48:18.108 INFO 1 --- [ main] org.apache.catalina.core.StandardEngine : Starting Servlet engine: [Apache Tomcat/9.0.71]
spring-boot-healthcheck-demo-config-service | 2023-01-29 22:48:18.214 INFO 1 --- [ main] o.a.c.c.C.[Tomcat].[localhost].[/] : Initializing Spring embedded WebApplicationContext
spring-boot-healthcheck-demo-config-service | 2023-01-29 22:48:18.214 INFO 1 --- [ main] w.s.c.ServletWebServerApplicationContext : Root WebApplicationContext: initialization completed in 1562 ms
spring-boot-healthcheck-demo-config-service | 2023-01-29 22:48:19.146 INFO 1 --- [ main] o.s.b.a.e.web.EndpointLinksResolver : Exposing 2 endpoint(s) beneath base path '/actuator'
spring-boot-healthcheck-demo-config-service | 2023-01-29 22:48:19.203 INFO 1 --- [ main] o.s.b.w.embedded.tomcat.TomcatWebServer : Tomcat started on port(s): 8888 (http) with context path ''
spring-boot-healthcheck-demo-config-service | 2023-01-29 22:48:19.228 INFO 1 --- [ main] p.a.h.d.c.ConfigServerApplication : Started ConfigServerApplication in 3.208 seconds (JVM running for 3.802)
spring-boot-healthcheck-demo-config-service | 2023-01-29 22:48:19.657 INFO 1 --- [nio-8888-exec-1] o.a.c.c.C.[Tomcat].[localhost].[/] : Initializing Spring DispatcherServlet 'dispatcherServlet'
spring-boot-healthcheck-demo-config-service | 2023-01-29 22:48:19.657 INFO 1 --- [nio-8888-exec-1] o.s.web.servlet.DispatcherServlet : Initializing Servlet 'dispatcherServlet'
spring-boot-healthcheck-demo-config-service | 2023-01-29 22:48:19.659 INFO 1 --- [nio-8888-exec-1] o.s.web.servlet.DispatcherServlet : Completed initialization in 2 ms
spring-boot-healthcheck-demo-config-service | 2023-01-29 22:48:21.400 INFO 1 --- [nio-8888-exec-2] o.s.c.c.s.e.NativeEnvironmentRepository : Adding property source: Config resource 'class path resource [config/simple-microservice.yml]' via location 'classpath:/config/'
spring-boot-healthcheck-demo-config-service | 2023-01-29 22:48:21.474 INFO 1 --- [nio-8888-exec-3] o.s.c.c.s.e.NativeEnvironmentRepository : Adding property source: Config resource 'class path resource [config/simple-microservice.yml]' via location 'classpath:/config/'
spring-boot-healthcheck-demo-simple-microservice |
spring-boot-healthcheck-demo-simple-microservice | . ____ _ __ _ _
spring-boot-healthcheck-demo-simple-microservice | /\\ / ___'_ __ _ _(_)_ __ __ _ \ \ \ \
spring-boot-healthcheck-demo-simple-microservice | ( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \
spring-boot-healthcheck-demo-simple-microservice | \\/ ___)| |_)| | | | | || (_| | ) ) ) )
spring-boot-healthcheck-demo-simple-microservice | ' |____| .__|_| |_|_| |_\__, | / / / /
spring-boot-healthcheck-demo-simple-microservice | =========|_|==============|___/=/_/_/_/
spring-boot-healthcheck-demo-simple-microservice | :: Spring Boot :: (v3.0.2)
spring-boot-healthcheck-demo-simple-microservice |
spring-boot-healthcheck-demo-simple-microservice | 2023-01-29T22:48:21.611Z INFO 1 --- [ main] p.a.h.d.s.SimpleMicroserviceApplication : Starting SimpleMicroserviceApplication using Java 17.0.6 with PID 1 (/app/BOOT-INF/classes started by spring in /app)
spring-boot-healthcheck-demo-simple-microservice | 2023-01-29T22:48:21.614Z INFO 1 --- [ main] p.a.h.d.s.SimpleMicroserviceApplication : No active profile set, falling back to 1 default profile: "default"
spring-boot-healthcheck-demo-simple-microservice | 2023-01-29T22:48:21.661Z INFO 1 --- [ main] o.s.c.c.c.ConfigServerConfigDataLoader : Fetching config from server at : http://config-service:8888
spring-boot-healthcheck-demo-simple-microservice | 2023-01-29T22:48:21.661Z INFO 1 --- [ main] o.s.c.c.c.ConfigServerConfigDataLoader : Located environment: name=simple-microservice, profiles=[default], label=null, version=null, state=null
spring-boot-healthcheck-demo-simple-microservice | 2023-01-29T22:48:21.661Z INFO 1 --- [ main] o.s.c.c.c.ConfigServerConfigDataLoader : Fetching config from server at : http://config-service:8888
spring-boot-healthcheck-demo-simple-microservice | 2023-01-29T22:48:21.662Z INFO 1 --- [ main] o.s.c.c.c.ConfigServerConfigDataLoader : Located environment: name=simple-microservice, profiles=[default], label=null, version=null, state=null
spring-boot-healthcheck-demo-simple-microservice | 2023-01-29T22:48:22.408Z INFO 1 --- [ main] o.s.cloud.context.scope.GenericScope : BeanFactory id=49a25eee-6804-3b6a-b244-29bc17e80bd9
spring-boot-healthcheck-demo-simple-microservice | 2023-01-29T22:48:22.827Z INFO 1 --- [ main] p.a.h.d.s.SimpleMicroserviceApplication : Started SimpleMicroserviceApplication in 2.162 seconds (process running for 2.544)
spring-boot-healthcheck-demo-simple-microservice exited with code 0
As you can see, all logging lines at the beginning are from spring-boot-healthcheck-demo-config-service
container, and after Started ConfigServerApplication
line you can see logging lines from spring-boot-healthcheck-demo-simple-microservice
container.
Summary
In this article I described how to:
- use Spring Actuator and it’s probes to verify if a service is ready
- use Docker Compose
depends_on
long syntax, and describe dependencies usingcondition
- control containers startup order & conditions in Docker Compose
You can always use tools like jq
to parse Actuators response, or add your own actuator endpoint. Additional health check configuration options like interval etc. are also available. If you don’t want to describe health checks in Compose file, You can also use HEALTHCHECK
instruction inside of Dockerfile.
References:
- GitHub repository: https://github.com/akolata/spring-boot-actuator-docker-health-check
- https://docs.docker.com/engine/reference/builder/#healthcheck
- https://docs.docker.com/compose/startup-order/
- https://docs.docker.com/compose/compose-file/compose-file-v3/#healthcheck
- https://docs.docker.com/compose/compose-file/compose-file-v3/#depends_on
- https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-startup-probes
- https://docs.spring.io/spring-boot/docs/current/reference/html/actuator.html#actuator.endpoints.kubernetes-probes
- https://spring.io/guides/gs/spring-boot-docker/