Spring Cloud Config Server Composite Configuration (JDBC + Redis + AWSS3)
This story focuses on composite configuration in spring cloud config with backend as Jdbc (Postgres), Redis and S3. When I am implementing this I didn’t find enough details available on internet with Redis, JDBC and S3. But a lot are available on GIT as backend. So I am not going to talk about GIT here.
High level information about Spring cloud config server
The idea is that our program or programs can move your settings to an external place so that our application is easily configurable and can even change their settings.
This is widely used in microservices. A same service or application can be launched many times in different containers, and it is interesting to have a central place where they can read the settings to these services.
- Spring cloud config is used as central location for all services configuration details in a micro service architecture
- It needs backend to support.
Different types of backend that it supports
File-Base
With file-based (git, svn, and native, bit-bucket) repositories, resources with file names in application*
(application.properties, application.yml, application-*.properties, and so on) are shared between all client applications. You can use resources with these file names to configure global defaults and have them be overridden by application-specific files as necessary.
Vault-server
When using Vault as a backend, you can share configuration with all applications by placing configuration in secret/application.
For example, if you run the following Vault command, all applications using the config server will have properties.
JDBC
Spring Cloud Config Server supports JDBC (relational database) as a backend for configuration properties.
You can enable this feature by adding spring-jdbc to the classpath and using the jdbc profile or by adding a bean of type JdbcEnvironmentRepository. If you include the right dependencies on the classpath (see the user guide for more details on that), Spring Boot configures a data source.
It supports Postgres, Mysql and other RDBMS and NoSql.
Redis
Spring Cloud Config Server supports Redis as a backend for configuration properties.
You can enable this feature by adding a dependency to Spring Data Redis.
AWS S3
Spring Cloud Config Server supports AWS S3 as a backend for configuration properties.
You can enable this feature by adding a dependency to the AWS Java SDK For Amazon S3.
How Spring Profiles can be used to enable spring-cloud-config?
We have to enable either of above profiles in config-server application so that config server knows that it is using one of backend to support cloud config.
When we use spring-cloud dependencies, we can use bootstrap.properties or yml file. The significance of this file is it loads even before spring boot application.
Let us create spring-cloud composite application which has JDBC, Redis and AWSS3 as its profile. The spring cloud service is backed by not one but three backend systems.
bootstrap.yml
For now let us forget about bus.enabled property. I will talk on this at the end of the topic.
application.properties
server.port=8090
spring.datasource.hikari.connection-timeout=5000
spring.datasource.hikari.maximum-pool-size=10
spring.datasource.driver-class-name=org.postgresql.Driver
spring.datasource.url=jdbc:postgresql://localhost:5432/db-config?useSSL=false
spring.datasource.username=developer
spring.datasource.password=postgres
spring.jpa.show-sql=true
spring.redis.host=localhost
spring.redis.port=6379
management.endpoints.web.exposure.include=*I have enabled all the endpoints of actuator just for demo. We need few of them to refresh when there is a change in server to notify the clients.
We need to have following dependencies in the pom file in addition to spring config dependencies
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-data-jdbc</artifactId>
</dependency>
<dependency>
<groupId>com.amazonaws</groupId>
<artifactId>aws-java-sdk-s3</artifactId>
</dependency>
<dependency>
<groupId>org.postgresql</groupId>
<artifactId>postgresql</artifactId>
<scope>runtime</scope>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-data-redis</artifactId>
</dependency>As we are trying to do involve multiple backend systems we need all the respective dependencies here.
JDBC -> Postgres as backend
We need to create a table “properties” with below structure and insert data in to the table.
create table PROPERTIES (
id serial primary key,
CREATED_ON timestamp ,
APPLICATION text,
PROFILE text,
LABEL text,
PROP_KEY text,
VALUE text
);We will insert two records with
application name as “spring-config-jdbc-client”
profile as “dev”
label as “latest”
INSERT INTO properties (CREATED_ON, APPLICATION, PROFILE, LABEL, PROP_KEY, VALUE) VALUES (NULL,'spring-config-jdbc-client','dev','latest','configA','Config A');
INSERT INTO properties (CREATED_ON, APPLICATION, PROFILE, LABEL, PROP_KEY, VALUE) VALUES (NULL,'spring-config-jdbc-client','dev','latest','configB','Config B - Dev');JDBC works with key pattern <application-name>/<profile>/<label>.
In this case spring-config-jdbc-client/dev/latest. To see the values in config server we have to invoke api in server with <base-uri>/spring-config-jdbc-client/dev/latest.
Redis as backend
Redis used HMSET collection to store config values data. Here also it follows same structure.
<application-name>-<profile>-<label> in HMSET.
HMSET spring-redis-client-config-dev config "Config Redis Dev" configb "Config Redis B Dev"Here label is not used.
When we want to see the config properties from server we have to invoke <base-uri>/spring-redis-client-config/dev to load data.
AWS S3 as backend
For S3, we need bucket name. If we want to use existing bucket with sub folders, we have to pass full canonical path for the folder in application config.
It naming convention followed is {application-name}-{profile}.properties/yml/json. This is the convention followed.
How it works?
Let us start the server here. Now that in server we have three different backend systems available. And we have inserted the details here
We can directly check in config server whether the profile is available or not. This is supported by spring config server dependency. It has implicit RestControllers that will expose the property sources.
The context path is <base-uri>/<application-name>/<profile>/<label if applicable>
Let us check for JDBC Backend. Our URL will be http://localhost:8090/spring-config-jdbc-client/dev/latest
Let us check for REDIS Backend. Our URL will be http://localhost:8090/spring-redis-client-config/dev
Let us check for S3 Backend. Our URL will be http://localhost:8090/spring-config/dev
Configure clients to use these configurations
JDBC-Client:
First define bootstrap.properties
spring.cloud.config.uri=http://localhost:8090
#spring.cloud.config.profile=dev
spring.cloud.config.label=latest
spring.profiles.active=devSampleController.java with RefreshScope and config values. We haven’t defined configC in table. So we will define it in application.properties as “configC = Config C — static”. So application will not stop. I will create another tutorial to use implicit encryption mechanism to encrypt at rest.
If you have observed we have used @Value annotation. We can also use ConfigProperties annotation as well. Spring cloud config has implicit functions to load the Value beans.
Let us start and test this application.
The controller class remain same so we will start the respective clients and hit the urls.
For Redis :
For AWS S3:
Please find source code from here
References:
