Spring Cloud Config Server Composite Configuration (JDBC + Redis + AWSS3)

Eresh Gorantla
Jan 12 · 6 min read

This story focuses on composite configuration in spring cloud config with backend as Jdbc (Postgres), Redis and S3. When I am implementing this I didn’t find enough details available on internet with Redis, JDBC and S3. But a lot are available on GIT as backend. So I am not going to talk about GIT here.

High level information about Spring cloud config server

The idea is that our program or programs can move your settings to an external place so that our application is easily configurable and can even change their settings.

This is widely used in microservices. A same service or application can be launched many times in different containers, and it is interesting to have a central place where they can read the settings to these services.

  • Spring cloud config is used as central location for all services configuration details in a micro service architecture
  • It needs backend to support.

Different types of backend that it supports

File-Base

With file-based (git, svn, and native, bit-bucket) repositories, resources with file names in application*
(application.properties, application.yml, application-*.properties, and so on) are shared between all client applications. You can use resources with these file names to configure global defaults and have them be overridden by application-specific files as necessary.

Vault-server

When using Vault as a backend, you can share configuration with all applications by placing configuration in secret/application.
For example, if you run the following Vault command, all applications using the config server will have properties.

JDBC

Spring Cloud Config Server supports JDBC (relational database) as a backend for configuration properties.
You can enable this feature by adding spring-jdbc to the classpath and using the jdbc profile or by adding a bean of type JdbcEnvironmentRepository. If you include the right dependencies on the classpath (see the user guide for more details on that), Spring Boot configures a data source.
It supports Postgres, Mysql and other RDBMS and NoSql.

Redis

Spring Cloud Config Server supports Redis as a backend for configuration properties.
You can enable this feature by adding a dependency to Spring Data Redis.

AWS S3

Spring Cloud Config Server supports AWS S3 as a backend for configuration properties.
You can enable this feature by adding a dependency to the AWS Java SDK For Amazon S3.

How Spring Profiles can be used to enable spring-cloud-config?

We have to enable either of above profiles in config-server application so that config server knows that it is using one of backend to support cloud config.

When we use spring-cloud dependencies, we can use bootstrap.properties or yml file. The significance of this file is it loads even before spring boot application.

Let us create spring-cloud composite application which has JDBC, Redis and AWSS3 as its profile. The spring cloud service is backed by not one but three backend systems.

bootstrap.yml

For now let us forget about bus.enabled property. I will talk on this at the end of the topic.

application.properties

server.port=8090

spring.datasource.hikari.connection-timeout=5000
spring.datasource.hikari.maximum-pool-size=10
spring.datasource.driver-class-name=org.postgresql.Driver
spring.datasource.url=jdbc:postgresql://localhost:5432/db-config?useSSL=false
spring.datasource.username=developer
spring.datasource.password=postgres
spring.jpa.show-sql=true

spring.redis.host=localhost
spring.redis.port=6379

management.endpoints.web.exposure.include=*

I have enabled all the endpoints of actuator just for demo. We need few of them to refresh when there is a change in server to notify the clients.

We need to have following dependencies in the pom file in addition to spring config dependencies

<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-data-jdbc</artifactId>
</dependency>

<dependency>
<groupId>com.amazonaws</groupId>
<artifactId>aws-java-sdk-s3</artifactId>
</dependency>

<dependency>
<groupId>org.postgresql</groupId>
<artifactId>postgresql</artifactId>
<scope>runtime</scope>
</dependency>

<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-data-redis</artifactId>
</dependency>

As we are trying to do involve multiple backend systems we need all the respective dependencies here.

JDBC -> Postgres as backend

We need to create a table “properties” with below structure and insert data in to the table.

create table PROPERTIES (
id serial primary key,
CREATED_ON timestamp ,
APPLICATION text,
PROFILE text,
LABEL text,
PROP_KEY text,
VALUE text
);

We will insert two records with

application name as “spring-config-jdbc-client”

profile as “dev”

label as “latest”

INSERT INTO properties (CREATED_ON, APPLICATION, PROFILE, LABEL, PROP_KEY, VALUE) VALUES (NULL,'spring-config-jdbc-client','dev','latest','configA','Config A');
INSERT INTO properties (CREATED_ON, APPLICATION, PROFILE, LABEL, PROP_KEY, VALUE) VALUES (NULL,'spring-config-jdbc-client','dev','latest','configB','Config B - Dev');

JDBC works with key pattern <application-name>/<profile>/<label>.

In this case spring-config-jdbc-client/dev/latest. To see the values in config server we have to invoke api in server with <base-uri>/spring-config-jdbc-client/dev/latest.

Redis as backend

Redis used HMSET collection to store config values data. Here also it follows same structure.

<application-name>-<profile>-<label> in HMSET.

HMSET spring-redis-client-config-dev config "Config Redis Dev" configb "Config Redis B Dev"

Here label is not used.

When we want to see the config properties from server we have to invoke <base-uri>/spring-redis-client-config/dev to load data.

AWS S3 as backend

For S3, we need bucket name. If we want to use existing bucket with sub folders, we have to pass full canonical path for the folder in application config.

It naming convention followed is {application-name}-{profile}.properties/yml/json. This is the convention followed.

How it works?

Let us start the server here. Now that in server we have three different backend systems available. And we have inserted the details here

Image for post
Image for post

We can directly check in config server whether the profile is available or not. This is supported by spring config server dependency. It has implicit RestControllers that will expose the property sources.

The context path is <base-uri>/<application-name>/<profile>/<label if applicable>

Let us check for JDBC Backend. Our URL will be http://localhost:8090/spring-config-jdbc-client/dev/latest

Let us check for REDIS Backend. Our URL will be http://localhost:8090/spring-redis-client-config/dev

Image for post
Image for post

Let us check for S3 Backend. Our URL will be http://localhost:8090/spring-config/dev

Image for post
Image for post

Configure clients to use these configurations

JDBC-Client:

First define bootstrap.properties

spring.cloud.config.uri=http://localhost:8090
#spring.cloud.config.profile=dev
spring.cloud.config.label=latest
spring.profiles.active=dev

SampleController.java with RefreshScope and config values. We haven’t defined configC in table. So we will define it in application.properties as “configC = Config C — static”. So application will not stop. I will create another tutorial to use implicit encryption mechanism to encrypt at rest.

If you have observed we have used @Value annotation. We can also use ConfigProperties annotation as well. Spring cloud config has implicit functions to load the Value beans.

Let us start and test this application.

Image for post
Image for post

The controller class remain same so we will start the respective clients and hit the urls.

For Redis :

Image for post
Image for post

For AWS S3:

Image for post
Image for post

Please find source code from here

The Startup

Medium's largest active publication, followed by +754K people. Follow to join our community.

Eresh Gorantla

Written by

Experience in Open source development, Technical Leader. Expert in Java/J2EE, Integration, analytics. Loves Cricket, cooking, movies and travelling.

The Startup

Medium's largest active publication, followed by +754K people. Follow to join our community.

Eresh Gorantla

Written by

Experience in Open source development, Technical Leader. Expert in Java/J2EE, Integration, analytics. Loves Cricket, cooking, movies and travelling.

The Startup

Medium's largest active publication, followed by +754K people. Follow to join our community.

Medium is an open platform where 170 million readers come to find insightful and dynamic thinking. Here, expert and undiscovered voices alike dive into the heart of any topic and bring new ideas to the surface. Learn more

Follow the writers, publications, and topics that matter to you, and you’ll see them on your homepage and in your inbox. Explore

If you have a story to tell, knowledge to share, or a perspective to offer — welcome home. It’s easy and free to post your thinking on any topic. Write on Medium

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store