Solving the Configuration Management Challenge using Spring Cloud Config Server & Vault

Spring Boot is a fantastic framework for developing Java micro services. The Spring Cloud project incorporates modules providing implementations of common patterns required to implement distributed cloud native systems effectively. These patterns include: configuration management, service discovery & circuit breakers.

Spring Boot and Spring Cloud combined are great options for developing cloud native java micro services architectures. This post is the first in a series illustrating the capabilities provided by the Spring Cloud project using the Hoxton release.

The Configuration Management Challenge

One of the common challenges faced in implementing micro services architectures is how to achieve externalised configuration management. Simply embedding configuration hardcoded in a deployable component or container presents security and operational challenges.

In distributed systems architecture we have a need for a secure external centralised configuration management solution capable of supporting multiple environments & services. This solution needs to support the ability to change configuration without incurring significant downtime. Furthermore devising a cloud provider agnostic solution can be important.

Spring Cloud Config Server Background

The Spring Cloud Config module provides support for the creation of config servers and config clients. It fits nicely with the existing Spring Environment and PropertySource concepts. It supports the creation of config servers backed by a range of external providers including: file system based, git repository, Vault & S3.

Diagram illustrating a typical micro services architecture utilising Spring Config Server

In this post we explore how to introduce external centralised configuration management into a micro services architecture making use of a Spring cloud config server backed by a Vault backend.

Diagram illustrating the micro services architecture utilising Spring Config Server & Vault built in this post

In order to build the simple architecture indicated above 3 key phases will be followed:

• Creation & Configuration of a Vault server.
• Implementation of a Spring Cloud Config Server.
• Implementation of a Message Service Spring Cloud config client.

Why Use Vault?

Vault is a tool for securely accessing secrets. A secret is anything that you want to tightly control access to, such as API keys, passwords, certificates, and more. Vault provides a unified interface to any secret, while providing tight access control and recording a detailed audit log. (Source: Hashicorp Vault docs)

The features provided by Vault meet our requirements of providing secure storage and fine grained access to configuration within our distributed systems architecture.

Setting Up Vault

On a mac Vault can be installed via brew:

brew install hashicorp/tap/vault

Instructions for installing Vault on alternative platforms are provided here.

Post installation we now need to start a development server. This can be achieved by running:

vault server -dev

Confirmation that the dev server is running will be provided in the terminal output. The output will include a suggestion to export the VAULT_ADDR and details of the root token for the sev server.

Start a new terminal and run the suggested export VAULT_ADDR command e.g:

export VAULT_ADDR='http://127.0.0.1:8200'

We can now verify the status of our dev server from the new terminal:

vault status

Adding Configuration

Having setup our Vault dev server we now need to add the required configuration for our application.

Vault allows secrets to be added using the vault kv put command. From the terminal run the following commands:

vault kv put secret/application message.subject="Hello World"
vault kv put secret/message-service message.text="A special message"
vault kv put secret/message-service,dev message.text="A very special message"

Tip: Note storing a secret using the syntax ‘secret/message-service,dev’ allows us to store a property specific to the message-service dev profile.

To verify the secrets have been stored correctly we can retrieve the secrets using the vault kv get command from the terminal as follows:

vault kv get secret/application
vault kv get secret/message-service
vault kv get secret/message-service,dev

It is also possible to retrieve specific fields:

vault kv get -field=message.text secret/message-service

Defining access control

In order to ensure secure managed access to our Vault based configuration we need to create a policy and a token.

Create a file called config-server-policy.hcl defining a fine grained read only policy with the following contents:

path "secret/data/message-service" {
  capabilities = ["read"]
}
path "secret/data/message-service,dev" {
  capabilities = ["read"]
}
path "secret/data/application" {
  capabilities = ["read"]
}
path "secret/data/application,dev" {
  capabilities = ["read"]
}

This policy will ensure that our message service will only have access to configuration specific to itself (message-service) and general system wide configuration (application).

Run the following command to create a policy:

vault policy write config-server-policy config-server-policy.hcl

Finally to create a token providing access based on our policy rules run the following command:

vault token create -policy=config-server-policy

(Make a note of the token created as indicated in the terminal output)

This token will be passed by our micro service config client to our config server and in turn then passed in calls to Vault to ensure managed access to our configuration.

Tip: If we were to extend the use of our Config server beyond one micro service client we could introduce further fine grained access control policies per service. Each policy would provide access to common properties and properties relevant to the specific service in a similar to the approach used above.

Creating a Config Server

Having setup our Vault backend we are now going to create our Config server which will take the form of a Spring Boot service with the spring-cloud-config-server dependency.

Bootstrap Project with Spring Initializer

In order to bootstrap the creation of our config server project we can configure Spring Initializer with the following settings:

The key addition illustrated above is the inclusion of the spring-cloud-config-server dependency.

Having configured the initializr settings as desired utilise the ‘Generate’ option to create a bootstrapped project and import the skeleton project into your favourite IDE.

Add @EnableConfigServer annotation to our Application class

Next we need to add the @EnableConfigServer annotation to our Application class as illustrated below:

Enable Vault backend

As mentioned earlier the Spring cloud config server supports multiple backend providers for configurations. In order to utilise the Vault as the backend provider we need to set the spring.profiles.active property to vault within our application.properties file as shown by the screenshot below:

NB Also importantly the spring.cloud.config.server.vault.kvVersion property has been set to 2. The default value for this property is 1. However with Vault servers > v0.10.0 the kv api version has been updated so we need to update this property to 2 otherwise our config server will be unable to retrieve configuration via our Vault server.

To illustrate how to configure the server host the spring.cloud.config.server.vault.host has also been set.

Startup Config Server Application

It is now necessary to run your config server Spring Boot application either via the command line or via your IDE.

Verify the Config Server Application is running correctly

Having started the config server application we can verify the server is running as expected by triggering a get request to retrieve the configuration for the ‘message-service’ with the dev profile. This will lead to 3 property sources being returned:

1. ‘message-service’ ‘dev’ properties
2. ‘message-service’ ‘default’ properties
3. system wide ‘application’ properties

In order to authenticate our request we need to provide a Vault token. The get request can be triggered using curl:

curl -X "GET" "http://localhost:8888/message-service/dev" -H "X-Config-Token: <your vault token>"

(Note you need to substitute your vault token into the command above as captured earlier in the Vault setup section.)

You should receive a response json object including the configuration as stored earlier in our Vault backend store including 3 property sources returned in order of precedence :

Creating a Config server client

Having created our config server we will now create a Spring Boot message-service config server client. This service will expose a REST service comprised of 1 simple endpoint.

Bootstrap Project with Spring Initializer

Again we can utilise Spring Initializr to bootstrap our new service. The key step is the inclusion of the spring-cloud-starter-config dependency and the spring-boot-starter-web dependency.

Having adjusted the Spring Initializer config as indicated above, ‘Generate’ the skeleton project and import into your favourite IDE.

Add bootstrap.properties file

In order to instruct our service to load it’s configuration from our config server at startup we need to create a bootstrap.properties file and add the following properties (substituting the placeholder text with your Vault token):

Note we have set the active spring profile to ‘dev’.

MessageServiceController creation

Next to illustrate the seamless use of the config client we will create a simple new MessageServiceController exposing a GET messages/{id} endpoint whose stubbed implementation will reference 2 configuration properties we added to Vault.

In addition this basic service makes use of a Message POJO:

Run Message service & Call service

Run the message service via your IDE or the command line.

Call the GET messages endpoint using curl:

curl -X "GET" "http://localhost:8080/messages/1"

The service endpoint will return the following response:

{
    "id": 1,
    "subject": "Hello World",
    "text": "A very special message"
}

The response illustrates the ability to resolve properties in order of precedence from both the service specific message-service property sources and the shared application property source as a client of a Vault backed Config server.

Conclusion

In this session we have created a secure, centralised and cloud provider agnostic configuration management server using Spring Boot & Spring Cloud Config backed by a Vault backend.

In follow up posts in this series we will explore further Spring Cloud modules.

Source Code

The source code for the modules created in this post are available here:

https://github.com/cumulo-nimbus/spring-cloud-config-tutorial

References

https://www.hashicorp.com/products/vault

https://cloud.spring.io/spring-cloud-config/reference/html/