Building microservices with Netflix OSS, Apache Kafka and Spring Boot — Part 1 : Service registry and Config server

• Building microservices with Netflix OSS, Apache Kafka and Spring Boot — Part 1 : Service registry and Config server
• Building microservices with Netflix OSS, Apache Kafka and Spring Boot — Part 2 : Message Broker and User service
• Building microservices with Netflix OSS, Apache Kafka and Spring Boot — Part 3 : Email service and Gateway
• Building microservices with Netflix OSS, Apache Kafka and Spring Boot — Part 4 : Security

As Microservice Architecture projects gained popularity the last years, a possibility to work on such project logically appeared in our team. Knowing the process of building microservices becomes a must. In the following lines I’m going to summarize the knowledge I got through the last few months of research and development of microservice projects.

Using Netflix OSS, Apache Kafka and Spring Boot, I’m going to build a relatively simple system mixing the two main ways for communication between microservices: REST and messaging. You can find all projects that I will build in Git

The solution

The goal is to build a registration system that will send a confirmation mail after new user is registered. Following components will be part of this system:

1. Service registry (Eureka) — Where all services will register themselves

2. Config server (Spring Cloud Config) — Where all services will take their configurations from. Config server will keep configuration files in git repository

3. Gateway (Zuul) — that will redirect all the requests to the needed microservice

4. User service — using this one the new users will register. On every new registration the User service will send a message “USER_REGISTERED” to the message broker (Kafka)

5. Email service — using this one we will send emails. On “USER_REGISTERED”message received the Email service will send a confirmation email to the new user

Service registry (Eureka)

In an environment with lots of services able to communicate with each other, they will need to know each other (their network addresses). At first glance you may think that keeping this in the configuration files may do the job. And it will, if you stick with 2, 3 or 4 services. But maintaining 30, 40 or even 100+ services will bring you a configuration hell. And with cloud‑based microservice applications this will get even worse.

Here the service registry comes to help. Its purpose is to store the network locations of service instances. Netflix Eureka is good example for such a registry where service instances register to the registry. Eureka for its part pings the registered services every 30 seconds to verify they are up and running.

Implementing such registry with Spring Boot and Netflix OSS stack is an easy task. Create new project (ms-discovery) with SPRING INITIALIZR and add “Eureka Server” dependency. We should obtain spring-cloud-starter-eureka-server dependency in the pom.xml automatically.

/pom.xml

Then just add the @EnableEurekaServer annotation in the main application *Application.java file. this annotation will activate the Eureka Server related configuration

/MsDiscoveryApplication.java

And finally tune the configuration files. In the bootstrap.yml put only the name and the running port of the application.

/bootstrap.yml

In the application.yml put standard application configuration. register-with-eureka and fetch-registry will prevent the registry complaining that there are no replica nodes for it to connect to. You will need to change these when going to prod but for local testing it is okay.

/application.yml

‘short’ command line arguments

If you prefer using shorter commands to set configuration properties on the command line, you can use placeholders ${port:8761}:

java -jar ms-discovery-0.0.1-SNAPSHOT.jar — port=3232

instead of the standard

java -jar ms-discovery-0.0.1-SNAPSHOT.jar — server.port=3232

To test if everything is fine, build and run the project

1. mvn clean install
2. go to the target folder
3. run java –jar ms–discovery–0.0.1–SNAPSHOT.jar
4. Check Eureka running on http://localhost:876

Config server (Spring Cloud Config)

Any enterprise application will run on different environments (DEV, QA, PROD). And to modify property files before the deploy is not welcomed. One of the solutions is the so cold externalized configuration pattern where the configurations are read from external source. Pivotal’s implementation of this pattern is the Spring cloud config server. It works the following way: property files for each service and each environment are kept in git repository. On startup every service is asking the config server for the proper configurations needed.

In the current solution we are going to build the cloud config server as a discovery service client. So on startup each microservice will take the config server location from the discovery service. Again with the help of SPRING INITIALIZR create new project (ms-config-server) and add the “Config Server” and “Eureka Discovery” dependencies

Except the ms-config-server we need the git repository with the config files. We will name it ms-config-properties and will have a tree structure containing the property files for every microservice and for every environment like:

ms-config-properties 
    ms-user
        default
            ms-user.yml
        dev
            ms-user.yml
        prod
            ms-user.yml

Again automatically in the pom we should have spring-cloud-starter-eureka and spring-cloud-config-server

/pom.xml

We need to modify the main *Application.java file adding @EnableEurekaClient annotation to enable Eureka discovery for clients. By default it will look for Eureka server running onhttp://localhost:8761 to register. We should also enable the config server that other applications will talk to adding the @EnableConfigServer annotation

/MsConfigServerApplication.java

By default the config server runs on port 8888.

/bootstrap.yml

In the application properties we configure the uri to the git repository and the way how the config server search in the repository. They are going to be kept in ms-config-properties subfolder of the repository so:

/application.yml

To test if everything is fine, build and run the project

1. mvn clean install
2. go to the target folder
3. run java –jar ms–config-server–0.0.1–SNAPSHOT.jar
4. Check Config server will return some of the configurations we have in ms-config-properties folder: http://localhost:8888/ms-service/dev
5. Check Config server managed to register to the eureka discovery

You can check Part 2, Part 3 and Part 4 : Security of ‘Building microservices with Netflix OSS, Apache Kafka and Spring Boot’.

Originally published at dreamix.eu on October 24, 2017.