Lets Build Microservices — Part 1

This article, Part 1 of the series, uses Java Spring + Consul to build components of a microservice system.

Published in

The Startup

10 min readNov 18, 2019

I have been working on and in the process, learning various aspects of microservices. I will describe in detail various components of a microservice architecture based system and their code as well. I hope this will help you gain more knowledge and also experience to build such systems. I would recommend writing the code and building these components yourself while or after reading this blog.

Building stuff yourself, making mistakes and fixing them is the best way to learn and also remember that learning.

In Part 1, we will build the below components. I also describe further down, why we need these components.

3 Microservices with RESTful endpoints with JSON output
Service discovery using Consul
External configuration using Consul

Code:- https://github.com/amollimaye/microservices-ecommerce

To be able to understand the mentioned code, I assume you have basic knowledge of Java and Spring framework.

Tools & Technologies used: Spring Boot, Spring Cloud, Spring Data, H2 DB, Consul

Part 2 — Dockerize microservices— https://medium.com/@amol.limaye/lets-build-microservices-part-ii-9620c930587

Part 3 —Build NGINX API Gateway — https://medium.com/@amol.limaye/lets-build-microservices-part-iii-20e9e5c780a0

The Flow

About: Explaining the microservice architecture
Step 0: Describing what we will build
Step 1 — 4: Building Various components using Java Spring and Consul
Step 5: How to learn more from what we just built
Finally: What next and some useful resources

About: What is Microservices architecture

Quoting from Martin Fowler’s resourceful microservices website:

The microservice architectural style is an approach to developing a single application as a suite of small services, each running in its own process and communicating with lightweight mechanisms, often an HTTP resource API. These services are built around business capabilities and independently deployable by fully automated deployment machinery. There is a bare minimum of centralized management of these services, which may be written in different programming languages and use different data storage technologies.

The microservices architecture also reflects the organizational structure. Each service is not a project, but a product. A small group of developers owns that product — From development to production deployment and monitoring.

Step 0: What we will build in part I

In part I, we will build the actual micro-services and some cross-service components. In the coming articles of this series, we will build other important cross-service components, described at the end of this article. Following is what we build in this article.

E-Commerce Service vends out a list of e-commerce products like smartphones, TV, watch, etc. It uses Product service and Image service to gather this data.

Product Service is responsible for details of each product, like name, price, and description.

Image Service is responsible for giving images for products.

Service discovery will be done using the consul service discovery client. So, Inter-service calls can be done using only application id, without needing to know other service’s IP and port numbers. This is even more relevant in the age of cloud, where we use virtualized/containerized environments. Here, ports and hosts are dynamically allocated and services could have multiple instances. So, a service instance going down or a new one coming up should not need any manual intervention to update the service registry.

External configuration — We will store the configuration externally to consul KV store. This means we do not have to recompile and redeploy the code when any configuration changes. The configuration will be automatically reloaded. This also enables us to have a different set of properties for different environments.

We will also use spring cloud to enable consul integration with the 3 spring boot based RESTful services listed above.

To keep things simple, we will use H2 in-memory DB. Also, the services will have only one ‘readAll’ endpoint each. In a learning project, keeping the application logic to minimum speeds up building each component and hence helps retain the interest and curiosity in that learning till the end. We can always add more stuff in the next iterations.

Step 1: Product Service

Spring Initializr makes our life easy to build spring boot services. https://start.spring.io/

Select below dependencies.

Spring Web — To create RESTful service using spring boot
Spring Data — Read and return data with almost no code
H2 database — In-memory database
Spring boot actuator — Enables health endpoint needed for service discovery

We will build one restful endpoint in this service ‘/products’ that vends a JSON of products. Below is the product entity POJO.

Below is the RestController that uses ProductRepository

ProductController

ProductRepository

Now we will add the following 2 files in the resources folder for H2 In-memory DB. Spring detects these files and uses them to create the schema and populate it with given data.

schema.sql — Creates the PRODUCT table in H2 DB during startup

data.sql — Inserts the data in PRODUCT table.

schema.sql

data.sql

Below properties are added to application.properties. I also added some explanations for properties.

Once the code is complete, start the service using mvn spring-boot:run

The application should start up and you will see the following response on http://localhost:8090/product-service/products

All work as expected? Great. Let’s move to the next step.

Not working? Compare your code with mine to see what's different.

Step 2: Consul

We will use consul for service discovery and external configuration.

Download consul and install it locally — — https://learn.hashicorp.com/consul/getting-started/install

Start consul locally by browsing to consul folder and typeconsul agent -dev

This is a dev only command for running it on your dev sandbox. Also, we will be running only one agent. For a production env, you would need a cluster of such agents.

Now, we discuss changes to be done in service. First, add consul discovery dependency in product-service.

<dependency>            
<groupId>org.springframework.cloud</groupId>                 <artifactId>spring-cloud-starter-consul-discovery</artifactId>        </dependency>

Also, add below to bootstrap.properties file, which resides in the resources folder. We have added a custom health check path because we have a custom context path. The service health status will be checked every 1 second.

Start product-service with new changes. Refer to complete code in case of any issues.

Hit http://localhost:8500/ui to see consul UI. If all goes well, you will see productApp service registered. Clicking on the service will show service names and more details.

Step 3: Image Service

Now we create image service which vends out image corresponding to a product. Select the following modules in spring initializr.

Spring Web
Spring Data JPA
H2 Database
Spring Boot Actuator
Consul Discovery

Below is the Image Entity

Below are data.sql and schema.sql

schema.sql

data.sql

Adding ImageController and ImageRepository in the same way.

Image Controller

Image Repository

Below are the properties. All of the properties are same as product-service, except ones shown below:

consolidated view of application.properties and bootstrap.properties

When the service is complete and started using, image service will also show up in the consul UI.

Great. Now, to the next and important part.

Step 4: E-Commerce service

This is the service that aggregates Product and Image entities to vend out ECommerceProduct. This service will use consul discovery to call product and image service. The service is created using the same dependencies and properties for the other two services, except below 2 differences.

Add consul configuration dependency.

This allows us to use consul KV store for external configuration of service properties.

2. We won’t use H2 DB since this service is not storing/retrieving any data of its own.

The entity class includes Product and image.

In the git repository code, I have copied the entity classes Product.java and Image.java from respective services into ecommerce-service. A production ready solution could have such common code into a separate library. We can then include those libraries as dependencies in required applications. This will avoid code duplication and rework in case of any changes to those classes.

Inter-service calls using service discovery

Now, we want to read data from 2 services and merge it. To call each service, we will make use of service discovery as below. I have added comments describing how we do it.

It is important to note that we use Service ID instead of physical URLs to call the service endpoints. So, product service will be called using http://productApp/product-service/products , where productApp is the service id, specified by property spring.application.name.

Ribbon is a Netflix library available with Spring cloud. The @ LoadBalanced annotation is used to create a ‘discovery service-aware’ restTemplate, that uses Ribbon to do client-side load balancing. It will use this service Id to pick up a registered instance from consul and get its actual URI as http://localhost:8090/product-service/products .

We merge the responses of product-service and image service using ‘productId’ as a relation between Product and Image entities.

In the controller, we use ProductAssembler to list out all e-commerce products as JSON.

ECommerceController

We would get a response like below for http://localhost:9080/ecommerce-service/ecommerceProducts

So, we have a microservice that calls two other microservices using service discovery and merges their data.

Next, we want to enable or disable attaching images to our eCommerce products. For this, we will use Consul KV store.

External configuration

We will have a property, ‘useImages’. If it is set to true, the images will be attached to the eCommerce product, else not. For that, first, we will define a bean in the application.

Here, we autowire the ‘useImages’ property. We have also annotated the bean as @ RefreshScope . Whenever a key/value in consul changes, a RefreshEventListener detects these changes and reloads the properties into environment variables. The beans annotated @ RefreshScope also get re-initialized with the latest values. This way, redeploy and restart of the application is not needed to reflect the configuration changes.

Next, we enter the property key values in consul. In Consul, the key would be in the folder config/ecommerceApp/ , where ecommerceApp is the app name. So, we create the property as follows

In the consul UI, click ‘Key/Value’ navigation item at the top. This opens up the screen to enter key values.
Enter path config/ecommerceApp/useImages and set the value as true.

As shown below:

Now, we will merge the images into ecommerceProduct only if the ‘useImages’ property is true.

We need to explicitly enable the external configuration in spring using property spring.cloud.config.enabled=true

Another code is similar to two other services.

Once the code is complete, start the ecommerceService. If all goes well, you can see the ‘green’ status of all three services in Consul UI.

Change the useImages property in consul to false. Within a few seconds, see the response of http://localhost:9080/ecommerce-service/ecommerceProducts The images will not be attached to ecommerceProduct anymore.

You can also try adding more consul k/v properties and see them reflect in your application by hitting http://localhost:9080/ecommerce-service/actuator/env endpoint. This endpoint is disabled by default. To enable this and every other actuator endpoint, add property management.endpoints.web.exposure.include=*

Please keep in mind, we have used a single instance of consul, which is not recommended for product environment. Also, we are using KV store of consul which is in-memory. This means, when consul is restarted, all the key/values in it will be gone. A solution like git2consul can be used to externalize the properties into GitHub repository.

Understanding more about what we just built

Now we have one microservice, that reads data from two other microservices and aggregates that data as a JSON response. Consul is being used for inter-service communication, and also for health check of the service instances.

Consul UI shows the live health status of these services. We are now able to control the behavior of one service using external configuration stored in the consul key-value store, without a need to redeploy and restart that service.

Below are some pointers to learn more about various components we used in part I. Try to find answers to the below questions.

Spring Boot — Add environment-specific properties. Say, ‘dev’ and ‘stage’. Will new property files need to be added? Will you need to do changes in the configuration classes? What changes in consul property names would be needed? Also, explore various Actuator 2.0 endpoints to gain a deeper understanding of the application and various properties in it.
Spring Data — Replace the H2 in-memory DB with an On-disk database like postgreSQL for product-service.
Spring Cloud — Create two instances of product-service, both refer to the same on-disk DB. See how @ LoadBalanced restTemplate works in that case.
Consul discovery — If we have 2 instances of one service, how does the discovery server behave when one instance goes down? Do the calls flow seamlessly even when on service stopped? How much time went by before the status is updated in consul.
Consul Key-Value store — As mentioned previously, we have used the in-memory K/V store. Use git2consul so that key/values are stored in git.