Microservices service discovery on Oracle Cloud with Spring Cloud and Zookeeper

For a master table-of-contents for blog posts on microservice topics, please refer — https://medium.com/oracledevs/bunch-of-microservices-related-blogs-57b5f1f062e5

This blog demonstrates the following techniques (which are commonly used in microservices ) with help of an example app built on Oracle Cloud

• Service Registry and Discovery
• Synchronous inter-service communication (REST based) with client side load balancing

Be it monoliths or microservices, applications seldom work in isolation. They need to collaborate with other services to fulfill business requirements. In order to do that, the dependent service(s) needs to know the co-ordinates of the others service(s) i.e. host and port details (in most cases)

Static configuration is the simplest possible option(e.g. property files, environment variables etc.) but it does not play well with distributed, cloud based architectures since the co-ordinates are not fixed/predictable due to the elastic & ephemeral nature of these services

There are many problems which crop up, but here are the common/obvious ones

• System downtime: to change the service URIs to factor in the added/removed instances
• Complexity: there are multiple such services (and dependencies) which need to be dealt with.

A widely used pattern/solution to address these problems are Service Registry and Discovery

• Service Registry: applications register themselves to this centralized registry and their life cycle (de-registration etc) is also managed
• Service Discovery: the dependent apps (client apps) query the registry to obtain instances of the services they want to invoke

The sample app in the blog uses projects from the popular Spring Cloud umbrella, specifically Spring Cloud Netflix

• Zookeeper — the central Service Registry component
• Netflix Feign — a declarative REST client for synchronous inter service communication (RPC), and it is
• further enriched by Spring Cloud annotations to activate Ribbon client (another Netflix project). This enables seamless Service Discovery as well as client side load balancing
• there are a couple of (micro) services (simple Spring Boot apps) to demonstrate service discovery in action

the Spring Boot apps run on Oracle Application Container Cloud which is a Cloud Native, Polyglot aPaaS (application platform-as-a-service)

Application Container Cloud | ACC | Oracle Cloud

Architecture & solution deep dive

the sample app is available on Github

abhirockzz/accs-microservices-service-discovery

Here is a high level diagram

• Inventory service registers itself with Zookeeper
• Product service fetches inventory service co-ordinates from Zookeeper
• All the communication happens over an internal/private overlay network i.e. all the apps use the “isClustered” : “true” setting in manifest.json

More details on the ‘clustering’ magic in the documentation — TL;DR is that it setups a private network for internal communication between the applications

Inventory

As mentioned above, its a regular Spring Boot web app (@RestController) which exposes a REST endpoint to fetch inventory/stock for an item (the implementation itself is slightly contrived, but its simple enough to demonstrate the concept)

Here are the most important points about this service

• @EnableDiscoveryClient ensures that it automatically registers itself with Zookeeper
• It’s a Worker application i.e. it does not have a public URL. It is an internal service which is only required to be accessed by other (possibly public facing) apps

here is an introductory blog on Worker applications

• it uses a custom ZookeeperDiscoveryProperties bean to configure aspects of interaction with Zookeeper
• the service is configured to advertise its IP address to Zookeeper and the IP info is obtained using the application name (in this case its InventoryService)

InetAddress inetAddress = Address.getByName( System.getenv(“ORA_APP_NAME”));
hostIP = inetAddress.getHostAddress();

Product

This is yet another Spring Boot web app. It acts as a client to the inventory service

• Uses a @FeignClient annotated interface which defines the interface for the inventory service REST endpoint
• Uses @EnableFeignClients to activate Ribbon based features which automatically enables the feign client to discover app instances from Zookeeper (based on name) and load balance the invocations across all the instances

Zookeeper

As mentioned above, Zookeeper is the central service registry and is deployed as a separate infrastructure component and is reachable from the aforementioned services running on Application Container Cloud

Build & deployment

Start by fetching the project from Github— git clone https://github.com/abhirockzz/accs-microservices-service-discovery

Build

Inventory service

• cd inventory
• mvn clean install

The build process will create inventory-dist.zip in the target directory

Product service

• cd product
• mvn clean install

The build process will create product-dist.zip in the target directory

Deployment a.k.a push to cloud

With Oracle Application Container Cloud, you have multiple options in terms of deploying your applications. This blog will leverage PSM CLI which is a powerful command line interface for managing Oracle Cloud services

other deployment options include REST API, Oracle Developer Cloud and of course the console/UI

You can download and setup PSM CLI on your machine (using psm setup) — details here

Deploy both the applications

Once executed, an asynchronous process is kicked off and the CLI returns its Job ID for you to track the application creation

Before you start deploying the services, you need to make sure that

• you have Zookeeper running and accessible from Oracle Application Container Cloud
• update the deployment.json (for both apps) to enter the Zookeeper info

{
 “memory”: “2G”,
 “instances”: 1,
 “environment”: {
 “ZOOKEEPER”: “<as per your setup>”
 }
}

• Inventory service — psm accs push -n InventoryService -r java -s hourly -m manifest.json -d deployment.json -p target/inventory-dist.zip
• Product service — psm accs push -n ProductService -r java -s hourly -m manifest.json -d deployment.json -p target/product-dist.zip

You should see both the services on the Application Container Cloud application page

Spring Boot apps deployed to App Container Cloud

Confirm service registration

Before we move on, let’s inspect Zookeeper to confirm that the Inventory and Product services have been registered and look at the relevant details

You can download Zookeeper from here and then use its CLI — bin/zkCli.cmd

By default, the information is stored in the /services path — ls /services will show that both the services have indeed been registered

ACCS service registration in Zookeeper

Let’s just look at the Inventory Service details since that’s the one which will be ultimately discovered (and invoked) by the Product Service — digging in further into the inventory service using ls /services/InventoryService , you will see that a unique instance has been registered

ACCS Inventory service instance in Zookeeper

The unique instance ID is created using a format which is a combination of the application name (obtained by ORA_APP_NAME environment variable) and the runtime (Docker) container name (APAAS_CONTAINER_NAME variable) in Application Container Cloud

We can fetch exactly which info has been registered — get /services/InventoryService/<instance_id>

Instance information in Zookeeper

The JSON snippet (which got cropped from the above snippet) contains the required info — we are specifically interested in the host (IP) and port. Here is the JSON payload for your reference (notice the highlighted parts)

Service registration (JSON) payload

Another important thing to note is the value for address (10.44.0.1 in this case) — it is the internal IP of the Inventory Service. Since we have configured private overlay network communication between services, the Product Service will be able to invoke Inventory Service using this IP

Test drive

Alright, everything is set — we can now play around

Sanity test

Let’s start off with a simple test by invoking the Product service endpoint

e.g. curl -X https://ProductService-ocloud100.apaas.us2.oraclecloud.com/product/iPhoneX

You should get a JSON response. Note the node attribute (kind of odd) which has been added on purpose to show which app instance (container) processed the request — you will get a different (randomly generated) inventory count each time you invoke the service

If you were following carefully, you should have notices that the value of the node attribute was in fact a part of the service instance ID for the Inventory service in Zookeeper (refer the above picture to check this again)

Product service response

Scale out

Let’s bump up our Inventory service to two instances and then see how the entire flow (service registration, service discovery and client side load balancing) again — this time with multiple instances of the target service

Scale out the Inventory service

Let’s check Zookeeper again — ls /services/InventoryService

New ACCS app instance added to Zookeeper service registry

Notice the additional/new instance (ID is cropped) we just added as a result of the scale out

Now we can access the Product service again (a few times) — e.g. curl -X https://ProductService-ocloud100.apaas.us2.oraclecloud.com/product/motoZ

Response from Product service

Notice the highlighted part again — the value of the node attribute points to the new instance which we spawned (which gt registered in Zookeeper). If you keep invoking the Product service, you will see that it balances between the two instances of the Inventory service

Scale down

Once you scale back the Inventory Service app, you will be left with one entry in Zookeeper

Quick recap

• we deployed a couple of Spring Boot apps which registered themselves to Zookeeper — service registry in action
• we were able to see service discovery in action when the Product Service was able to invoke the Inventory Service by figuring out its co-ordinates from Zookeeper
• the invocations from the Product Service were load balanced across multiple instances of the Inventory Service

Additional considerations

These were not covered in this blog but are worth mentioning

• Polyglot support — Spring Cloud Zookeeper is restricted to Java apps (if not Spring alone). Generally, a multi-language support is obtained by sidecar (typically REST based) or a language specific binding
• Spring Cloud also supports other service discovery back ends like Consul, Netflix Eureka etc.

Alright, that’s all for this blog post !

Don’t forget to…

• check out the tutorials for Oracle Application Container Cloud — there is something for every runtime!

Oracle Application Container Cloud Service — Create Your First Applications

• other blogs on Application Container Cloud

Latest stories and news about App Container Cloud — Medium

Cheers!

The views expressed in this post are my own and do not necessarily reflect the views of Oracle.