Kubernetes — Microservices Discovery
In non-Kubernetes-based projects, we have a discovery server (like the Eureka service) to take care of service discovery. So I was curious to know which component we should use in Kubernetes. I did search on internet to check if any article covers discovery with upstream microservice calling downstream microservice and found none; hence thought to write this article.
The following are the prerequisites:
- Spring Boot version: 2.7.3
- Minikube version: v1.25.1
- Kubectl: v1.22.5
In this article, I will explain how microservices discover each other in the Kubernetes (K8s) environment. I will demo it using SpringBoot-based microservices (Upstream calls downstream using a Feign Client).
How does Kubernetes supports discovery?
Discovery in Kubernetes is supported in two ways.
1. Environment variables
Kubernetes adds {SVCNAME}_SERVICE_HOST
and {SVCNAME}_SERVICE_PORT
environment variables for each pod created in a cluster. The following is an example.
PAYMENT_SERVICE_SERVICE_HOST=10.108.155.131
PAYMENT_SERVICE_SERVICE_PORT=8080
TICKET_SERVICE_SERVICE_PORT=8080
TICKET_SERVICE_SERVICE_HOST=10.108.155.131
In Feign Client, use these variables to make service URLs, as shown below.
url = ${PAYMENT_SERVICE_SERVICE_HOST}:${PAYMENT_SERVICE_SERVICE_PORT}
This approach is not commonly used as it has one problem: if the client Pod gets created before the service, the service environment variables are unavailable. So you must create the Service before the client Pods come into existence. Otherwise, those client Pods won't have their environment variables populated.
2. Discovery using CoreDNS
Each Kubernetes cluster has a CoreDNS pod which basically acts as a DNS server. It watches the Kubernetes API for new services and creates sets of DNS records for each service.
Now let's see practically how this happens. We can see that the CoreDNS pod is available out of the box by querying, as shown below.
kubectl get pods -n kube-system
To get data about the CoreDNS service, use the following command.
kubectl get service -n kube-system
When one microservice wants to connect with other microservices, they do a lookup using the CoreDNS service.
To see how this works let's create two simple SpringBoot-based projects.
1. ticket-service
(upstream)
2. payment-service
(downstream)
Add the following dependencies in pom.xml
.
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-openfeign</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-kubernetes-fabric8-all</artifactId>
</dependency>
- Step 1: Create the
ticket-service
application and add the below code:
@SpringBootApplication
@RestController
@EnableFeignClients
@EnableDiscoveryClient
@Slf4j
public class TicketServiceApplication {
@Autowired
IPayment iPayment;
public static void main(String[] args) {
SpringApplication.run(TicketServiceApplication.class, args);
}
@PostMapping("/ticket/pay")
public String pay(){
log.info("Calling payment service");
//wallet payment
String paymentResponse = iPayment.pay();
log.info("Payment done..");
return paymentResponse + ", Ticket confirmed!";
}
}
Add IPayment
client to the ticket-service
project as shown below:
@FeignClient(name="payment-service")
public interface IPayment {
@PostMapping("/pay")
public String pay();
}
- Step 2: Create a
payment-service
application and add below code:
@SpringBootApplication
@RestController
@EnableDiscoveryClient
public class PaymentServiceApplication {
public static void main(String[] args) {
SpringApplication.run(PaymentServiceApplication.class, args);
}
@GetMapping
public String status(){
return "success";
}
@PostMapping("/pay")
public String pay(){
return "payment successful";
}
}
- Step 3: Call ticket endpoint using Postman as shown below (which internally calls downstream —
payment-service
).
As we can see, the ticket microservice successfully discovered the payment microservice and got a response from the payment service!!! We can discover and call downstream service now.
Now let's deep dive into internals to see how this works.
First, let's list down and note the IPs of pods and services as shown below.
Note : #1 kube-dns IP is : 10.96.0.10 #2 payment-service IP is : 10.108.155.131
Whenever a new POD is created, Kubernetes adds a DNS server entry into the pod's resolv.conf
file. (Pod OS — Linux keeps track of networking information inside the /etc/resolv.conf
file ). To see this lets sh
into one of the pod as shown below.
kubectl exec <PODName> -it sh
Example:
kubectl exec ticket-service-84479b4b59-rqdjz -it sh
Now navigate to /etc
and open resolv.conf
as shown below,
# cat /etc/resolv.conf
As we can see in the output, DNS service IP— 10.96.0.10 is available (cross-check with noted IP above #1); hence each POD will be able to contact the DNS service to resolve the service-name to IP address of the actual service to communicate with each other.
As we are on the ticket pod console, let's query the payment-service
IP using nslookup
as shown below.
As we can see in the above output, the ticket microservice pod can query payment-service
(which delegates requests to payment-pods
) based on the name, which resolves it to the actual IP where the service is running.
Conclusion
We have successfully deployed a SpringBoot microservice and called a downstream microservice, and saw how Kubernetes discovers services using the CoreDNS server.
Thanks to Federico Kereki for his inputs.