Hacking Netflix Eureka!

TL;DR SSRF, internal traffic hijack, XSS

Eureka is a service registry used to make micro-services to find each other. What can go wrong if the registry server becomes available from the internet? What if a malicious user registers his new micro-service?

It looks like the registry can become a point of failure from a security point of view. So that means we should properly secure it. But unfortunately, it is not always the case.

In this short research, you will look at Eureka from the attacker’s perspective and learn how to exploit SSRF and hijack internal infra’s or victims’ traffic.

Both attack vectors are performed on the application layer. That means we can easily mitigate it on network and transport layers. We will talk about defense at the end.

“Whose Registry Is This”?

Official Netflix Eureka description from the GitHub repository:

Eureka is a RESTful (Representational State Transfer) service that is primarily used in the AWS cloud for the purpose of discovery, load balancing and failover of middle-tier servers. It plays a critical role in Netflix mid-tier infra.

Let’s check how many Eureka servers can be found by google (BTW you probably can improve this dork):

I talked to the pentester’s community if they encounter Eureka during penetration testing, and it looks like leaving the registry service “outside” is a common potential security misconfiguration.

Find me a microservice!

Let’s start with some basics to understand how it works based on test lab example:

Suppose we have:

1. Two services on the backend inside the local network, webservice and secretservice
2. Spring Cloud Gateway(further referred to as gateway) configured to route all HTTP requests to webservice
3. !!! there is no HTTP route from the gateway to secretservice
4. registry eureka server is in DMZ, supposing that it is exposed by mistake
5. and user/attacker on the internet.

On startup, each micro-service will send a REGISTER request to the Eureka server (and later will send a STATUS update every N seconds, 30s by default). Based on this information registry knows how many instances of particular micro-service are currently up and running and their addresses (address can be both DNS or IP).

But registry itself is not a proxy. It is “passive” storage and contains info about currently running instances of particular services. “They” call it a mid-layer proxy.

To balance and proxy HTTP requests, we use Spring Cloud Gateway. Gateway collects the same information from the Eureka server to know where it should balance requests. In the test lab, the gateway handles only one route to webservice:

.route("webservice",
    r -> r.path("/**")
         .uri("lb://WEBSERVICE"))

That means every path "/**" (which is not registered on gateway separately) will be balanced ( "lb://WEBSERVICE") to the micro-service with the name webservice. There is no HTTP route to secretservice, so it is not supposed by design that the user will communicate with secretservice directly.

In the wild, it can be any internal micro-service, which potentially may trust all incoming requests. Because usually, “secretservice” is supposed to be used only in internal service-to-service communications.

For PoC secretservice contains flag(CTF guy detected :p) we need to extract.

Webservice, at his turn, communicates with secretservice internally with the following code:

here web service asks Eureka client to find secretservice address and then send request there

Eureka client keeps all info collected from the Eureka server in memory and updates it once per N seconds(30s by default).

Exploitation

The main idea for attack vectors is changing HTTP routing with some profit for exploitation.

Getting access to the registry we can register fake service and try to access this service through Spring Cloud Gateway. After the fake service is registered, the gateway begins to balance requests to this service if we register the right service name.

At this point, multiple security issues arise. If an attacker registers a new instance of an existing micro-service and maps it to some internal micro-service, which doesn’t have a public HTTP route, the attacker can send HTTP requests to this service, which is about SSRF(Server Side Request Forgery). OR if the attacker registers his malicious service outside, both gateway and internal services will send requests to fake service outside, which means it is possible to hijack victim's traffic(+exploit XSS) or hijack internal services connections.

Attack Vector 1: Server Side Request Forgery

For successful exploitation, we need to find a service mapped to root URL “/”(which is not really a requirement but provides more freedom in path walking). For the lab environment, Spring Cloud Gateway already maps “/” to webservice.

At first, find the address of the targeted micro-service to get access to (secretservice), you can find it on the registry:

Then register new microservice with name webservice but with address:port of secretservice(to do this change ipAddr, hostName and port parameters in eureka registration request):

Now there are 2 instances of webservice. After gateway synced information from the registry, it begins to balance attacker requests between 2 instances of webservice. And due to one of the instances points to secretservice now, we bypass route restriction on HTTP layer and successfully perform SSRF attack. In case of a successful attack, you will see 2 different pages during multiple requests, 404 error(because there is no route for /flag on legit webservice) and FLAG{EUREKA_PWNED} :

Attack Vector 2: Traffic Hijack and XSS

In another scenario, what if an attacker will register an instance of webservice and point it to the malicious server? In this case, the gateway will begin to route HTTP traffic to the attacker-controlled server. This leads to secrets leakage(e.g. token or sessionId) and XSS. For the browser, because the communication with the malicious server is performed in the backchannel, it looks like an official site(TLS on Gateway doesn’t help).

We can do the same trick by hijacking internal services calls, leading to information leaking or even more critical vulnerabilities.

How to Communicate with Eureka

Just try this

update fields app(both in path and param), ipAddr, hostname

…or if the above request doesn’t work

If the protocol has changed somehow since this publication, the easiest way to extract the correct request is to run simple eureka client+server locally and use Wireshark(it should take ~10mins). Check any tutorial about registry, e.g. https://spring.io/guides/gs/service-registration-and-discovery/

Defense is boring, but we have to…

Bear in mind that keeping Eureka NOT exposed to the internet must not be considered the main defense because IT happens.

Security has to be implemented with a multi-layer approach, which guarantees that if one of the layers fails due to 0day or misconfiguration, other layers still keep you safe.

Keep it Safe on Application Layer

Eureka supports basic HTTP authentication; use it. Yes, basic authentication is not the best practice, and you have to share one password between multiple micro-services. But it still saves you some time after the attacker finds an exposed registry.

Keep it Safe on Presentation Layer

mTLS between micro-services will help against registering malicious service outside(against traffic hijack).

Keep it Safe on Transport/Network Layers

And, of course, micro-segmentation and strict firewall rules. There is no need to have network connectivity between the gateway and secretservice - it will safe from SSRF. As well as an uncontrolled outbound network connection to public networks must be prohibited - it will save from traffic hijacking (and from reverse shell :p).

Thanks for reading and stay safe!