Critical CloudFlare Vulnerability Causes Sensitive Data Leak. Are Reverse Proxy Solutions Worth It?
User data of Uber, Fitbit, Ok Cupid, 1Password and leading companies was risked for weeks together due to a critical CloudFlare vulnerability. The ‘Cloudbleed Bug’ was caused because of servers running past the buffer and returning memory containing private information. Something similar was seen in the heartbleed bug reported in 2014 too. The vulnerability was reported by Google security researcher Tavis Ormandy. Graham-Cumming, CTO at CloudFlare said that it if difficult to point out which of the 6-million websites have been affected. A number of security experts believe that Cloudbleed has much more impact that CloudFlare is claiming.
Consequences of the Cloudflare Vulnerability
A number of CloudFlare’s services rely on parsing HTML and modifying it on the fly. Cloudflare was using a parser written in ragel and last year decided to write their own parser. Both the ragel and new parser where deployed as nginx modules. While the bug causing the vulnerability was present in ragel parser always, when the new parser was introduced the way parsers interacted with the server changed. This caused a memory overflow and hence the vulnerability surfaced. An in-depth analysis of the cause can be found here. The direct consequences of Cloudbleed are:
- Search engines cached a lot of leaked data and are displaying them on the search results
- Leakage of personal chats on social/dating websites
- User identity information leakage
- Cookies/IP information
The examples we’re finding are so bad, I cancelled some weekend plans to go into the office on Sunday to help build some tools to cleanup. I’ve informed cloudflare what I’m working on. I’m finding private messages from major dating sites, full messages from a well-known chat service, online password manager data, frames from adult video sites, hotel bookings. We’re talking full https requests, client IP addresses, full responses, cookies, passwords, keys, data, everything. -Travis Ormandy
Immediate Precautionary Steps
- Change you passwords. Right from FTP, cpanel, admin panel to all others.
- Force a password change to your users
- If possible, use two-factor authentication
- Clear your websites cache. Purge old cache.
Since surfacing of this vulnerability, CloudFlare’s stance has been very firm and upfront. Both the CEO & CTO of CloudFlare have given public statements and taken charge of the situation. However, this entire incident raises questions about reverse proxy solutions expecting you to route your web traffic through them. The biggest consequences of such solutions is that if the server gets hacked, all the websites go down. As seen with CloudFlare, with one flaw millions of websites came to risk. One major reason why we did not use reverse proxy implementation with Astra was fear of a situation like this.