What they’re not telling us

If you’re living under a rock, you might have missed the news about Heartbleed, a serious bug on the Internet that can lead to stolen information.

Heartbleed is a bug in OpenSSL, a software package that provides many encryption tools for many different services. One of these is Transport Layer Security, or TLS, which is used to encrypt communications on the Internet, in particular on the web, but also the IMAP and POP mail protocols, among others.

One of the features of TLS is a heartbeat, which is standard fare in many protocols. The heartbeat basically keeps both sides informed that the connection is still alive even if no other traffic is flowing.

In this case, the part of the software that implements the heartbeat was vulnerable to an attack where an attacker could send unexpected data in the heartbeat and trick the server into dumping a random portion of it’s memory back to the attacker (with apologies to the technical folks for oversimplifying things a bit). The website would “bleed” data, hence the name “Heartbleed”.

Sometimes, or perhaps even often, this random piece of memory would contain any personal data that the server might be storing in memory. Like credit card numbers or passwords. This is part of the reason why this is a big deal. Primary attacks using Heartbleed can directly result in stolen information, without leaving any traces behind that there has been an intrusion.

This is essentially what they are telling us about Heartbleed. But there’s a significant problem that hasn’t received as much discussion.

One of the things that can be stolen from a server is the private key for the server’s web certificate. (A web certificate is the proof , digitally signed by a trusted third party, that the people you’re communicating with on the web really have a right to use the domain, e.g. www.google.com.) The private key is the private part of the public key encryption used to secure web traffic in TLS. If the private key isn’t secret, it enables third parties to snoop your web traffic and pull out things like passwords and credit cards, and anything else you’re doing on the web.

This is perhaps the larger problem with Heartbleed that hasn’t received enough attention. If an attacker obtains a server’s private key with the primary Heartbleed attack, it allows for secondary attacks using this private key. These secondary attacks can be passive—watching and decrypting traffic on the Internet; or active—setting up fake webservers that pose as the real webserver using a 100% legitimate private key.

So the big advice we have all heard about Heartbleed is basically, change your passwords, check your credit card transactions for awhile, and you’ll be fine. Of course we’re told to wait until the service provider has fixed the problem, otherwise your password could simply be stolen again. But what does “fixed”mean?

For starters, you should know that many service providers were not affected by this bug. The Heartbleed bug in OpenSSL is recent, so many providers that upgrade infrequently were actually protected by running what some would call out-of-date software. Also, OpenSSL isn’t the only implementation of TLS out there. There’s a popular one from Mozilla that’s used in most browsers in many servers.

So the first question before you go changing all of your passwords is “did my provider have the Heartbleed bug”. If not, then the data on that site is as secure as it was before all this happened. On the other hand if they did have the bug, then of course it’s important that they update their OpenSSL software to a fixed version. But because of the possibility that their private key could have been unknowingly stolen, it’s also critically important that they obtain a new web certificate based on new private/public key pair.

Here’s where things get a bit sketchy. No big-name service providers like Google or Facebook are providing this level of explicit detail, but they are all probably dealing with the same problems — a clash between engineers and public relations. PR might feel that this is too much detail, and if you confuse the customers they might lose confidence. But realistically, we can’t know if we should change our passwords unless we know whether or not the web certificate has been updated, even if they’ve “fixed” the OpenSSL problem.

We can look of course. The little lock icons on all of our web browsers allow us to inspect details for the server certificates. For example, when I look at Twitter’s web certificate, it says this:

details from web certificate at twitter.com

This is reassuring. It looks like Twitter got a brand new certificate the day before all of this broke. Perhaps they knew ahead of time. Both Google and Facebook in public statements have claimed that they got ahead of this and fixed it before it became public, so perhaps Twitter did the same. It’s also not uncommon for a certificate authority to backdate the start date on certificates by a day, just to make sure there’s no problems if there’s a clock in the world that’s a little off. At any rate, they got a new certifcate pretty much exactly when we would want them to.

Yahoo also had the Heartbleed bug, however they were criticized for being a bit slow on fixing it. People were able to exploit their servers for a few hours after the news broke. Nevertheless they fixed the problem, and they also got a new certificate:

details from the web certificate at yahoo.com

One little problem here is that we know Twitter and Yahoo got new certificates, but we don’t know if they actually got new private keys. It’s common practice to generate new certificates with existing private keys, because it’s simpler. The certificate authorities that sign certificates make it easy, and it’s the quickest way to renew certificates. Now probably, nobody is that stupid, and probably they would revoke the old certificate and create new private keys and get a new certificate from those. Probably. But there’s no way to verify any of this.

Facebook is even more problematic. As I said, they claimed they fixed this bug before the story broke. This sound a lot like they are saying they did in fact have the Heartbleed bug. But here’s their certificate dates:

details from the certificate at facebook.com

As you can see, it really looks like they did NOT get a new certificate. This is a serious problem! It doesn’t really matter if they fixed it before the public knew about it. The question is, did they fix it before some small number of hackers knew about it? This is impossible to know, therefore they should’ve updated their certificate, and yet they almost certainly didn’t. Why do I say “almost”? Well, if you’re big company, you can probably have the certificate authority put any dates in there you like. It’s possible that they updated their certificates, but in an early and misguided effort to conceal that they had this bug, they tried to obscure it. It’s possible, but it’s not very likely. Or so I assumed until I saw Google’s certificate.

The situation with Google is more complicated. Here’s Google’s certificate dates (specifically, gmail.com):

details from the certificate at gmail.com

So did they or didn’t they? This certificate is only good for less than three months. That’s a really strange period of time. Most certificates are good for a full year at least, often two or three. And they’re generally exactly a year (or two or three), although some certificate authorities add in a free extra month for good measure.

It does expire exactly a month after this story was a day old. I don’t know what we can read into that. Perhaps they really did backdate their certificate to make it seem unconnected to Heartbleed? As I said, that would be misguided, because customers need to know if and when providers have updated their certificates (and private keys). But in the early hours of an emergency decisions are not always rational. I’ll note here that Google serves as their own certificate authority, so it would be particularly easy for them to generate certificates that say anything they want.

There’s another, even scarier possibility. Perhaps March 12 is the date when Google fixed the Heartbleed problem and updated their certificates. This would be the responsible choice for Google. But it would also mean that Heartbleed was out there in the wild, and known about by many many people, for almost an entire month.

What we need to be secure right now is for companies to tell us what happened, and when it happened, and exactly what they have done about it. Were they using OpenSSL? Was it a compromised version? Did they update or replace the software? And most importantly, have they created a new private key for their server. The vague statements we’re seeing in the press and the nebulous information we can gleen from poking at web certificates is not sufficient, and at the moment it is not inspiring confidence.

Show your support

Clapping shows how much you appreciated Thomas A. Fine’s story.