An Anecdote

contents of my LevelDB logfile.

Updated Preface:

5/20/2018 — I’m not sure why people are still reading this. While “never” is still a dangerous option, the rest of the issues I raise here are mostly resolved. You can read more about this in the updates at the bottom. Please consider this article to be a historical record, and read the entire thing.

6/21/2018 — I don’t intend for this article to damage BitWarden’s reputation in perpetuity. The title has been changed to something less inflammatory to draw less traffic from search engines.

Original post:

As a preamble, this isn’t the article I wanted to write. I would have much rather drafted a success story about the power of Open Source and how it always leads to better software.

Alas.


BitWarden is a password manager, and a really good one. They provide feature-parity with some of the giants like LastPass and 1Password, and they do it completely for free. Kudos on those victories.

There are less good things about it too.

I recently discovered a security flaw.

A few weeks back, I discovered a flaw in a Chrome extension called CKP. It was storing user’s master keys in a special file within the browser’s file space without warning about how dangerous this was. Knowing the master key, which is deterministically generated by a hash function from the user’s master password, will allow an attacker to decrypt the entire database of stored secrets — passwords, credit card info, etc — stored elsewhere on disk.

This was an accident. The developer of CKP didn’t understand that the chrome.storage.local browser API was journaled. In other words, you store a piece of information, and when you want to update it, you just append a new version to the file. CKP stored the master key through this API and did not understand that changing the value left both copies on user’s hard drives.

This week, I found the same flaw in BitWarden.

When you first install BitWarden in your browser, the default option is to never forget the master key that unlocks your secrets. The first time you log in, that key is written to disk, and it might never get unwritten.

Even if you change the default option immediately, you get a copy of your master key on disk.

I tried to report what I found.

I opened an issue on the GitHub issue tracker, and I tried to explain what I had found. Kyle Spearrin, the lead developer behind BitWarden, responded 20 minutes later.

You can read the full conversation on GitHub. Here’s how the conversation went:

Me: I’ve found this issue, and it might be a flaw. You can find the evidence on your own computer at /path/to/Bitwarden/data . You can see that key is plainly kept in the file.

I just checked this on my local windows machine…and I cannot find any trace of key. I am using a lock option which would have cleared the default key which was originally saved after first login.

Fine, he didn’t read my issue that closely. I understand he must be busy, but I find it disconcerting that Kyle has no idea how thechrome.storage.local API works. Chromium uses LevelDB, a fast key/value store, as the implementation of chrome.storage.local .

He’s also wrong about how the lock option would have cleared the default key.

An appropriate response would have been “Oh wow that doesn’t seem right. Can you provide exact steps to reproduce? What OS are you using and what version of what browser? And are you sure you see two copies of key?

Me: Here are the steps to reproduce.

Looks like it’s there after toggling which is weird…I can definitely say that it wasn’t there before which means the value is not saved forever. Maybe it’s some kind of cache?

It isn’t a cache. The file in question is called 000003.LOG for God’s sake. Again, the correct response here would have been “OMG this looks really bad.

At this point, we seem to have agreed that “clearing” the key with an overwrite doesn’t work, but we have drawn different conclusions about how insanely bad that is.

Me: No it isn’t a cache. It’s a journaled file(etc.) I think this is a serious issue.

We are planning to change this setting’s default in an upcoming release. BitWarden syncs and overrides the user’s entire encrypted vault to chrome.storage multiple times per day so I imagine that log fills up and rolls over quickly. In any case it's not there forever so I don't think we have to worry about trying to come up with a solution to cleanup those log files.

You’re kidding. Users might generate enough data during normal operation to cause log rollovers and hopefully that will be often enough so that most people aren’t affected probably?

Kyle didn’t know anything about chrome storage 20 minutes ago. I don’t know how he’s done enough research since then to come to this conclusion.

This response demonstrates carelessness regarding the personal security of users who rely on BitWarden. “Good Enough” is not the attitude I expect.

Me: Imagine a scenario where I install BitWarden, port over my data from LastPass, and then get sidetracked for a week. No sync, no updates, no fresh data. Then I lose my laptop on the train and there’s a good chance both my encrypted vault and the key to unlock it are in one convenient location on disk.

This should resolve it going forward: 4a34f9b . This will be bundled with next release which is going out rather soon.

And then Kyle closed the damn issue.

4a34f9b changes the default from remember forever (which causes key to be written to disk) to remember until you restart your browser. Because of this, users who never use remember forever will be safe.

Users who choose remember forever even once will be unable to change their minds — they’re stuck with credentials on disk until a log compaction or a complete uninstall.

4a34f9b will mitigate the number of people affected by this bug, but it doesn’t come close to solving it.

What does this mean for BitWarden users?

As of Jan 17, 2017, every BitWarden user on Google Chrome is potentially affected. Firefox users are safe, and I have no access to Microsoft Edge to test it.

You might have a copy of your master key saved to disk. It might have been overwritten in a log compression and disappeared. It doesn’t matter how you configured the remember time — you have no way of knowing until you look.

I’m not going to be a sensationalist and tell you it’s time to bail on the product forever, but I would absolutely do the following things:

  1. Uninstall BitWarden’s browser extension. This triggers the settings and log files to get cleaned up and erased from disk.
  2. If you use a full-disk backup tool such as Carbonite, it probably caught your master password and encrypted files. Clean those up.
  3. Wait for release 1.24.0 — there’s no way to escape this bug until that gets published.
  4. Install BitWarden again if you want, or check out some of the alternatives.

The sensitive information exists here:

  • Windows: %APPDATA%\Local\Google\Chrome\User Data\Default\Local Extension Settings\nngceckbapebfimnlniiiahkandclblb\
  • Mac: ~/Library/Application Support/Google/Chrome/Default/Local\ Extension\ Settings/nngceckbapebfimnlniiiahkandclblb/
  • Linux: ~/.config/google-chrome/Default/Local\ Extension\ Settings/nngceckbapebfimnlniiiahkandclblb/

About me.

I’m a developer for the KeePass ecosystem.

My original motive was to prevent BitWarden from making the same mistakes CKP did. Now, I just want to help BitWarden’s users stay safe and informed.

Update 1: Version 1.24.0 is released.

This is a good step, sort of.

1.24.0 is not a fix for the issue I’ve described, but it will mitigate how many people are affected in the future. If you haven’t seen it, read Kyle’s blog post.

I’d also like to add that the way this issue was patched was deeply irresponsible. Most users will never see that blog post. Most users will _not_ know to uninstall BitWarden before upgrading, and will not clear their sensitive info from disk in doing so.

BitWarden touts itself as being user friendly — a password manager for everyday folks. Most of them could still be in danger.

It would have been simple to avoid this. Chrome provides the runtime API, and with it an event you can listen for: chrome.runtime.onInstalled . As soon as user’s browsers automatically upgraded to BitWarden 1.24.0, that event is fired. From there, it’s trivial to clear ALL of storage with chrome.storage.local.clear() which DOES cause the old logfiles to get wiped out, and protect 100% of users.

I estimate 95% of users will not know to Uninstall, and some un-guessable number of them will still be affected by this issue.

Update 2: Version 1.24.2

This version automatically purges local storage. Users should be safe now as long as they never, ever use the “never forget” option of Bitwarden.

BitWarden still contains a dangerous feature with undisclosed consequences, but users are much safer now that the default is changed. I’ll call that a good day’s progress. Thank you, Kyle and BitWarden team.