Complying with NIST Guidelines for Stolen Passwords

It seems everyone today is talking about stolen passwords, but this is an older problem than people realize. Protecting your enterprise from credential stuffing attacks and account takeover as a result of stolen credentials is at the heart of the discussion — and as more business moves online it’s an increasingly expensive problem.

The Stolen Password

During the final phase of the Peloponnesian War, a series of tactical errors led to the defeat of the superior Athenian forces. Among the many errors was an inadequate identification system, reliant on a shared watchword. At the final and crucial battle of Syracuse, the besieged Syracusan army discovered the Athenian watchword that was used for identifying allies. Quietly disseminating this password between them, the Syracusan forces created havoc during a nighttime battle, preventing the dazed Athenian forces from identifying ally from foe and ultimately leading to their devastation.

Step forward a few thousand years to the 1960s, when limited computing resources at MIT resulted in the Compatible Time-Sharing System. Given the limited power of computers at the time, a short phrase was the simplest way to identify users on the platform. But, the first password breach soon followed when in 1962 Allan Sherr, looking for a way to increase his allotted time on the platform, managed to request a printout of the entire password file.

Since then, the history of the password remains consistently problematic. Passwords are complicated, easily forgotten, and usually represent a single point of failure. Wake up to find your password compromised and the results, although not quite as devastating as for the Athenians, can often be financially or socially shattering. In efforts to make passwords themselves more secure, increasingly arbitrary rules on their construction have been enforced. Unfortunately, these rules often force users to adopt practices that directly contradict the intent, driving users to adopt the same password across websites, or preventing the use of tools like password managers.

Meanwhile, despite growing awareness of multi-factor authentication systems, statistics around adoption rates are difficult to find and are widely thought to be worryingly low.

Introducing NIST

NIST (or the National Institute of Standards and Technology) is a non-regulatory United States Government Agency with a mission to “promote U.S. innovation and industrial competitiveness by advancing measurement science, standards, and technology in ways that enhance economic security and improve our quality of life.” Within this mandate, NIST has established the NIST Cybersecurity Framework. The original intent was to develop a voluntary framework to help organizations manage cybersecurity risk across critical infrastructure, but the framework has been adopted much more widely throughout the world. Of particular interest is the four-volume Special Publication 800–63 on Digital Identity Guidelines which is available on the NIST Website and NIST GitHub. It includes:

Password Guidance, the In-and-Out

Section 10.2 of 800–63B includes updated rules which contradict much of the conventional wisdom on passwords.

In:

• Support passwords of at least 64 characters, allow spaces, and encourage users to make passwords as long as possible to encourage the use of passphrases.
• Compare new passwords to a dictionary and don’t allow common, easily-guessed passwords (such as password, abc123, etc.).
• Offer the option to display the password, rather than dots or asterisks.

Out:

• Don’t enforce composition rules (no more: your password should include upper and lower case characters, and at least one number). These encourage passwords with the illusion of complexity, like Passw0rd, which any dictionary attack will take into account.
• Don’t use password hints, as users tend to populate these hints with enough information to make guessing the password trivial. Instead, focus on supporting easily memorized passwords and phrases.
• Don’t expire passwords unless there’s a user request or evidence of authenticator compromise.
• Don’t use Knowledge Based Authentication (e.g. what was the name of your childhood pet?).
• Don’t use SMS as a 2-Factor Authentication method.

Updated Password Storage Guidance

NIST also includes guidance on encryption and storage of user passwords. As we’ve seen from previous breaches, weak and reversible encryption lets attackers access vast sets of credentials that can easily be used against other sites. To limit the effect of breaches on other sites, NIST recommends that:

• Passwords should be salted and hashed using a suitable one-way key derivation function.
• Use approved key derivation function PBKDF2 using SHA-1, SHA-2, or SHA-3 with at least 10,000 iterations.

Breach Corpuses

The appendix of 800–63b lays out some hard truths about the choices we make as users:

Users’ password choices are very predictable, so attackers are likely to guess passwords that have been successful in the past. These include dictionary words and passwords from previous breaches, such as the “Password1!” example above. For this reason, it is recommended that passwords chosen by users be compared against a “black list” of unacceptable passwords. This list should include passwords from previous breach corpuses, dictionary words, and specific words (such as the name of the service itself) that users are likely to choose. Since user choice of passwords will also be governed by a minimum length requirement, this dictionary need only include entries meeting that requirement.

Maintaining a list of compromised credentials from previous breaches is a noble effort, but there are a number of factors to balance. For example, Facebook attracted criticism when it announced in 2016 that it had been purchasing credentials from the dark web in order to secure its own users. Such purchases could effectively help power the market, funding and encouraging further breaches and supporting the black market credential ecosystem. Plus, the huge window of time that exists between a data breach and the eventual emergence of the stolen credentials means that traditional breach corpus lists are often ineffective — and that’s something Shape Security is addressing with Blackfish. Shape co-founder Sumit Agarwal explains it best:

Shape has grown into one of the largest processors of login traffic on the entire web. We have built machine learning and deep learning systems to autonomously identify credential stuffing attacks in real-time. These systems now generate an important byproduct: direct knowledge of stolen usernames and passwords when criminals are first starting to exploit them against major web and mobile apps. What this means is that we see the stolen assets months or years before they appear on the dark web.

Of course, once you’ve found a way to compile it, a vast list of the freshest credentials is itself a major target, so to minimize risk (as well as ensure absolute compliance with regulations such as GDPR), Shape does not store any direct username/password pairs but instead leverages a probabilistic data structure called a Bloom filter.

Conclusion

As Jim Fenton, one of the publication contributors, points out, “If it’s not user friendly, users cheat.” Frustrating password policies have been long overdue for an overhaul and the new NIST Digital Identity Guidelines rightly place the burden upon the verifier, not the user. While verifiers of users should ensure they’re following the guidelines to give users the best chance of securing their accounts, they should also take additional steps to ensure that security breaches originating from outside their own organization are stopped before they create more damage closer to home. In light of the recent FTC ruling on credential stuffing, it might be more than just best practices that encourage verifiers to comply.

To learn more about NIST guidelines, stolen credentials and the breach corpus, watch “After the Data Breach” — a live webinar with Justin Richer, co-author of NIST Special Publication 800–63B.

[1] https://en.wikipedia.org/wiki/Peloponnesian_War

[2] http://perseus.uchicago.edu/perseus-cgi/citequery3.pl?dbname=GreekTexts&getid=1&query=Thuc.%207.44.6
[3] https://it.slashdot.org/story/12/01/28/024220/how-allan-scherr-hacked-around-the-first-computer-password
[4] https://nakedsecurity.sophos.com/2013/04/23/users-same-password-most-websites/
[5] https://www.infoworld.com/article/2623504/data-security/study-finds-high-rate-of-password-reuse-among-users.html
[6] https://mashable.com/2017/02/28/passwords-reuse-study-keeper-security/
[7] https://www.nist.gov/director/pao/nist-general-information
[8] https://medium.com/shape-security/2017-credential-spill-report-1ec31c411472
[9] https://www.csoonline.com/article/3142404/security/security-experts-divided-on-ethics-of-facebooks-password-purchases.html
[10] https://www.slideshare.net/jim_fenton/toward-better-password-requirements

Originally published at blog.shapesecurity.com on March 1, 2018.