Burp-Hash at BlackHat 2015 Tools Arsenal

Originally published here and here with Matt South and Scott Johnson.

One day a few months back, teammates Matt South and Tim MalcomVetter reviewed a report from an application security assessment performed by another teammate, Scott Johnson. These reviews occur as part of our normal quality assurance process, but it’s always interesting, and even fun, when a teammate comes across a rare type of vulnerability. In this case, Scott tested an internet-facing web application belonging to one of our company’s clients, and he discovered a flaw in the way the web application handled password resets. On the surface, it appeared to be similar to many other web apps: if you forgot your password, you could simply tell the web app your email address and have it mail you a link to reset your password. This, at least in theory, “proves” you are the user since you control the email account. The link in the email takes the user back to the web app with a URL. Something like:

https://www.example.com/reset?email=user@example.com&key=b4c9a289323b21a01c3e940f150eb9b8c542587f1abfd8f0e1cc1ffc5e475514

Normally, it’s fairly common for penetration testers to see this and just assume that a long dynamic hexadecimal string in the URL is encrypted with a sufficiently strong algorithm and a key only known by the server — so good luck attacking that. However, Scott observed that it looked to be the same number of characters long as a common cryptographic hash algorithm: SHA-256. With piqued curiosity, Scott tried hashing the email parameter, “user@example.com.” Then the unbelievable happened: it was a perfect match to the dynamic hexadecimal string in the URL. From there, of course, it was a simple matter of exploiting the helpful error messages on the application’s login pages to enumerate valid email addresses. This is easy when apps are polite, saying things like: “Email address not found. Would you like to sign up?” and “Invalid password. Need to reset your password?” If you don’t get the “Would you like to sign up?” message, you just found a valid user’s email address.

While the application had intended to prove that the user clicked the link in the password reset email, the truth is that the server had no way of knowing if a password reset email had even been sent to that email address. There was no application state — just an email address and its hash. So, an attacker could reset any user’s password simply by hashing their email address, without the user ever getting an email with a password reset link in it.

Our client fixed the issue and we validated the fix, but it left us wondering: with something so subtle, have we ever missed that before? The three of us decided to create a tool to automatically find this vulnerability, should it ever happen again. Since Burp Suite Pro is our favorite tool for testing web applications, especially since it has a good way to extend functionality with its plugin API, we began designing a plugin before the pixels on our report edits were dry. Thus, Burp-Hash was born: a Burp Suite plugin that locates cryptographic hashes in web applications, hashes observed parameters, and saves penetration testers the hassle of manually discovering if any observed parameters match the observed hashes.

You also can watch a quick two-minute video overview of the plugin onYouTube.

To install Burp-Hash, you can pull down the latest burp-hash.jar file fromGitHub, launch Burp and import it on the Extender tab. Alternatively, you can install Burp-Hash from the BApp Store by navigating to Burp > Extender tab > BApp Store and locating Burp-Hash from there.

When we started to develop the Burp-Hash plugin, we set out to accomplish two things:

  1. Hash all the things! Locate as many parameters as possible and pre-compute their cryptographic hashes, sort of like miniature, targetedRainbow Tables.
  2. Match the hashes! Locate any hashes passed from the server and compare them to the hashes we have pre-computed, so we can find a match.

Along the way, we decided to use SQLite to contain our observed parameters and their hashes. SQLite provides a relational database with SQL syntax for single user applications, such as mobile apps. Additionally, the SQLite database files are portable, which means a Burp-Hash user can re-import the database file and carry the table of observed parameters and corresponding pre-computed hashes across burp installations. SQLite turned out to be a great fit for our project’s goals.

For performance reasons, Burp-Hash runs within Burp’s Passive Scanner and only for URLs that are in the Burp Target Scope. Burp-Hash will not observe or hash parameters passed to or from web applications that are outside the scope. To get Burp-Hash to start scanning a particular application, just add that application’s URL to the Burp Target Scope. Burp-Hash will then scan any proxied traffic that matches the Target Scope.

Burp Suite provides a very extensible API, as demonstrated by the variety of extensions available for download in the Burp Suite BApp Store. The Burp Extension API provides programmatic ways of grabbing many key/value pair parameters such as the query string of HTTP GET requests and URL-encoded values from HTTP POST requests. However, there are limitations we knew we needed to overcome for Burp-Hash to become more powerful. Fortunately, the Burp Extension API also provides access to the raw HTTP requests and responses.

Many web services leverage JSON and XML, but data encapsulated in JSON or XML cannot be extracted by the Burp Extension API’s getParameters() method. This is for good reason: there’s no easy way to identify parameter values in object-oriented data since it can be nested in complex ways, unlike simple name/value pairs found in HTTP GET query strings. As a result, Burp-Hash does some regular expression based searching to identify the parameter data to hash. Besides parsing JSON, Burp-Hash uses regular expressions to help locate data such as email addresses, even if those values live in places like the HTTP headers. Burp-Hash also grabs complete HTTP headers since some web services include those as part of an HMAC calculation. If developers forget to key or salt their HMACs, then Burp-Hash may be able to identify the source HTTP header values.

The following is an example screenshot of the SQLite parameters table with some test data:

Once a parameter is located, Burp-Hash will check the SQLite database to see if it’s a value that was previously found. If it’s a new parameter, Burp-Hash then pre-computes hashes using the cryptographic hash algorithms that are enabled in its configuration. Out of the box, Burp-Hash supports:

  • MD5
  • SHA-1
  • SHA-224
  • SHA-256
  • SHA-384
  • SHA-512

However, a Burp-Hash user can choose to disable some or all of the hashes for performance reasons, or even disable the hashing of parameters, leaving Burp-Hash in “Report Only” mode, which simply notifies the user when potential hashes are observed passing to and from the server.

Burp-Hash will even pre-compute the URL-encoded form of the parameter’s value, just in case a developer happens to use that version as the source of the cryptographic hash. Over time, this becomes sort of a Rainbow Table of all observed values:

Identifying potential hashes in an HTTP request or response turns out to be somewhat tricky. The first hurdle is dealing with encodings. Most commonly on the web, hashes are Base64 or ASCII Hexadecimal encoded. Creating regular expressions to locate hexadecimal strings is not that hard, but base64 is another beast entirely. For example, the following two strings both satisfy the character sets for base64:

  1. “ZXhhbXBsZTEz”
  2. “ThisIsNotB64”

However, it is much more obvious to the human eye that the second string is not properly base64 encoded. Burp-Hash mostly gets past this hurdle by enforcing total string length rules that match the common cryptographic hash algorithms’ output, though it is possible for a false positive to trick Burp-Hash. It also is possible that those values could be URL encoded by the server in addition to their primary encoding, so Burp-Hash also takes that into consideration and can detect a variety of parameters and hashes with or without URL encoding.

The second hurdle is ensuring that a value previously identified as a longer hash output (e.g. SHA-512) does not also result in a false positive match for a shorter hash output (e.g. SHA-1). Since the character sets are the same, a subset of a larger hash string does appear to be a hash by itself. Burp-Hash attempts to solve this by locating longer hash strings first.

The third hurdle is identifying false positives. It is possible that Burp-Hash is only identifying a string of bytes that looks like a hash, but those bytes could have been generated by a variety of other computations that are not hashes at all. It is for this reason that if Burp-Hash only observes what appears to be a hash, the confidence is marked as “tentative” in Burp’s issues list:

If, however, the value can be matched to a specific pre-computed hash value in Burp-Hash’s SQLite database, then Burp-Hash adds a “Hash Match” to Burp’s issues list, with the plaintext source value clearly identified:

Burp-Hash also will highlight the location of the hash in the request or response where it was observed:

While it is unlikely that a penetration tester who installs the Burp-Hash to immediately find a hash match, at a minimum this extension will start to highlight the presence of hashes as they are passed back and forth among a variety of web applications. If your application happens to have the same type of vulnerability thatScott found in Part 1, then Burp-Hash will likely find it for you without the manual effort of extracting all the suspected parameters and checking them against the hash one by one.

For the Burp-Hash project overview visit http://burp-hash.github.io/.

Get the source code on GitHub and send us a pull request here:https://github.com/burp-hash/burp-hash.

View the list of Burp Extensions here: PortSwigger BApp Store:https://portswigger.net/bappstore/.