When Fast Just Isn’t Enough: Nobody inspects the spammish repetition

A message-digest provides a fingerprint for data and is used to prove the identity and integrity of messages and entities. The most common ones used are MD5 (128-bit message hash), SHA1 (164-bit message hash), and SHA-256 (256-bit message hash). So for “hello” we get an MD5 signature of “5D41402ABC4B2A76B9719D911017C592” and for SHA-1 it is “AAF4C61DDCC5E8A2DABEDE0F3B482CD9AEA9434D”.

Overall the hash signature should be fairly unique, so if we change any part of the data, it will produce a completely different hash signature. The longer the hash signature the less chance we will have of two pieces of data having the same hash signature.

These methods are fairly fast and can run on most computers, but they use cryptography methods for their processing. This makes it difficult to run the hashing method in parallel, so if we had 16 cores on our computer, with these methods we would only be using one of the cores. Along with this the methods such as MD5 and SHA-1 use numbers which are fairly difficult to process on a 32-bit or 64-bit processor.

Non-crypto hashing

So how do we improve the performance of our hashing process? The best solution is to use non-crypto hashing. The main methods are typically written in C++, in order that they are fast…