Load balancing, Hashtables and Consistent Hashing
I may be wrong, but at the core of this is technical and scientific excellence, and in addressing problems with new methods, or by using existing methods in new ways. Too often, technologists just focus on the methods they know and then try and match these to the problem, but a true innovator finds the tools/methods which fit the problem. So, let’s look at something that disrupted and trace its roots: hash tables and consistency hashes.
Hashtables
A hash table is a fast and efficient way to find and map data objects. If we have banking transactions, we can simply hash the transaction, and this hash value then becomes the index (or key) for the transaction. A great advantage of this is that we can then distribute our data infrastructure, and then maintain a consistent hash table across the whole infrastructure. We thus can create a hashtable with a number of slots (m), and where we map a number of keys (n). In the hashtable, we then get a key-value mapping for our data.
In a hashtable, we need to decide the total number of key pairs we wish to store. We thus end up with an array of m slots, and where we create a hash key then map this to a slot location in the array:
hash_value = key (mod m)
