Distributed Cache Design: 🖥

Published in

rtkal

6 min readMay 30, 2020

A Cache is like short-term memory. It is typically faster than the origin data source. You know Accessing data from RAM is always faster than accessing it from the hard drive. 😊
Caching means saving frequently accessed data in memory (short -term memory).

1. What is Distributed Caching?

A distributed cache is a cache that has its data spread across several nodes in an (a) cluster, (b) across several clusters, or (c) across several data centers located around the world.🌎

Why do we need Distributed Cache System?
This is being primarily used in industries today, for having the potential to scale on demand & being highly available.
(A). Scalability, High Availability, Fault-tolerance are crucial to the large-scale services running online today.
(B). Businesses cannot afford to have their services go offline. Think about health services, stock markets, military. They have no scope for going down. They are distributed across multiple nodes with a pretty solid amount of redundancy.
👉 Google Cloud🌨 uses Memcache for caching data on its public cloud platform

2. How a distributed cache is different from Traditional Caching?

👉 Distributed cache is designed to scale inherently, they are designed in a way such that their power can be augmented on the fly be it computing, storage, or anything.

👉 But a traditional cache is hosted on a few instances & has a limit to it. It’s hard to scale it on the fly. It’s not so much available & fault-tolerant in comparison to the distributed cache design.

👉 look at the above image (Distributed Cache) that clearly shows that how cloud computing uses distributed systems to scale & stay available.

3. Use case of Distributed Cache

What Are the Use Cases of a distributed cache? 🤔, Let’s see some popular use cases for caching.
1. Database Caching
The Cache can be placed between Application Server and Database. Where access the data from the cache instead of the main datastore which frequently accesses data in-memory to cut down latency & unnecessary load on it. There is no DB bottleneck when the cache is implemented.

2. User Sessions Storing
User sessions are mostly stored in the cache to avoid losing the user information in case any of the instances go down.
If any of the instances goes down, a new instance spins up, reads the user data from the cache & continues the session without having the user notice anything amiss.

3. In-memory Data Lookup
If you have a mobile / web app front end you might want to cache some information like user profile, some historical data, or some API response according to your use cases. Caching will help in storing such data.

3. How Distributed Cache Works

Let’s see how distributed cache works behind the scenes.

👉 A Distributed cache under the covers is a Distributed Hash Table that has the responsibility of mapping Object Values to Keys spread across multiple nodes.

👉 A hash table manages the addition, deletion, and failure of nodes continually as long as the cache service is online. Distributed hash tables were originally used in peer-to-peer systems.

👉 Speaking of the design, caches evict data based on the LRU(Least Recently Used policy). Will see the eviction policies in the next steps. Basically, it uses a Doubly-Linked-List to manage the data pointers, which is the most important part of the data structure. Please read the LRU implementation for more details.

4. Cache Invalidation

It does require some maintenance for keeping the cache coherent with the source of truth (e.g., database). If any data is modified in the database, it should be invalidated or modified in the cache also; if it is not so, this can cause inconsistent application behavior.
A cache can be invalidated in the following ways.

Read-Through → When the Client asks for information, It will find in the cache first. If the entry is already in the cache it will just return with data. If there is a cache miss or not found in the corresponding cache, Coherence will automatically delegate to the CacheStore and ask it to load particular information from the underlying data source/database.

If the information is found in the database, the CacheStore will load it, and return it to Coherence, then Coherence will place the information in the cache and will return the information to the Client. This is called Read-Through caching.

This scheme maintains complete data consistency between the cache and the main storage. Even this scheme also ensures that nothing will get lost in case of a crash, power failure, or any other system disruptions.

Image Left → Read operation => Read from cache but data already available in cache storage.
Image Right → Wirte operation => If a cache miss then write to cache from main storage

→ The disadvantage of this scheme is the Higher Latency since every write operation is done twice before returning success to the client😔

Write-Through → In this strategy, every piece of information(eg. new entries) is directly written to the database just bypassing the cache. Write-through can simplify the cache coherency protocol because it doesn’t need the Modify.

Image Left → Read operation => Read from cache but data already available in cache storage.
Image Right → Wirte operation => If a cache miss then do not wirte to cache from main storage just read from main storage and return.

→ The disadvantage of this scheme is the Higher Latency if in case of a read request for recently written data will create a “cache miss”😯 and then the read request must be made on back-end storage.

Write-Back → The data is written to the cache only and completion is immediately confirmed to the client. The write to the permanent storage is done after specified intervals or under certain conditions. This results in low latency and high throughput for write-intensive applications.

Image Left → Read operation => Read from cache but data already available in cache storage.
Image Right → Wirte operation => If a cache miss then do not wirte to cache immediately from main storage.

→ However, this speed comes with the risk of data loss in case of a crash or other adverse event because the only copy of the written data is in the cache.

5. Cache Eviction Policies

What if our Cache is full? So, A cache eviction algorithm is a way of deciding which element to evict when the cache is full. The following are some of the most common cache eviction policies:

First In First Out (FIFO): The cache evicts the first block accessed first without any regard to how often or how many times it was accessed before.
Last In First Out (LIFO): The cache evicts the block accessed most recently first without any regard to how often or how many times it was accessed before.
Least Recently Used (LRU): Discards the least recently used items first.
I mostly prefer to LRU only. 🙂
Most Recently Used (MRU): Discards, in contrast to LRU, the most recently used items first.
Least Frequently Used (LFU): Counts how often an item is needed. Those that are used least often are discarded first.

6. Some Popular Distributed Caches

The popular distributed caches used in the industry are Eh-cache, Memcache, Redis, Riak, Hazelcast.

👉 Memcache is the most popular cache which is used by Google Cloud in its Platform As A Service.

👉 Besides Redis is an open-source in-memory distributed system that supports other data structures too such as distributed lists, queues, strings, sets, and sorted sets. Besides caching, Redis is also often treated as a NoSQL data store.

Thank you, Guys! This article is over here and I think it’s pretty much about the distributed cache. If you have any queries please let me know in the comments.

For more posts like this follow us https://medium.com/rtkal