What is a Distributed Cache?

In this tutorial we are going to learn about what a cache is ? when we are going to use?, and How to use it? in a detailed manner.

So first of all,

What is a Cache?

Imagine that you have a system like this. Client Application request for some results from the server and the server asks those details form the Database. Then Database pullout the results to the Application server. Without pulling data from the Database all the time we can maintain another database/server to store data called Cache. Here there are 2 scenarios that you might want to use a cache.

• When you requesting for a commonly used data, and every time we ask for those data we need to provide from the Database. Instead of this, you can save those commonly used data in a cache (in-memory cache). Here we can reduce network calls.
• When you are doing a calculation by getting data from the database. You can reduce the number of calculations here. Store the result in cache and get the value from the cache without doing recomputations all the time. (Example: Assume you have a Student Management System and you need to calculate the average marks for a particular exam for a particular student. Store Average value in cache memory with key-value pair.)
• We have all servers and they are hitting the database. It’s going to be a lot of loads. Instead of getting one cache, we can use more caches as a distributed system for Avoid load in the Database.

Can we store all the data in the cache?

No! We can’t store all the data in the cache because of multiple reasons.

• The hardware that we use to make cache memories is much more expensive than a normal database.
• If you store a ton of data on cache the search time will increase compared to the database.

So that now you know we can store infinite data on the database and we need to store the most valuable data in the cache.

When do you load data into the cache? When do you evict data from the cache?

Loading or Evicting data from the cache is called a Policy. So the cache performance depends on your cache policy. There are a number of policies you can have. The Most popular one is LRU(Least Recently Used).

LRU — you can add recently used entries to the bottom of the cache and least recently used entries go to the bottom. If you want to add new entries but the cache is almost full, then you can evict(kick) out those least recently used data.

Some other Policies are,

• Least Recently Used (LRU)
• First In First Out (FIFO)
• Random

Problems with Caching

• It is harmful to have your cache. Because you are making an extra call if there are no data that you are looking for. It is completely useless.
• Trashing — Let's assume your cache is enough to store one entry. Client application1 is trying to store data on the cache(Ex: name1 = “Kasun”). Now meanwhile Client Application1 doing that Client Application2 is trying to store data to cache(name2 = “Dissanayake”). Now, name1 will kick-off, and name 2 will store. Assume again Client Application1 is trying to get the name1. But the name1 is not there now(already kicked off). That is a cache miss. You are inputting and outputting data into the cache without using data. This is a problem.

• Consistency — Let’s assume another Application server is trying to update data on the Database. But your cache is not updated. Now if you get the same data from the cache which is not updated it will be a huge problem. You need to get the updated realtime data.

Where to place the Cache?

We can place cache close to the servers as well as close to the Database. If you place cache close to the servers, How can you place it? You can place it in-memory itself (in-memory in the servers). Yes, it will be faster if we use an in-memory cache with the server. But there will be some problems. Take this example,

The first problem is the Cache failure. Assuming Application Server 1 failed. Now the Cache also failed. We will lose that data in Application Server1.

The second thing is the consistency. Application server 1 data and Application server 2 data are not the same. They are not in sync. If there are some critical data you can’t keep this(Ex: Updated Password or any other credentials).

If we place cache close to the database using a Global cache the benefit is that all servers are hitting this global cache. If there is a miss it will query the database otherwise it will return data to the servers. And we can maintain distributed caches here. And it will maintain the data consistency.

Now assume Application server 1 is failed. Then there is no problem with all other servers asking data from the Cache and the cache is still alive. And also it is more accurate.

So my preference is to make a Global cache.

Write-through and Write-back Cache

Write-through Cache

If you want to make an update to the data. First, you hit the cache and update that entry. Then you push it into the Database. So this is a write-through cache. I write the cache before hitting the database.

Problem: What is if other servers having this cache in memory? Application seerver1 will update the data on Cache1 but not the Cache2. There is not data consistency.

Write-back Cache

Here we are directly hitting the Database and once you have hit the database make sure to make that entry in the cache.

Problem: Here the problem is performance. When you updating the database you need to update all the caches based on that. So that is expensive and also slows down the performances.

What is the best way write-through or write-back?

You can go for a Hybrid(use both write-through and write-back) way. If there is critical/financial data go for the write-back way. If there is not you can go for the write-through way.

This is all about Distributed Caching and I hope you will get something about Caching and caching use cases.

Thank You!