Caching strategies — Read, Write, Invalidation and Eviction

What is Caching?

Caching is a technique to store and retrieve frequently accessed or computed data faster to improve performance.

Why do we need Caching?

Let's try to feel its importance via an example. Let’s say you log in to your bank website and want to see your account balance. A delay of 1 second will be acceptable in this case. But let’s say you log in to Facebook and if it takes 1 second for the next feed to show up, you would switch to a different platform.

So it makes much more sense for Facebook to cache your newsfeed and any images/videos related to it.

Caching Layers

There are various layers where caching can be used, below are some examples.

1. Web Browser Caching: Storing web page resources (e.g., images, stylesheets, scripts) locally on a user’s device to reduce load times for subsequent visits.
2. CDN (Content Delivery Network) Caching: Storing static content (images, videos, stylesheets) for serving it from servers closer to the end-users. Eg Agora.
3. Web Content Caching: Static files can be served directly by reverse proxy. Even dynamic content can be cached provided it doesn't change frequently. TTL can be configured while caching dynamic content. Eg Nginx can take care of both of these requirements.
4. Database Caching: Storing frequently queried database results or objects in memory or dedicated cache to reduce the time needed for database access. It reduces the load on the database.

Now let’s talk about some caching strategies.

Credit: ByteByteGo

Cache WRITE Strategies

There are three write strategies:

Write-Through

• Write operation is considered successful when data is written to both DB and Cache.
• This strategy ensures no data loss or mismatch.
• It has high latency among the 3 write strategies.

Write-Around

• Write operation is considered successful when data is written to just DB.
• Cache is updated when there is a cache miss.
• No chance of data loss but less performant for reads than write-through because reading recently written data will always lead to cache miss.
• It has medium latency among the 3 write strategies.

Write-Back

• Write operation is considered successful when data is written to just Cache.
• Data is written to DB asynchronously in batches.
• High chance of data loss if the cache server fails.
• Lowest latency among the 3 write strategies but has high throughput. Useful for write-intensive applications.

Cache READ Strategies

There are two read strategies:

Cache-Aside

• In case of cache-miss, the application fetches the data from DB and updates the cache.

Read-Through

• In case of cache-miss, cache fetches the data from the DB, updates itself and then respond to application.

Cache INVALIDATION Strategies

Entries in a cache may become invalid or outdated over time. TTL (time-to-live) is used to deal with outdated cache items. We can use two different approaches to deal with outdated items using TTL:

• Active expiration: This method actively checks the TTL of cache entries through a daemon process or thread.
• Passive expiration: This method checks the TTL of a cache entry at the time of access. But what if key is not accessed? Well, you can read here how Redis handles it.

Cache EVICTION Strategies

SSD is costly but Cache is even more expensive. So with limited cache size, not everything can be stored in cache. Once the cache reaches its capacity limit and new data needs to be written, one of the below cache eviction strategy is used.

1. Least Recently Used (LRU) — Evicts the least recently accessed items first. It assumes that items that have not been accessed for a longer time are less likely to be accessed again soon.
2. Most Recently Used (MRU) — Evicts the most recently accessed items first. It assumes that recently accessed items are more likely to be accessed again soon.
3. Least Frequently Used (LFU) — Evicts the least frequently accessed items first. It considers the frequency of access over time, not just recency.
4. Most Frequently Used (MFU) — Evicts the most frequently accessed items first. It assumes that items accessed frequently will continue to be accessed frequently.
5. Random Replacement — Selects a random item for eviction. It doesn’t consider access patterns or frequencies.
6. First-In-First-Out (FIFO) — Evicts the oldest items first, following the order in which they were added to the cache.
7. Size-Based Eviction — Evicts items based on their size or weight in the cache. Useful when cache entries have different sizes.

Feel free to leave comments and don't forget to clap if you liked the content.