Unveiling Redis’ Hidden Gems
Redis Beyond Caching
When someone says the word Redis, the first thing that comes to mind is often ‘cache.’ But Redis is far more than just a cache. It’s a powerful tool that empowers you to build robust systems.
In this blog, let’s delve into Redis’s multifaceted nature and explore how it can enhance your applications. From its role as a lightning-fast cache to its capabilities as a robust database, message broker, and more.
Also, we will look at how we at Bajaj Health leverage Redis to achieve high throughput and resilience.
Let’s examine various use cases and uncover how Redis can serve as a critical component in building scalable and performant systems.
“Redis: A poor man’s everything’ — database, cache, queue, broker, data stream, and whatnot”
Table of Contents
- What is Redis?
- Redis vs Redis Stack
- Exploring Redis’ Diverse Features
a. Persistence Options
b. Data Structures
c. Transactions
d. Lua Scripting
e. Redis Pub/Sub
f. Redis Streams
g. Bloom Filter
h. More… - How Bajaj Health Leverages Redis with Microservices
- Conclusion
What is Redis?
Redis stands for Remote Dictionary Server. It is an open-source key-value store. Redis is written in ANSI C which ensures that it is highly portable and can be compiled and run on a wide range of platforms.
Unlike traditional databases, Redis primarily stores its data in memory, making it fast for reads and writes.
Also, you can configure Redis to persist your data. Redis provides high throughput with low latency. Redis comes with support for replication and partitioning out of the box.
Redis vs Redis Stack
Redis Stack extends the core features of Redis with additional capabilities. It includes modules for probabilistic data structures such as bloom filter, queryable JSON documents, time series data support, etc.
If you need the core functionalities of caching, messaging, and session management, Redis might suffice.
If your application requires more advanced features like full-text search, working with JSON, time-series data, graph structures, etc then you should consider using Redis Stack.
Exploring Redis’ Diverse Features
Let’s look into the most common features offered by Redis one by one.
Persistence Options
While Redis primarily stores data in memory however persistence option allows Redis to be used as a primary database.
- Redis Database (RBD): This is also known as Snapshotting. In this periodic backups of your data are created at specific intervals, allowing recovery in case of server restarts.
- Append-Only File (AOF): In this every write operation is logged to a file. This offers better data durability but at the cost of some performance.
You can choose the persistence option that best suits your application’s needs, balancing speed with data safety. However, using both RDB and AOF together is common for optimal persistence and recovery capabilities.
Data Structures
At its core, Redis is a data structure server. It supports various data types, making it more than just a simple key-value store.
Redis provides various data structures such as Strings, Hashes, Lists, Sets, Sorted Sets, Bitmap, Hyperlogs, Geospatial Indexes, etc.
Each Data Structure has its unique capability and the existence of a wide variety of data structures allows Redis to be used for multiple use cases.
Transactions
Redis transactions provide a form of isolation between commands within a transaction and other operations happening concurrently in the database.
This allows performing multiple commands as a single transaction.
This ensures that either all commands in the transaction are executed or none at all which is essential for maintaining data consistency where multiple Redis commands need to be treated as a single unit of work.
Lua Scripting
Redis allows users to execute custom scripts directly on the server. Using Lua Scripting we can implement atomic operations across multiple Redis commands, preventing race conditions and ensuring data integrity.
We can perform custom aggregation or filtering on Redis data before returning results to the client. Lua scripting allows you to define and execute such logic directly within Redis.
Redis transactions and Lua scripting are both useful for atomic operations, but they serve different purposes. Redis transactions are suitable for executing sequences of Redis commands atomically, while Lua scripting allows for more advanced operations and custom scripts within Redis.
Redis Pub/Sub
Redis supports publish/subscribe messaging patterns, facilitating communication between different parts of the application in real time.
Pub/Sub in Redis is lightweight and efficient, making it suitable for building real-time messaging systems, chat applications, and event-driven architectures making it a great alternative to Kafka for simpler use cases.
Redis Streams
Redis can collect and distribute data in many solutions enabling the creation of real-time applications, event-sourcing systems, message queues, and distributed logging solutions.
Redis streams provide highly efficient inserts and reads. It’s similar to a message queue, where data is stored as an append-only log and consumers can read from it sequentially or randomly.
Bloom Filters
A Bloom filter is a probabilistic data structure used to test whether an element is a set member. It is very space-efficient and works by hashing elements and storing the hash values in a bit array.
Bloom Filters have extensive applications and are crucial for the implementation of recommendation and personalization systems, duplication detection, spell-checking, network routing, and more.
More…
It offers features for document storage, vector, and time-series databases.
Redis also supports Pipelining which is used to improve performance by sending multiple commands to the server in a single request.
Additionally, Redis includes built-in mechanisms for managing cache efficiently, such as LRU eviction and customizable LFU.
How Bajaj Health Leverages Redis with Microservices?
We at Bajaj Health, with hundreds of microservices, utilize Redis to its full potential wherein Redis serves different purposes for different microservices depending on the nature of its use case.
Caching
Redis serves as a caching layer for a significant number of our microservices. This helps us avoid expensive Network I/O, Disk I/O, or Computation by caching DB operations or storing computed Results.
We take advantage of both the Lazy Cache Population and the Eager Cache Population. In lazy cache population, data is only fetched from the database or the original data source when it’s requested for the first time. While in eager cache population, data is preloaded into the cache before it is requested.
Contest Ranks and Leaderboards
In Activity Tracker, one of the highly anticipated features that provides you with syncing steps and fitness data from your mobile and wearable devices takes advantage of Redis for Contests Ranks and Leaderboards.
We use Redis’s Powerful Data Structure Sorted Set for real-time calculation of User Ranks as it automatically sorts members based on their score along with Lua Scripting allowing us to perform atomic operations on the server side, reducing the need for multiple round-trips between client and server.
Concurrent Request Handling
Redis Pub/Sub emerged as a crucial solution in one of our microservices, where it played a pivotal role in managing concurrent requests efficiently. Leveraging the low latency of Redis, we implemented a robust locking mechanism that ensured seamless handling of concurrent operations.
In this scenario, the goal was to create a wallet for our GMC Customer. However, we needed to ensure that only one wallet creation process occurred, without failing any requests and blocking all other requests until the wallet was successfully created.
In-Memory Cache Invalidation
In our Content Management System, we have used in-memory caching for better performance. However, it presented a challenge with cache invalidation.
In a production environment with multiple application instances, each instance caches data independently. Here, API to clear the cache wouldn’t work effectively, as it would only invalidate the cache on the instance that handled the request, leading to inconsistency.
To address this, we leveraged Redis Pub/Sub. Whenever the cache-refresh API is called, a message is published, notifying all subscribed application pods. Each pod receives this message and promptly clears its in-memory cache, ensuring data consistency across all instances.
Tasks Scheduling
In addition to Redis’s pub/sub mechanism, we’ve also harnessed the power of Redis Queues, specifically utilizing the NodeJS’ Bulls Queue library, to schedule and manage certain events within our system.
This approach provided us with easy scheduling capabilities that were not easily achievable with Kafka.
And More…
One notable application is its use in implementing rate limiting for various cases, such as limiting the number of times a doctor is being called.
It has also been instrumental in implementing locking mechanisms for specific resources, such as ensuring that only one instance of a service can book an instant consultation with a doctor at any given time.
Redis's versatility has enabled us to implement a wide range of features and functionalities, enhancing performance, scalability, and reliability.
Conclusion
While there are numerous alternatives to Redis, this blog has aimed to spotlight the remarkable capabilities of Redis and advocate for its consideration in designing scalable applications.
Whether you’re building a small-scale application or architecting a large-scale system, consider Redis a valuable asset when designing and building your scalable application.
If you enjoyed this blog, I invite you to follow me and connect with me on LinkedIn.
“Redis : The Swiss Army Knife for Developers — because why settle for just one tool when you can have them all?” 😄
