Redis Vishwaroopam, Part-1
In the Bhagavad Gita, Lord Shree Krishna reveals his Vishwaroopam (the universal form) to Arjuna. Struck by this divine presence, Arjuna was immediately humbled, realizing the form’s mighty and terrifying aspects. He recognized his mistake in treating Krishna merely as a buddy, a subject of casual jests. Krishna, in his compassion, reassured Arjuna and returned to his familiar form at Arjuna’s request.
Now why am I diving into this?
For me, Redis epitomizes Krishna’s Vishwaroopam, with myself in Arjuna’s shoes, awestruck by Redis’s capabilities. While I was aware Redis was more than a simple data store, the extent of its applications and its critical role in various systems were revelations to me.
To my fellow readers just out of the academy, new to the industry, or unfamiliar with Redis, Redis is a key-value store that processes data in memory, catering to various use cases through an extensive range of data structures. It is predominantly utilized as a cache due to its low latency, a fact well-known across the industry.
Here’s the scoop:
While Redis is famously used as a cache, and perhaps less commonly known, as a NoSQL database among more experienced developers, it also excels as a vector database. This allows for the development of chatbots using RAG, building recommendation systems, and more. You can create vector embeddings and store these vectors in Redis to develop many exciting applications. This capability, when integrated with OpenAI models (or other open source models for that matter) and fancy architectures, fuels numerous so-called “AI” applications currently available in the market.
I discovered these extensive features accidentally while exploring the documentation on Bloom filters — a sophisticated data structure offered directly by Redis. This sparked my realization that it was time to delve deeper into Redis’s vast potential.
Given the breadth of Redis’s capabilities, covering them all in one blog post is impractical. Therefore, I plan to explore these features gradually and share my findings through this Vishwaroopam series.
Since you’ve read this far, let’s briefly discuss the Bloom Filters now that I’ve mentioned anyhow (Please, I insist!).
Bloom Filters
A Bloom Filter is a probabilistic data structure that uses minimal, fixed-size memory to determine if an element is present. It can definitively confirm the absence of an element (negative response) but only estimates the presence (positive response). Bloom filters store just the hashed representation of elements, trading off some accuracy for efficiency. Expect a false positive in one out of every several checks — a tradeoff often accepted due to the structure’s space efficiency and speed.
Example applications I’ve explored:
- Web Scraping: To check if a URL has been visited recently, preventing loops. This is particularly useful if you’re developing a search engine with a designed crawler, as bloom filters can be periodically rebuilt.
- Fraud Detection: As hinted at in Redis documentation, creating a filter per user allows checks based on user location to ascertain if a transaction might be fraudulent.
I’m pretty sure there are many more applications, but let’s consider these two at the moment and move on.
For those interested in a deeper exploration, I recommend investigating how Bloom filters are implemented and the specifics of the hashing functions used. Also suggest following some good tutorials out there or (even better) going through the documentation and trying to implement something small.
Conclusion
Redis, much like the mythological Vishwaroopam, is both formidable and awe-inspiring. Its versatility from simple caching solutions to complex AI-driven applications underscores its profound utility in modern technology. As we continue this series, we’ll uncover more of Redis’s expansive capabilities, much like peeling back layers to reveal the core of what Redis can truly offer to the world of technology. Stay tuned for more insights and explorations in the upcoming parts of Redis Vishwaroopam.
