A Brief History of Redis Caching at Personal Capital

Q&A with Ankit Kumar

We sat down with Ankit Kumar to discuss the history of Redis usage at Personal Capital — from benchmarking to the future:

What do we use Redis for at Personal Capital?

A ton of things, but primarily we use it as a:

- cache for high-performance data read and write operations, and for

- asynchronous event-based processing

Redis supports a bunch of different data types — which ones does Personal Capital use?

Just as I say that we use Redis for many purposes, we use many of their data structures.

For example, we use lists for queuing, sorted sets for storing time-series data, sets and hashes for storing some of our business-specific data.

We love sorted sets because they make it really easy to have extremely fast reads that allow us to create data visualizations for our clients to give them views of their financial wealth over time.

How does Personal Capital host Redis?

We lean on AWS Elasticache for our Redis hosting. This solves a great deal of administrative burden for us, it allows us to scale “elastically” while off-loading the maintenance to this managed service

Previous to Elasticache we only used an in-memory solution for caching: Ehcache, a Java-specific in-memory cache. Now we have best of best worlds by taking a hybrid approach of using in-memory and distributed caching.

What did Redis offer where Ehcache fell short?

I’ll list some pros and cons that we evaluated at the time we adopted Redis:

Pros:

• Seamlessly horizontally scalable where Ehcache was not,
• Alleviates resource load on our app servers by moving the cache from app server to dedicated servers administered by Elasticache,
• Allowed zero-downtime deployments by separating user sticky sessions from rolling app server deployment life-cycle (an app server dying should never impact a users session!)

Cons:

• Network latency (where there was none with Ehcache)
• Marshaling data costs

The cons are so negligible compared to the benefits, and adopting Redis was a no-brainer as a result.

What does the future hold for Redis usage at Personal Capital?

We will continue to lean on Redis to alleviate scalability issues that we may hit upon as we grow at this rapid rate.

At the same time, I’m personally excited about the opportunities to adopt some usage-specific alternatives to Redis.

For example, AWS Kinesis and its open-source alias Apache Kafka are promising to be a great alternative to Redis as a queue service.

Furthermore, there is scope for benchmarking and working with different data serializers, with the insight of which kind of data needs what level of compression and/or serialization

In summary, redis has given us a lot of performance gain in terms of not only being able to meet but also exceed our SLAs significantly.

We’ve only just scratched the surface and we look forward to exploring more of the redis feature-set such as the Distributed Locks and Redis Modules as we continue to grow.