Using Redis to Supercharge Your Data Delivery
If you are looking to improve performance as a software developer, caching is one of the best paths forward. Caching hastens the delivery of content using previously saved information, and there are many ways to take advantage of caching strategies. To deliver the best performance, a multi-layered caching architecture can smooth out variability in client latency or backend service response times. The term multi-layered caching architecture refers to the practice of utilizing multiple caching strategies that, when combined, offer the greatest performance improvement. For example, web browsers provide one form of caching when building web applications but combining browser-based caching with server and/or data layer caching will deliver better performance than any individual technique.
Below is a nonexhaustive list of potential locations where a development team may leverage caching today. The diagram’s scale represents the proximity of the cached information to the end user’s machine, with the ones on the left being on the end user’s machine itself and the one to the right located at the source data system. If the flow of requests travels from the user to the data source, and back to the user, a cache at any point allows for information to be returned early, delivering more timely results to the end user.
Quick Background
Let us focus on the server-adjacent layer and dive a bit deeper into one caching technology, Redis. Redis is an open-source, in-memory, data storage system that lends itself well to a caching solution. Redis offers flexible installation mechanisms including manual installation, Docker instructions, AWS via Elasticache, Azure via Azure Cache for Redis, or Google Cloud via Memorystore. The popularity of Redis ensures that language SDKs are readily available and easy to leverage in any project. The command documentation available on Redis’ website is comprised of easy to digest information along with code examples which can be manipulated directly within the documentation.
At FactSet, our applications cater to financial professionals whose workflows require answers and insight as fast as possible to ensure timely responses to their clients. In line with the theme of multi-layered caching strategies, our applications take advantage of multiple caching strategies to ensure optimal performance of our solutions. To supplement caching in one area of our product, Redis was introduced to provide low latency responses for data which does not change frequently.
Problem Definition
When researching a particular financial instrument, FactSet needs to determine what content sets contain relevant data and information. Due to the breadth of the content sets available on our platform, the identification of relevant content sources may involve making many network requests for different services. Making dozens of calls delays the user from obtaining critical information. While any individual call may be fast, in aggregate they could take multiple seconds and form a bottleneck, degrading user experience. Enter Redis, which can help reduce data fetching frequency and smooth out the overall response times.
Server-Adjacent Caching
In our application architecture, requests to obtain the universe of content are routed through a middle tier layer. This layer provides the perfect opportunity to attach a server-adjacent cache that collects information for faster future delivery.
Our application layer makes a request to the middle -tier to obtain the content universe. The middle tier quickly checks Redis for a cached result that can be instantly delivered. If the cache does not have an answer for the request, the middle tier issues requests to the various backends to collect the content universe. Returned universe data is added to the cache with an appropriate expiration time to ensure the information does not become stale. Subsequent requests to the middle -tier to obtain the universe for the same instrument can instantly return the response from the cache, saving precious time.
Redis can facilitate a caching workflow in several ways and a combination of storage structures, rather than a singular approach, optimizes performance. To store the raw data model, we leverage SET/GET calls to hold onto JSON based for a given key. For the code examples that you will see, they will contain a mix of JavaScript and Redis CLI commands to avoid promoting any specific Redis JS SDK library.
Optimization Note
If you will be setting multiple key/value pairs in the cache at one time, make sure to leverage a pipeline construct within the SDK you are utilizing. The pipeline will avoid round trips to Redis for each operation by sending the workload in bulk.
At this point, we have a basic caching structure for JSON content that we could retrieve via GET operations.
To avoid an expensive SCAN operation to gather insights about the entire cache, we also collect information relating to usage within a sorted set structure. The usage information benefits other workflows. The usage information is important as a seed for background jobs that keep the cache up to date. The sorted set is maintained via Redis’ ZINCRBY command. This command simply increments a numerical value for a given “member” of the set. As our middle tier layer receives requests for a given instrument, we increment usage with one simple command:
Redis gracefully handles potential edge cases, such as if the member does not exist in the sorted set already. This sorted set builds up usage/hits as requests come through the middle -tier. Unlike the general SET command, the sorted set does not have a built-in mechanism to automatically expire the data. While it may be valuable to know what the usage may be over a particular date range, say one week or one month, it would not be too useful to hold indefinitely. This brings us to an accessory to the cache mentioned briefly above, background jobs.
Going beyond with automation
Now that we have our basic cache in place, it is time to pair the cache with a background process that keeps it updated, thus extending its life. One downside to the basic implementation above is that we had defined the expiration time for any instrument in the cache to be one day. Considering a cache which expires every day, let us say that the following day every user accesses a symbol from yesterday which is no longer cached during their first several hits to the application. If the user never makes it past these first several hits, they may feel that the product is always slow as they were never able to experience the power of the cache.
To solve this challenge, we extend the initial approach by leveraging multiple sorted sets. These extra sorted sets help define weighted usage over a given date range. We combine the weighted usage to build a rolling picture of the popularity of instruments which helps feed the background job or other systems.
Let us continue with the daily theme and extend the sorted set storage into one table per day for an entire week. This means we would create seven sorted sets in Redis to store seven rolling days of traffic. We then need to adjust the usage incrementation logic to incorporate the current day, from the server’s perspective, that the usage relates to.
Once this code has been in place for at least a week. We have seven sorted sets collecting usage information. Since these are unique sets, we would need to combine them in some way to get the complete picture of the most popular symbols over the rolling week. Redis makes this easy with a command called ZUNION, or ZUNIONSTORE. This command can be slow since it needs to scan every set being merged. We will come back to these commands in a moment but, to overcome the need to run ZUNION frequently, we can materialize the union and keep it updated as we track usage from our middle tier. We achieved this with the creation of one more sorted set that contains the rolling seven-day union. This union set will store a weighted value of the usage to help us monitor any shift in popularity of the instruments.
In the snippet above, we start to introduce some weightings to the stored usage information. For the ZUNION function, weightings can be applied to each set during the merger to augment the resulting set’s rankings. The perfect weighted ranking algorithm involves a great deal of complexity, but we can keep this simple by assigning the current day a weight of 10 points and each subsequent day further back in history a weighting of one less until we reach four points for the weight six days ago in this rolling seven-day scenario.
Now, onto the scheduled job. This daily operation is responsible for three main tasks. First it will obtain the weighted usage across all sorted sets. Then it will clear the current day’s usage set to provide a fresh location to gather data for today. Lastly, it will process over the instruments in the sorted union and refresh the cached JSON data. For the first piece, obtaining the weighted usage, we utilize the ZUNIONSTORE command to compile the combined usage and store it in the union set shown above.
This command is a bit wordy, but it takes in N-number of sorted sets with N-number of weighting values to apply and combines the data into one output set. After this command has run, we have an up-to-date union table that represents weighted rankings for all the traffic over the past seven days. We can use this information to sequence a scheduled job more accurately such that it processes instruments in order of popularity when refreshing the cached JSON. The introduction of the scheduled job allows us to extend the expiration time for any given instrument from one to seven days. With a cache that now lasts for a week, we would no longer have the scenario where users are accessing instruments from the previous day which are not in the cache. The existence of the scheduled job allows for data to remain cached for a much longer period, providing the greatest potential for cache hits and near instantaneous responses.
Stats
When we first started this project, we encountered lookup times within the range of seconds. We needed to significantly reduce that response time to improve the user experience. We have found a massive reduction with the introduction of this cache. Excluding latency and focusing on pure server execution and lookup/processing times, here are the results for various percentiles:
Wrapping up
This implementation helped us significantly improve performance within a key area of FactSet with low to moderate engineering and maintenance effort. A solution such as the one described here can buy engineering teams more time to make larger improvements to a data model or system. Out of the box approaches and implementations of server-adjacent caching systems can supercharge response delivery. Despite the significant improvements that we achieved; we can continue to expand upon the optimizations within the server-adjacent caching layer of our applications.
Author: Alan Cenkus (Senior Director, Engineering)
Editors: Gregory Levinsky (Marketing Content Specialist) & Josh Gaddy (VP, Director, Developer Advocacy)
