Gabriel Kent
Aug 15 · 7 min read

Once upon a time, Rails had a feature under ActiveRecord called: IdentityMap.

Ensures that each object gets loaded only once by keeping every loaded object in a map. Looks up objects using the map when referring to them.

When does Rails pro-actively cache ActiveRecord objects?

Answer: On associations when the inverse_of option is properly configured or inferred by convention.

Navigating to the associated record(s), then navigating back, should not incur an additional query.

# 1 query for blog_posts table
> blog_post = BlogPost.first
SELECT * FROM `blog_posts` LIMIT 1
# 1 query for comments table
> comment = blog_post.comments.first
SELECT * FROM `comments` WHERE blog_post_id = blog_post.id LIMIT 1
# No query made
> comment.blog_post
# Objects are identical
> comment.blog_post.object_id == blog_post.object_id
=> true

If a similar record is freshly queried and happens to be associated with the same record, the same cache is not accessed and an extra query is made. Ideally, we should be able to realize that the associated record had already been queried and, therefore, an additional query is unnecessary.

# 1 query for comments table because it's queried directly instead of through the association
> comment = Comment.where(blog_post_id: blog_post.id).first
SELECT * FROM `comments` where blog_post_id = blog_post.id LIMIT 1
# Cascading effect: 1 query for blog_posts table because the comment wasn't cached
> comment.blog_post
SELECT * FROM `blog_posts` WHERE id = comment.blog_post_id LIMIT 1

Now an extra query or two may not seem like much, but if we consider the above example, there could be thousands of comments which are associated with any given blog. If those comments are loaded outside of the blog_post association, then attempting to reference the blog_post association from within each comment will result in N blog_posts table queries even if they all belong to the same BlogPost!

Why should you care?

Answer: high scale. At least, that’s why we care.

Scaling is exciting, but also painful. As customers begin to increase traffic, engineering teams may notice significant database load and contention. However, due to the size and complexity of most codebases, it can be extremely difficult to pinpoint specific inefficiencies around queries. We recently found ourselves in such a situation where the sheer volume of database reads began adversely affecting our services. These reads were largely due to the same associated records being re-queried across different source records.

Our Story

We were able to determine which tables were receiving the most queries, but that still didn’t quite help with pinpointing the exact code areas. We identified a few areas that were effectively making redundant queries, but we realized that targeted fixes were not very feasible nor scalable. We needed a broader, more effective solution to reduce redundant queries in order to reduce our overall database load.

IdentityMap could have been that solution for us, but it was deprecated after Rails v3.2.13 due to some noticed inconsistencies. According to our understanding of the inconsistencies, the feature was likely trying to support too many edge cases. All caching strategies have weaknesses and eventually break down if the usage is not properly scoped. Even so, there are rumors that this feature may be reintroduced in future Rails versions, but we didn’t want to wait that long.

Therefore, we set out to partially re-adapt the previous implementation of the IdentityMap feature to Rails 4.

What is appropriately scoped caching?

Caching is dangerous if not done correctly. For example, making decisions based on outdated data as if it was current. In our opinion, the following rule of thumb should be respected:

Don’t apply caching if the process is expected to react to changes during the caching period. i.e. Don’t cache when mixing reads and writes.

An example candidate for caching might be a nightly billing task which aggregates billing data for the past month. That kind of task is likely not expecting last minute updates while it runs. It assumes that the state of the world remains constant while processing.

What did we do and how did we do it?

Spoiler: we didn’t completely remake the IdentityMap feature set. We remade a valuable portion of it and also added in some new features of our own:

  • SingularAssociation#find_target cache read and writes
  • Persistence#instantiate cache writes only
  • Stats recorded for cache writes, misses, and hits
  • Dry run caching mode (no loading from cache; read attempts still increment)

SingularAssociation & Persistence Caching

Whenever a SingularAssociation record is accessed, we either load it from the cache or query it fresh and store it in the cache using the owner class + id or record class + id as the cache key. Separately, whenever a record is instantiated through Persistence, we also write it to the cache, but only using the record class + id since the owner is unknown at this point. In essence, we currently write to the cache in two locations, but read from it in only one location. There are other opportunities within ActiveRecord to also read from the cache, like within the definition of find, but we wanted to minimize the scope of change in order to reduce risk.

Enough stalling; here’s how we did it. We wrapped the find_target and instantiate methods in mixins like so:

module Mixin
module SingularAssociation
def find_target
foreign_key_value = target = nil
begin
if (im = ::IdentityMap.instance)
fk = owner[reflection.foreign_key]
target = im.cache_read(klass, owner, fk)
end
rescue => ex
# log error
end
if target
set_inverse_instance(target)
else
target = super
begin
if (im = ::IdentityMap.instance) && target && !fk.present?
im.cache_write(target, owner)
end
rescue => ex
# log error
end
end
target
end
end
end
module Mixin
module Persistence
module ClassMethods
def instantiate(attributes, column_types = {})
instance = super
begin
if (im = ::IdentityMap.instance)
im.cache_write(instance, nil)
end
rescue => ex
# log error
end
instance
end
end
end
end

We then prepended the mixins to the appropriate ActiveRecord classes during app initialization:

class ActiveRecord::Associations::SingularAssociation
prepend Mixin::SingularAssociation
end
class ActiveRecord::Base
prepend Mixin::Persistence
class << self
prepend Mixin::Persistence::ClassMethods
end
end

To further reduce risk, we went with a conservative block wrapper in order to contain the caching to a particular process or block of code. Consider the following (unoptimized) example where we can utilize our block wrapper for caching:

# Note how the example traverses multiple associations for each comment in order to obtain the appropriate emailIdentityMap.with_caching("email recent comments") do
Comment.where("created_at > ?", 1.week.ago).each do |comment|
EmailProxy.send(comment.blog_post.owner.email, comment.text)
end
end

In this example, we are likely to come across multiple comments that are associated with the same blog_post and, thus, the same owner.

Without caching (or optimizing through other means), this procedure would normally result in exactly 2N+1 queries (where N = recent comments). One query for getting the list of recent comments and two queries for each comment in order to retrieve the associated blog_post and owner.

With caching, the maximum queries would be 2N+1 if we assume that every comment is associated with a unique blog_post AND owner. However, if every comment was associated with the same blog_post AND owner, there would only be 3 queries in total. The blog_post and owner would be cached and re-used for every comment after the first. Potentially reducing 2N+1 queries down to 3 is a massive performance increase and highly scalable!

Performance Evaluation through Cache Statistics

As valuable as the partial IdentityMap caching has been, we also saw great value in simply reviewing the cache statistics after implementation. We found that there were cases where the natural Rails association caches were being broken due to explicit and redundant where queries. These showed up as a large number of cache writes which were easily visible:

# Top cache writes
[
[:"Data::Field", 8368], <--- highly excessive cache writes
[:TrackingModel, 19],
[:"Details::Order", 12],
...
]

The stats were also helpful for simply proving to ourselves that the caching was working and having a meaningful impact.

# Top cache hits
[
[:"Details::Order", 10], <--- every query matters
[:NumberGroup, 6],
[:Company, 5],
...
]

The hit stats may not look particularly impressive as far as query saving, but the overall improvement is actually much larger because of nested associations. Cached records also have their own associations cached by Rails. If the record had been re-instantiated, then its associations would have been also. The Rails caching bypasses our IdentityMap feature so we don’t see those cache hits that would have previously been misses!

Consider the following case study:

2 runs of the same process with the same inputs1. Caching ENABLED (Dry Run)
Cache read attempts: 92 <--- db reads in dry run mode
Cache read hits: 44
Cache read misses: 48
2. Caching ENABLED (Normal Run)
Cache read attempts: 75
Cache read hits: 27
Cache read misses: 48 <--- db reads in normal mode
Results:
92 -> 48 observed db reads
~1.9x improvement

Notice how the read attempts decrease when caching is fully in use. The attempts go down from 92 to 75. Without the dry run test, we may have incorrectly assumed that the net reduction in db reads was only 27 when it was actually 44 since we went from 92 to 48!

Future Plans

We currently have a few ideas for improving and extending the usage of our version of IdentityMap:

  • Move source code to gem
  • Intercept additional ActiveRecord instantiations for writes and reads
  • Log caller traces when write stats exceed a threshold in order to help pinpoint redundant queries
  • Publish metrics and define alerts around cache stats
  • Opportunistically extend caching to other appropriate areas of our platform!

TL;DR

Appropriate and intelligent caching can help identify and reduce redundant database queries.

Be safe and happy caching!

Invoca Engineering Blog

Invoca is a SaaS company helping marketers optimize for the most important step in the customer journey: the phone call.

Gabriel Kent

Written by

Software Engineer at Invoca

Invoca Engineering Blog

Invoca is a SaaS company helping marketers optimize for the most important step in the customer journey: the phone call.

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade