Optimizing and Scaling Your Rails App-Part II
Avoiding N+1 queries
Congratulations! Your awesome app is continuing to grow, and people just can’t get enough of it. You know that scaling your app requires some adjustments, so you’ve made extensive use of caching (need a refresher? check out my earlier post here) to lighten the load on your database. But with your market share continuing to grow and competitors right at your heels, you need to make sure your app stays lightning fast. Enter eager loading.
Active Record is lazy, and that’s a good thing. Generally, it will only load associations as you need them, preventing you from returning from more than you need with every call, and making load times (generally) faster. This is called lazy loading.
The problem arises when you have code like this:
@posts = Post.limit(10)
@posts.each do |post|
It looks pretty benign. All it’s doing is retrieving ten Post records, and printing out the name of the associated author. The problem is that in the background, active record is generating 11 queries! 1 query for the 10 Post records, and then another query for each record to find the associated author. Because Active Record was being lazy, it never loaded the authors, so it is required to issue a new query for each one. This is called the N+1 query problem, and it can be a disaster for a large scale application. If you’re only searching for ten records, it won’t be much of an issue, but when working with more complex apps, N+1 queries can really slow you down.
Luckily, there is a simple solution. Active Record allows you to make use of a technique called eager loading, which will cut the number of queries in our example down to 2! Eager loading will load all of the specified associated data along with data requested, reducing the number of queries drastically. Here’s how to use it:
@posts = Post.includes(:author).limit(10)
@posts.each do |post|
By adding in .includes(:author), we’re telling Active Record to eager load the author association, thereby bypassing the need for all those individual queries in the each loop. Want to eager load multiple associations? Not a problem. Just include them as comma separated arguments in the includes method and you’re good to go. You can even tack on a .where clause, though the official Rails docs discourage this practice.
But wait, you say! I must have N+1 queries all over the place in my app! How can I fix this issue without reading through all of my code line by line? The answer, of course, is a gem. Bullet, when included in your gemfile, will notify you with an alert whenever you encounter an N+1 query, and tell you where and how to fix it. Additionally, Bullet will tell you where you may have been overeager in your eager loading, and recommend changes where necessary. Always use your judgement when following Bullet’s suggestions, as they may not be ideal for your particular case. In general, however, Bullet will put you on the right track toward speedier performance.
One final thought: Depending on your needs, it might be worth checking out the Goldiloader gem. Goldiloader purports to provide the “just right” amount of eager loading, by prefetching associated models, but only when required. This ensures that your app is performing at its fastest. There are situations where Goldiloader is not a good fit though, so proceed with that in mind.