Shrine: Best-in-Class File Attachments for Rails

Volley is building a platform for travelers to share their unique stories with the world. A huge part of this is making it effortless to share beautiful photos, which led us to Shrine. In our opinion it’s the current best-in-class way to attach images (or other files!) to models in a Rails app.

shrinerb/shrine

The Road to Shrine

… is paved with Active Storage. Our backend is written in Rails and we like to start things with the simplest solution possible, so Active Storage was the obvious choice.

It has a lot going for it. For one, Active Storage is built-in to Rails starting with Rails 5.2 (April 2018). Shipping with Rails in and of itself is a pretty strong endorsement, as the framework has a great track record (get it?) of supporting the stuff you need out of the box. Active Storage is also very easy to set up and the documentation is thorough. As a bonus it supports asynchronous image processing using “variants” with Active Job, client-side uploads, and S3 as a storage backend. It integrates tightly and simply with Active Record… has_one_attached :thing and you’re done. The list goes on…

We rolled it out, and it worked fine. For a bit. Our pages are full of images. It’s kind of our thing. In fact the front page is infinite-scrollable images loaded a dozen a time. That may not seem like a lot, but it adds up pretty quickly if you’re trying to run a lightweight server and every image involves N+1 or N+2 database access, plus generating a signed S3 URL, and default behavior of synchronously processing an image if the requested variant did not already exist.

We implemented hacks and workarounds for these all of these issues. We rolled out Cloudfront and hacked around the URL signing logic. We tweaked our queries to eliminate the N+’s. We hacked around the synchronous variant processing so it occurred completely in the background or upon upload, rather than upon download.

That’s an awful lot of hacks and workarounds for something that “just works out of the box”. Clearly Active Storage was aimed at the polar opposite of our intended use-case.

Active Storage is great for private attachments in accessed in limited numbers, but it is pretty abysmal for public attachments accessed in large batches.

We searched for an alternative which would address these issues. We were looking for something that:

1. Optimized for mass public distribution.
   That means it works with CDNs and is highly cacheable, and definitely does not insist on using expiring signed URLs.
2. Does not stress out the database by default.
   So it probably stores data about the attached file in a column rather than a buddy table(s) that lend themselves to N+1's.
3. Processes images in the background.
   Active Storage already does this, but we we need a solution that can fall back to a less preferable version if the job has not run yet, instead of falling back to synchronous processing.
4. Works with S3 out of the box.
   Totally optional criteria, but why switch when we already have this set up?

Enter Shrine

After a thorough search and a few quick spikes to tinker with our options we found Shrine. It’s highly configureable and customizable for whatever your use-case is yet it manages not to get overly complex as a result.

It quickly passed our super secret technology evaluation criteria with flying colors.

1. Does it have enough Github stars and forks to survive next month?
   Yep!
2. Is it easy to get started?
   Super.
3. Are the docs good?
   Massively. Including complete operational playbooks for scenarios you might encounter like moving to a different storage backend or changing your mind about how you process images.
4. Can I read the code if needed and actually understand it?
   Yep.
5. Oh right and what issues were we actually solving again? Does it tick boxes 1–5 from above?
   Every one of them.

Can it be optimized for mass public distribution?

Yes, and trivially. Like most functionality in Shrine, There’s a Plugin For That™.

Just make sure to tell the S3 plugin to make uploads public and add a Cache-Control header for maximum cache-ability. To put it behind Cloudfront, just use the default_url_options plugin to override the host.

Here’s the relevant part of our Shrine configuration.

Does not stress out the database by default?

Tick that box! Shrine integrates exceptionally well with Active Record, and stores all of its data in a column on the model.

As a side effect, this also means you get the standard Active Record semantics around saved/unsaved data in columns. This is a nice plus because you can apply the same thinking as you do to any other data on the model, instead of having to treat it specially and tip-toe around transactional correctness.

This approach not only solves our performance concerns with N+1 queries, but it also puts all the power in your hands as the developer to make the tradeoffs relevant to your use-case. If you need to attach multiple images or need a polymorphic buddy table, you can still do that quite easily, it’s just not forced upon you.

So how’s it work? Simple. Your migration simply adds a text column, and integration code in the model is a one-liner:

I’ve excluded ThingImageUploader, because they’re so customizable ours is unlikely to be relevant to your usage. They allow you to do everything from validation, to processing, to defining multiple versions, to metadata extraction, you name it, and it all ties in magically.

Processes images in the background?

Yes it can! Like everything in Shrine, it’s up to you to decide to do this or not, and it’s not forced upon you. Just a couple lines of configuration is required, and it’s up to you to use whatever background jobs framework you’d like. We use Sidekiq, but you can replace the code in the configuration blocks with whatever you use.

If an image has not been processed yet, Shrine falls back to serving a signed S3 URL for the raw uploaded content instead. This is great because it gets a consistent user experience while preventing the raw uploaded content from being further cached somewhere. If everything is going smoothly, the only person that’s going to see the raw version is the uploader themselves. If your job queues are backed up, people get a degraded experience of having to download larger unprocessed images, but the site continues to work. This is exactly the fallback behavior we were looking for.

Supports S3 out of the box?

We already covered that, but yes it does. It also supports a laundry list of other storage backends and if you’re doing something super special new ones border on trivial to write too.

In development we use the filesystem backend. In testing we use the memory backend. This allows us to keep tests snappy and the development experience simple and functional.

So what’s the verdict?

Shrine is great and we highly recommend it! It’s plenty flexible enough for any use-case you might encounter without getting complicated or getting in your way. The documentation is top-notch, which makes getting started easy and future maintenance easy as well. Give it a spin!