How to store files to S3 by Paperclip

Which gem do you use when you want to store files to S3?
Paperclip? Carrierwave? Dragonfly? I often use Paperclip because it is simpler than others.

In this article, I’ll introduce how to store files to S3 by Paperclip, and refer to what we have to care then.

How can I store files to S3 by Paperclip?

Paperclip stores to local file system in default, so we have to change some options like below.

config/initializers/paperclip.rb

Paperclip::Attachment.default_options.merge!(
storage: :s3,
bucket: <bucket_name>,
s3_credentials: {
access_key_id: <access_key>,
secret_access_key: <secret_key>
}
)

As you can see, we have to set :s3 to storage, <bucket_name> to bucket, and s3_credentials.

What do we have to care?

As I previously described, it is easy to change Paperclip options to store to S3. But it is not enough when you use Paperclip in various environments (development, test, staging, etc…). There are 4 points to use in that environments.

We don’t want to store files to S3 in test environment, and sometimes in other environments.

First point is about when you store files to S3. If all files are always stored to S3, it is difficult to write specs, so I think they are stored to local file system in test environment. In addition, you may want to store in the same way in other environments when you don’t prepare AWS yet.

if ENV[‘NO_S3_ATTACHMENT’] || Rails.env.test?
Paperclip::Attachment.default_options.merge!(
path: ‘:rails_root/tmp/paperclip/:filename’,
url: ‘:rails_root/tmp/paperclip/:filename’
)
else
Paperclip::Attachment.default_options.merge!(
storage: :s3,
bucket: <bucket_name>,
s3_credentials: {
access_key_id: <access_key>,
secret_access_key: <secret_key>
}
)
end

As you can see, in test environment or when you set ‘NO_S3_ATTACHMENT’ to environment value, you can store to local file system.

We should set private permission to uploaded files

Second point is about S3 permission. In many cases, we should set private permission to uploaded files to prevent anyone could get them, right?

Paperclip::Attachment.default_options.merge!(
storage: :s3,
bucket: <bucket_name>,
s3_permissions: :private,
s3_credentials: {
access_key_id: <access_key>,
secret_access_key: <secret_key>
}
)

We should set other S3 information

Third point is about other S3 information. For example, if you don’t set a path option, you cannot control where files is stored. In addition, sometimes S3 host name is required, so you should set it, too.

Paperclip::Attachment.default_options.merge!(
path: ‘/xxxx/:filename’,
storage: :s3,
bucket: <bucket_name>,
s3_host_name: <host_name>,
s3_credentials: {
access_key_id: <access_key>,
secret_access_key: <secret_key>
}
)

We should set v4 to s3_signature_version

Final point is about S3 signature version. You should set :v4 to s3_signature_version. Let me explain about this in detail in next section.

AWS.config(s3_signature_version: :v4)

About S3 Signature version

If you set private to s3_permissions, you must get the pre-signed url to access the file by calling expiring_url.

In paperclip (now latest version 4.3.6, and uses aws-sdk gem 1.66), default s3 signature version is v3, so you can get the following url when you call expiring_url.

paperclip_object.expiring_url(5.minutes)
=>https://xxxx.s3-ap-northeast-1.amazonaws.com/xxx.pdf?AWSAccessKeyId=AKIA…&Expires=1459224186&Signature=….

Is this a problem? Maybe sometimes yes. It’s because anyone can get the AWSAccessKeyId. Of course, that is no problem only that, but it might cause something which leads to security issue, so we should not inform it if possible.

So I believe that we should set like below.

AWS.config(s3_signature_version: :v4)

When we set this way, expiring_url generates the url by s3 signature version 4.

paperclip_object.expiring_url(5.minutes)
=>https://xxx.s3-ap-northeast-1.amazonaws.com/xxx.pdf?X-Amz-Algorithm=…&X-Amz-Credential=…&X-Amz-Date=…&X-Amz-Expires=300&X-Amz-Signature=…&X-Amz-SignedHeaders=…

That’s good. We can no longer get the AWSAccessKeyId.

Conclusion

As I construct points of this article, I suggest following code.

if ENV[‘NO_S3_ATTACHMENT’] || Rails.env.test?
Paperclip::Attachment.default_options.merge!(
path: ‘:rails_root/tmp/paperclip/:filename’,
url: ‘:rails_root/tmp/paperclip/:filename’
)
else
AWS.config(s3_signature_version: :v4)
Paperclip::Attachment.default_options.merge!(
path: ‘/xxxx/:filename’,
storage: :s3,
bucket: <bucket_name>,
s3_host_name: <host_name>,
s3_permissions: :private,
s3_credentials: {
access_key_id: <access_key>,
secret_access_key: <secret_key>
}
)
end