ActiveStorage to Cloud-based Storage
Here Today, Gone Tomorrow
Ever launched an app in Heroku and realized when you opened it the next day it wasn’t working and your data is gone? What the hell?!
The problem is that Heroku’s drive is ‘ephemeral’ — meaning you can write files to it, but they will not be persisted when the app is re-started. If you’re using Ruby on Rails’ Active Storage, the information will be accessible through local storage temporarily but will disappear after 24 hours.
So, what now?
Enter Cloud-based Solutions
Instead of storing uploads to Heroku’s drive, you can utilize cloud-based storage to keep your data persisted. There are a lot of services out there that you could use, but Heroku’s documentation suggests using Amazon’s S3.
S3 is built for storing objects with a focus on security and scalability. It’s also free to use with some limitations. Storing data for a small application will likely not cost a dime.
The way that it works is by uploading your data in a ‘bucket’. You can have multiple buckets if you need to keep some data separated. Once you have that setup, you’ll need to make some small changes to your application to hook it up.
Before we get started, you’ll need to make sure you have already successfully launched your application on Heroku and verified that your data is available locally. If you haven’t done this, check out Heroku’s easy-to-follow guide here.
Pro-tip: Defining a dynamic API variable that allows you to switch from production and development URLs can save you time and possible confusion when testing your local storage vs. S3 storage. For example, if you’re using a React front-end you can create a file labeled API.js and add the following:
const PROD_URL = 'https://your-app.herokuapp.com'const DEV_URL = 'https://localhost:3000'const API = process.env.NODE_ENV === 'development' ? DEV_URL : PROD_URL;export default API;
Getting Started on AWS
The first step in migrating your data to AWS is to create an account. It’s pretty self-explanatory. Navigate to AWS S3 where you’ll create your first bucket.
After you click on the orange ‘Create Bucket’ button, enter a name for your bucket that is fitting for your application and/or the data you plan on storing there. Pay close attention to the AWS Region you have chosen. You’ll need it later along with your secret access key and secret access id. You can leave the other settings alone for now.
Link AWS S3 Data to Application
Congratulations! You’re ready to link up this cloud-based storage to your app. There are a few files we’ll need to change to get this up and running properly.
To keep your data secure, you’ll need to set up a hidden variable containing your AWS secrets. In case you didn’t save the secret key and id from when you created your bucket, you can create a new access key to use by logging into your AWS account, clicking on your username on the top right, and selecting My Security Credentials.
Once you have that, visit your config/storage.yml file. You’ll find a hint here for your next step.
#Use rails credentials:edit to set the AWS secrets (as aws:access_key_id|secret_access_key)
Follow those instructions first, then include this code below it:
amazon:service: S3access_key_id: <%= Rails.application.credentials.dig(:aws, :access_key_id) %>secret_access_key: <%= Rails.application.credentials.dig(:aws, :secret_access_key) %>region: enter-region-herebucket: 'enter-bucket-name-here'
Now make sure config/environments/production.rb includes:
config.active_storage.service = :amazon
In your gemfile you’ll need to add:
gem 'aws-sdk-s3', require: false
Don’t forget to bundle install after!
If you’re running into CORS errors, it could be because you are sending unsecured files or you don’t have read/write permissions for your bucket set up properly. You can read more about this issue and how to resolve it here.
Another important thing to note is that you may not be able to use Rails’ :includes method. If this gives you problems, simply use a serializer instead. This blog is a great resource for setting up serializers, in case you haven’t tried it. It’s a game-changer.
Final Tip: Be Patient
There are a lot of variables at work from AWS to the back-end to the front-end to Heroku. Take your time here. Test your input and output with debugging tools each step of the way. Making assumptions or too many changes at once can lead to bugs that are hard to track down later. Enjoy the process.