File Upload: Rails + Refile + Dropzone
Finding a good tutorial on how to add file upload to a Rails application is easy. But most of them will focus on how to integrate a specific gem like paperclip or carrierwave or refile into your application, but very rarely do you have a tutorial that will walk you through the full process of building out a feature from start to finish. So let’s change that…
Requirements
First, let’s start with the requirements. Let’s assume you are building an application which requires users to upload a document. That seems simple, right? It is, theoretically. In this theoretical app, we have a few models that we can represent: TaskCreator, Task, TaskDoer, and Doc.
TaskCreator's job is to create Tasks and assign them to a TaskDoer, and each Task will have a name which serves as the description of the Doc that is required from the TaskDoer to upload. The model architecture would look something like this:
I will be leaving out some details that are outside of the scope of this post, but in a nutshell, what you need to know is that we have a User model that can sign in, and we have 2 types of Users, a TaskCreator and a TaskDoer which are what we need to focus on. In our theoretical app, a User can only be one of the two roles, not both. Your app design choices might be different, which is why I won’t go into details about this part. Another thing to note is that in our architecture we have an attribute called number_of_items on the Task model which allows us to specify how many Docs are required to be uploaded for each Task. For the sake of this post, we will assume all Tasks require only 1 Doc but you can see how this can be flexible and allow for multiples if your app requires it.
So far so good, the requirements are clear, but there’s one component that is missing which will drive some of the technical decisions. That component is the UI. So here’s a quick sketch:
Note: Keynote (or PowerPoint) can be a handy tool for quick sketching wireframes, you don’t need a fancy tool to do that.
The hidden requirements that were extracted from these sketches is that the UI will have a list of Tasks that are assigned to a TaskDoer, once the TaskDoer clicks on the upload button, a modal will pop up asking the TaskDoer to upload a Doc either by drag/drop over the drop area (which is the cloud icon) or by using the native browse functionality. Once the upload is complete the list of Tasks is automatically updated to mark the Task as complete by switching the button text from Upload to View and color from blue to green. For simplicity, we will assume that a Task is complete if it has a Doc associated with it, but completed Tasks will be grouped together at the bottom of the list and pending ones will be grouped at the top of the list.
Here’s the interaction as a gif because you can’t have a serious post without a gif:
So, to make the obvious clear we will need a way to handle both the native browse functionality to select files to upload and the drag and drop method. While making it all happen without reloading the page. You might be expecting me here to say that I will be using something like ReactJS or AngularJS to handle the process in the browser, but as you can tell from the title of the post, I won’t. But I will note that handling the list of Tasks and updating based on each one’s status is a good candidate for something like React, but when it’s possible to avoid using another dependency that usually wins for me, especially when the new dependency has the potential of adding complexity instead of reducing it. As you will see from the rest of the post that in this case using what Rails gives us will be super simple, but then again, this is a theoretical app with its complexity reduced fictitiously so YMMV.
Tech Choices
Ruby on Rails
Rails is an obvious choice for me. It’s a powerful framework that lets me focus on the business requirements while its convention over configuration philosophy reduces the amount of repetitive work I need to do. Looking at the model architecture from above you will see that the most intricate part is the associations between the models but with Rails' has_one / has_many / belongs_to methods that are provided by ActiveRecord we won’t have much work to do on that front and we can focus on the important part which is to provide the user the best experience handling the file uploads.
Refile
There are a couple of gems that handle file uploads in Ruby and paperclip might be the most known one, but I had a great experience using carrierwave in the past on many projects and refile is the better, more modern version of that written by the same people. It does say “Ruby file uploads, take 3” on the project github page after all, which means that all of the lessons learned from the first couple of times have led to a better tool this time around.
DropzoneJS
After some quick googling to see what people are using for a drag and drop solution, I landed on the DropzoneJS website and that seemed to provide the cleanest solution of all of the ones I had seen, and a good documentation. A nice looking site also helps when you’re trying to learn a new tool, because ugly sites are hard to read and if it’s hard to read it’s hard to comprehend. So that was my choice for this piece of the puzzle.
If you have a suggestion for another library to use, share it with us.
Time to Build
Let’s get to the code, shall we?
User Stories
First, we start with the user story description:
The story above sets up the scenario we care about but is outside of the scope of this post, so I’ll let you fill in the details. The one we care about the most is this one:
and with that goes the steps implementation:
That’s a lot of code. Most of it is not specific to our case, it’s code that you would use to test any feature using capybara and cucumber. The one thing worth noting is that capybara gives us a helper method that we can use to simulate someone attaching a file attach_file. You may have noticed that we’re lacking code to test the drag and drop feature, which is impossible to do. In either case, we wouldn’t want to test the functionality of libraries that we use anyway. We are not concerned with testing Rails functionality, neither would be concerned with testing refile nor dropzonejs. What we do want to test however is that the workflow we’ve described works. In other words, do the things we expect to happen after a user’s interaction do indeed happen. And that’s what you see in our example, we test that the user is seeing the right feedback messages, an email is sent out, etc.
Of course, none of this code will pass at the moment because we have not yet implemented any of it. So we should get to that next.
DISCLAIMER: I would not normally write out the entire set of tests first before doing any code. I would build it out piece by piece and get the first test to pass and move on to the next. But as of this writing, Medium does not support pasting in individual files from a gist, instead, it spits out all of the files of a gist together, which means if I only want to show one snippet at a time I have to create a separate gist for each. That can be very tedious. So instead, I’ve created a separate gist for each file, and that’s what you see. Just keep that in mind if you ever feel that it would be daunting to try to reproduce this effort yourself.
Gems
The next step would be to install the gems, add the following to your Gemfile:
# Use Dropzone for drag and drop of document to upload.
gem 'dropzonejs-rails'# Use Refile for uploading document
gem 'refile', require: ['refile/rails', 'refile/simple_form'], github: 'manfe/refile'
gem 'refile-mini_magick'
gem 'refile-s3'
and run bundle install. Note that we are using the 'manfe/refile fork because of a Rails 5 compatibility issue currently. You might not need to do that, so check the gem’s page for the latest info.
UI
From here I like to work backwards from the UI to the backend. The reason I like to do that is because I want to work from the user’s perspective, not the developer’s perspective. It’s the same reason I write the feature descriptions first (the cucumber tests from above). User behavior is the primary driver for the design of the app. When working in this manner I tend to notice many things that I had not thought of at first glance when I started architecting the feature and it allows me to go back to the user stories and update them. These changes usually come at no cost because I still have not implemented any code, which means that the design of the code structure can change easily.
I am going to skip the part where the TaskCreator creates the Tasks, that could be a good exercise for you to do on your own. Instead, I’m going to pick up from the TaskDoer's perspective. Which means a bunch of Tasks has already been created, and the TaskDoer is looking at their dashboard with a list of Tasks and wants to upload a Doc for one of them. For that, we assume that we have a Dashboard controller that fetches all the Tasks associated with the TaskDoer and then lists them out. In other words, the index.html.erb view template has the following code:
<%= render partial: 'tasks/list', locals: { completed_tasks: @completed_tasks, pending_tasks: @pending_tasks } %>Where the tasks/_list.html.erb partial is the following:
I’ve removed any classes or styling related info from this HTML snippet and kept only the necessary stuff. We have a heading at the top that lets the user know how many completed and pending Tasks there are, and then lists each group separately since our requirement was to group the completed Tasks at the bottom. This part is also very simple, the real work starts in the upload modal we referenced on line 27.
Obviously, for the modal we are using Bootstrap but you can ignore that part. The first thing to note <%= simple_form_for @doc, remote: true, html: { class: ‘dropzone' } do |f| %>, and more specifically remote: true and class: 'dropzone'. We want the form to submit the Doc remotely so we can update the list without refreshing the page, and DropzoneJS requires us to add the dropzone class to our form.
Then comes the part below <!-- Dropzone --> which specifies the area that we want to consider the drop zone for the drag and drop and matches the template that we saw in the wireframes above. We have the cloud icon and the fake button that serves as a visual cue but in fact, if the user clicks on any part of that whole area the file browser will pop up since we specifically designated it all with the tag data-dz-clickable. Then we have the part that glues it all together which is this <%= f.input :file, as: :attachment, direct: true, label: false, input_html: { class: 'hidden' } %>. This is the part that lets refile upload the document to the server and attach it to the Doc. We want it hidden because we don’t want to show the default file input the browser displays, instead, we’re using the layout we mentioned as the visual element that lets the user know that they need to upload a file and how to do it.
We then have the preview section that we use to preview files that are images. You can leave that out or use the default template that DropzoneJS provides. In our case we just want to show the filename and the size of the file, and in case it’s an image display the thumbnail. Then we’ve set up a progress bar that will display the progress of the upload as it happens. That’s pretty much it for the template. Next, we’ll take a look as the js that makes it all work as expected.
JS
To make DropzoneJS hook into our form, we need to add some configurations using js, so let’s add a uplod_doc.coffee file to our assets and require it into our application.js manifest (I’ll let you handle that part on your own).
In our case, we’re customizing quite a bit of the DropzoneJS's behavior. We don’t want it to do anything automatically for us because we have multiple Tasks on the same page and each one of them has its own modal which then contains its own specific form. So we want to avoid the library trying to attach a dropzone to the same area multiple times because that will result in errors and weird behavior. And we want it to be active only if the modal is visible. That explains lines 2, 4, and 9.
Then we’ve specified some of the options that DropzoneJS makes available to us. Primarily we only want one Doc per upload, and we told it which input to use as the file input by specifying the paramName. We don’t want it to process the file before the user clicks on the Upload button, so we set autoProcessQueue to false.
In this snippet:
maxfilesexceeded: (file) ->
this.removeAllFiles()
this.addFile(file)we are making sure that if the user selects a new file after already selecting one, that it overrides it instead of trying to upload both ( DropzoneJS lets you select and upload multiple files at the same time, but that’s not what we want for our app).
Skipping ahead to line 47, here we specifying that once the button is clicked and there are files in the queue to be uploaded, we should disable the button and replace the text with a sinner, then process the queue. Then on line 44 we listen to the event broadcasted by DropzoneJS which returns the progress of the upload and update the progress bar.
Now back to line 21 which handles the response of the call once it’s done. First, it restores the button back to its original state (re-enables it and removes the spinner), then it hides the modal and displays the success message. Then we want to update the list of Tasks so in this case, we make another call to fetch the updated list. This where using something like AngularJS or ReactJS would have been handy to clean this up. But since I said we were going to keep it simple, the 2 $.get calls will have to do.
Lines 68 to 78 don’t do a lot in this case. They hook into the Refile event listeners and update the state of the button when the upload is triggered and complete. It’s a bit redundant since we are already handling this in the code before it, but I left it there to illustrate that it’s something available to you to use.
If you’ve gotten this far, you have a modal that pops up for the user to upload a Doc and the user can either drag and drop or select a file from the file browser. Once they do that though, they might see the preview but the upload does not work. So we should have 1 or 2 passing scenarios but that’s about it. Time to make them all pass.
Backend
We’ve gotten this far without writing too much Ruby or Rails code if you haven’t noticed, so let’s get to it already.
Based on the work we’ve already done, we know exactly which routes we need. Those were determined by the tests and by the work we did in the UI. We don’t have to guess whether we need a certain route or controller, we already know, and that’s a big benefit of working through it while being mindful of the user interaction.
Here are the routes we need:
resources :docs, only: [ :create, :show, :update ] do
collection do
get '/reload' => 'docs#reload'
end member do
get '/reload' => 'docs#reload_doc'
end
end
which tells us that we need a DocsController:
Quick note, we haven’t mentioned view a Doc before but in the controller you might have noticed the show action. This is another example of what you might be able to do even though it might not fit in our theoretical app per se.
The relevant section is in the create action. You might be wondering what’s up with the @operation, @model, and @form objects are all about. They come from using trailblazer and that’s a topic for another day. What you need to know is that the trailblazer operation takes the form params, runs the appropriate validations, and then process them according to whichever business logic you specify. In this case, it looks something like this:
Which basically takes the params passed the controller, which includes the Task id, the file, and the current_user and creates a Doc object and saves it to the database. Our Doc model is the following:
class Doc < ApplicationRecord
# Associations
belongs_to :task
belongs_to :uploader, class_name: 'TaskDoer' attachment :file,
extension: %w[pdf jpg jpeg gif png],
content_type: %w[image/jpeg image/png image/gif application/pdf]
end
and the missing piece is the template that goes with the create action:
which takes care of loading the DOM with the updated list of Tasks.
That’s it! Easy, right?
I have mostly talked about the create action but the code you see includes parts that are used for the update action, which is very similar even though I have not specifically talked about or included all the required pieces. I trust you’d be able to fill in the blanks.
I hope this can help you implement file uploads in Rails with drag and drop capabilities, and customize it to your own needs.
DISCLAIMER: This post did not tackle the issue of security. Could be a topic for another post in the future. So be sure to follow and be on the lookout for more content like this.
