Uploading a file with react frontend and rails api backend.

Mendy Wolosow
Sep 2, 2018 · 3 min read

So I recently graduated from Flatiron School and was working on an app where i had to temporarily upload a Tsv file to parse.

A tsv file is very similar to a Csv file but the separator is not commas but rather tabs (Tab Separated Values). You can still use the ruby csv library with just defining what the separator method is. In this case you can define the column separator as follows:

col_sep: "\t"

Uploading the file

To temporarily upload a file to parse you do not need to use paperclip or any special gem to help upload and store the file as it is only temporarily being uploaded to parse, after that you do not need the file at all.

On you front end you need to make an input with the type file. I am using a controlled form so it looks like this:

<form onSubmit={this.handleUpload}><input type="file"name='file'onChange={this.handleChange}value={this.state.file}/><button type="submit">Submit</button>

And here my on submit function.

handleUpload = (event) => {event.preventDefault()const formData = new FormData();const upload_file = this.state.file;formData.append('file', upload_file);Adapter.uploadFile(this.state.type, formData).then(r => r.json() )
.then(console.log)
}

You need to attach the file to a new formData.

On you rails controller you now have access to the file and you can get the path which you can then pass to the csv parse method.

file_path = params[:file].path
parser_method(file_path)

To help with memory on a large file i use the csv.foreach instead of loading the entire file into memory.

def parser_method(file_path)
CSV.foreach(file_path, encoding: "iso-8859-1:UTF-8", col_sep: "\t", quote_char: "\x00", headers: true) do |row|
// Do stuff here with the data
end
end

So all of this worked perfectly until i uploaded my app to heroku.
Heroku by default times out if no response is sent back within a short period of time.

When parsing a large file directly from your controller and it will take a while, the process will still run on your backend but the frontent will receive an error code.

The proper way to do this would be through a background worker.

I found this great article here which explains really well how to set that up.

The issue with passing this work to the background worker is that the background worker which is running on a separate dyno does not have access to the file anymore. So you would need to upload and store the file and then pass the uploaded file to the background worker, and have that do the parsing.

After spending a few hours getting all this set up i realized the file issue.

Since the app i was working on was for in house use only and the file was still uploading we reverted back to directly parsing in the controller and although it still times out we manually keep an eye on the backend process through heroku logs — tail and then we know exactly when it is finished.

This isnt the ideal situation but for now this works for us.

My main take aways that were difficult for me to figure out was that to upload a file for temporary parsing you do NOT NEED to store it, just passing the file path is enough.
As well as sidekiq and redis for background jobs were not difficult to set up at all but they have the limitation here that the separate dyno does not have access to the file anymore.

Here is another great article going which goes in depth on this topic explaining the advantages and disadvantages on the methods mentioned above.

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade