Creating a Minimal Sinatra App For Hosting Test Files on Heroku

At Reflektive, we integrate with several third-party data providers in order to make it convenient for our customers to use our products in tandem with other services.

It’s essential that we test each integration end-to-end. However, all the providers we integrate with require some form of authentication (basic HTTP authentication, API tokens, e.g.). Most of them do not provide sandbox accounts to facilitate testing.

While testing one such integration, we needed data files to be hosted on a server which is able to restrict access using basic HTTP authentication. This would replicate what would happen in a production environment, where we have to fetch CSV files with employee records from third-party servers.

The first thought that came to our mind was to use hosting services like Google Drive or Dropbox for the task. However, we faced a number of obscure technical issues.

First, the hosting services we tried would not easily allow us to generate direct links to the files. Instead, the links would point to web pages that’d need the user to download those files manually.

Second, even once we could make direct links work with Google Drive, we hit upon a much more substantial issue — Drive wouldn’t respond with success to HEAD requests aimed at the shareable URL. We need that functionality because we validate that a provided URL to a CSV file is valid before saving it to database by issuing a HEAD request and ensuring we get a success response.

Thus, after some discussion, we decided that the best way to host test files would be to create a minimal Sinatra (a Rack-based framework) app and host it on Heroku.

The idea was to have an app that takes files in the public folder in the app directory and make them available on a corresponding URL. For instance, a file called reflektive.csv would be available at /reflektive.csv. Moreover, the app would also support basic auth.

We implemented it, and in case you want to do so too, here are the steps you should follow:

1. Create a fresh directory and add to it a minimal Gemfile as shown here:

2. Add another file server.rb in the same directory. Note that you do not need to specify any routes to serve static files from public folder (more on this below).

3. Place a minimal rackup file config.ru

4. Run bundle install to install the gems in the Gemfile. If you do not have bundler installed, run gem install bundler.

5. Create a public directory and place any sample file in it, e.g. reflektive.csv

6. To test this locally, run the Sinatra app using the command bundle exec rackup . A server should start up and listen on the port 9292. And, your file should be accessible at localhost:9292/reflektive.csv.

The reason you did not need to specify any routes in order to serve the file is that Sinatra takes care of serving static files in the public folder automatically. By default that folder is public but you can change it if you want.

Now, all you need to do is to deploy it to Heroku using these commands (install Heroku CLI and login into it, if you haven’t already):

$ git init
$ git add .
$ git commit -m 'sinatra app'
$ heroku create
$ git push heroku master

Once, the deployment is over, you should be able to access your file at

https://your-app-name.herokuapp.com/reflektive.csv

Awesome! You can keep adding as many files as you want, and they’ll be served. Note that due to how Heroku works, these files need to be committed into the git repository and pushed to Heroku, just like you did above with reflektive.csv.

Now, if you wish, you can also add basic auth to your app, like we did. Just follow these steps:

1. Create a class that can be relied upon to enable basic auth when appropriate (that is when both username and password are set).

2. Require it in server.rb and call the appropriate method:

Now, just commit these changes to git, and push to remote Heroku repository. Then, set the basic auth credentials using the Heroku CLI.

$ heroku config:set BASIC_AUTH_USERNAME=james_madison BASIC_AUTH_PASSWORD=password

Once done, try visiting https://your-app-name.herokuapp.com/reflektive.csv in a web browser — you should be prompted for credentials (as you can see below) and once you enter them correctly you should get the file!

Basic Auth Prompt

Thanks for reading! If you have any suggestions or anything else you want to share with us, feel free to reach out by writing a response below.

P.S.: While building this app, I discovered a bug in the way Sinatra implemented a feature which allows a user to easily specify which request parameters are mandatory for a given endpoint. This was a good opportunity for me to dabble in open source, so I went ahead and fixed it.