Streaming data with Server-Sent Events

Although the last alternative shown seems to be perfect, it isn’t. It would be if the browser had to send messages to the server too, instead of just receiving. For our case, we are going to use Server-Sent Events (a.k.a. SSE).

By checking out this link, you can know more about SSE. The author has covered a lot of SSE’s features and showed some examples. I specially liked of this quote:

SSEs are sent over traditional HTTP. That means they do not require a special protocol or server implementation to get working. WebSockets on the other hand, require full-duplex connections and new WebSocket servers to handle the protocol.

Ok then, show me the code!

Rails Streaming

First, put the Puma gem into your /Gemfile and install it:

bundle install

After that, you need to allow concurrent connections in your development environment. It’s necessary because I bet you want to use the system at the same time that SSE is opening a request. Just add this to your /config/environments/development.rb:

config.allow_concurrency = true

Now, create a controller that seems like this:

class ImportsController < ApplicationController
  include ActionController::Live

  def progress
    sse = SSE.new(response.stream)

    response.headers['Content-Type'] = 'text/event-stream'

    5.times do |i|
      sse.write({count: i}.to_json)
    end
  rescue IOError
  ensure
    sse.close
  end
end

And a route for it:

get 'imports/progress', to: 'imports#progress'

Now you can run up the Puma and open your browser to see this:

Server-Sent Events

Now you only have to create a new EventSource to listening to the URL that you’ve created and define a callback function to receive the data:

var sse = new EventSource('/import/progress');

sse.addEventListener('message', function (e) {
  console.log(e.data);
});

Open the browser’s Developer Tools and go to the Network tab. Note that there’s a persistent connection that shows every message coming from the server.

Now you’ve to decide how that data will behave on your app. That’s it!

Improving with Redis

As the same done with Puma, you need to install Redis gem before that.

Let’s assume that there is a model called List which receives the file and begins to import it over a background process. The idea behind that is to publish the import progress and the controller defined above to subscribe this progress and send back to the browser. Let’s see how this model seems like:

class List < ActiveRecord::Base
  def import
    redis = Redis.new

    5.times do |i|
      redis.publish('import:progress', {count: i}.to_json)
    end
  end
end

Now we need to change our controller to subscribe this event:

class ImportsController < ApplicationController
  include ActionController::Live

  def progress
    redis = Redis.new
    sse   = SSE.new(response.stream)

    response.headers['Content-Type'] = 'text/event-stream'

    redis.subscribe('import:progress') do |on|
      on.message { |e, data| sse.write(data) }
    end
  rescue IOError
  ensure
    redis.quit
    sse.close
  end
end

I think that you’ve got the idea. Try to call this List’s method via rails console and see your app working like a boss.

Deploy in Production

There’s an important thing you need to do before getting your app in production. SSE needs that your web server to allow a persistent connection, otherwise, for every message received he has to close the connection and the browser will try to reconnect. I’m assuming that you’re using Nginx in a VPS. Just put it into your nginx.conf inside the location directive:

location @app-name {
  ...

  proxy_http_version 1.1;
  proxy_set_header Connection ‘’;
  proxy_buffering off;
  proxy_cache off;
  chunked_transfer_encoding off;
}

Now all that thing is supposed to work fine. At least, it works on my machine :P