React + Rails for Business Grants Portal

When we first built the Business Grants Portal (BGP), page rendering was done purely on Rails via .slim templates, with jQuery adding dynamic behavior as and when necessary. As the forms became more dynamic, we started to run into problems. Development time was spent fixing bugs instead of working on stories.

I have to apologize to our product owners. Velocity was precariously low at that time. Desperate times, desperate measures.

Dynamic form sections

The easiest way to explain our pains is to use an example. As part of the evaluation process for grant applications, a rough budget breakdown may be required from the applicant. For these form sections, we allow users to dynamically add and remove subsections.

“Jayce Kemmer” was the only vendor. After clicking “Add New Item”, a new vendor is created.

We modelled this as a one-to-many association in ActiveModel. For templating, we used Cocoon gem to help us dynamically add and remove associations. Cocoon stores the view partial for the vendor section in a `data` attribute.

Partial stored in data-association-insertion-template

When “Add New Item” is clicked, the stored HTML content is copied and appended to the DOM. Whenever the user submits the form with a full page refresh, Cocoon “magically” creates the new associations and populates the database with user data. By the way, we use Oracle DB which sometimes makes me want to (╯°□°）╯︵ ┻━┻ .

Cocoon on AJAX

This worked well but it depended on a full form submission to save the new subsections (like Google Forms). We want to enhance the user experience by using AJAX to save the page (much like Google Docs). Whenever a user saves the form, the IDs of the new subsections have to be synchronized with the DB so that for subsequent requests the sections are updated with the correct records.

Logic looks something like this. Note that this is purely for illustration. The real code would require us to manipulate the DOM.

A unique client_id needs to be generated at the front end and sent to the DB. The server response matches each DB record (server_id) to its DOM element (client_id). Updating the DOM (generated with simple form at that point of time) with the correct IDs is a lot of pain and incredibly hard to reuse across different subsections as they all had varying DOM structures.

A simpler alternative would require us to send the HTML over the network and replace the current DOM structure with the new structure.

Some JS-styled pseudocode for illustration. After receiving the response we replace the entire DOM with the new DOM from server.

We tried this approach and the performance stank. The entire page would freeze whenever we did these heavy DOM manipulations by hand. Moreover, sending DOM nodes over AJAX (seriously?) and is not exactly a “modern web standard”.

Data race conditions

As if those problems were not enough, we encountered race conditions trying to hack Cocoon + AJAX into our bidding. When clicking on the “Add New Item” button does not add a new record into the DB. The new record is persisted only when the user saves the entire form (both new records are created and old records are updated). We look at 1 example to better illustrate what we mean:

1. Action: User adds item
   Effect: Appends HTML DOM
2. Action: User saves form
   Effect: Submits form data to server (1st HTTP POST)
3. Action: User deletes item
   Effect: Sets _destroy flag and hides the DOM
4. Action: User saves form
   Effect: Submits form data to server (2nd HTTP POST)
5. Action: Server receives 2nd HTTP POST
   Effect: Server deletes record (Oops! There is no item to be deleted)
6. Action: Server receives 1st post
   Effect: Sever creates record (Oops! The item is supposed to deleted)

We can better ensure consistency by mutating the DOM only after the server has added and persisted the record in the DB. Something like:

Action: User adds item
Effect:
POST /articles
Server receives HTTP POST
Server creates record
Server replies with HTTP OK
Client receives server’s HTTP OK
Appends HTML DOM

This is just one of the many problems we faced dealing with heavy interactions. At that point of time, we realized that we had deviated so much from Cocoon’s original feature set that we might be better off using a frontend JS framework instead of hacking the DOM.

How my fellow colleague Samuel felt trying to make Cocoon fit our use case. He was so traumatized, he left BGP to work on another project.

Cool things about React

After some exploration with various frameworks, we went with React because of its performance and ease of migration. React it has been a good solution to many of the problems we have encountered in this project.

Performance

React’s virtual DOM is blazingly fast, as compared with some of the other frameworks we had evaluated at that point of time. But even the mighty virtual DOM could not keep up with government bureaucracy (government forms are among some of the most complex things I have seen in my life). In some cases, changing text in an input field can trigger hundreds of re-renders. We use React’s shouldComponentUpdate together with ImmutableJS to re-render only the components where data has mutated.

We used a common decorator to implement shouldComponentUpdate for all our components.

Better Testing

Previously when we were used Cocoon (or more specifically using a monolithic approach), the only way to test behavior (for instance: adding an item) on the front end is to use feature testing frameworks such as Capybara with Selenium. These tests were unreliable and often took a long time to run. At one point our suite of tests took up to 20 minutes to run.

We currently use enzyme, chai and mocha for our unit tests. Tests now run against the virtual DOM which is much faster and less brittle as compared to running tests in the browser. Unit tests have also shorten the feedback loop; we now know which exact component is breaking since each test only asserts on a single component.

JSX

Despite criticism of JSX by some developers, we at the BGP team love it. Our designers are able to jump into the codebase to create new components easily (albeit needing to camelcase everything and replace class with className) because it is so similar to HTML. The component based approach espoused by React also makes it easy for designers and developers to create reusable components.

Ease of Integration

We progressively integrated React into the existing Rails application. The best part about using React is that we did not need to change our existing Rails monolith into an API-based backend. Suppose we have a dashboard controller in Rails which looks something like this:

To use it in conjunction with React without changing existing controller code, we modify the dashboard/index.html.slim template.

We are interpolating Ruby variables in our inline Javascript.

The webpack_bundle_tag is a function which inserts a javascript tag referencing to a JS file emitted by Webpack, a module bundler often used in React projects (think of it as a superior asset pipeline). Do take a look at Dave Clark’s post on how to integrate Webpack with Rails.

Finally, we implement the init function which renders the component with the props {grants: grants, bgpRole: role} passed in from dashboard/index.html.slim.

The file to be compiled by Webpack. The compiled file will be referenced in “webpack_bundle_tag :dashboard”.

With this approach, we are able to use React without overhauling Rails code. This is especially useful if you want to explore React before transiting over from an existing Rails application. I highly recommend this approach over using react-rails gem because totally decoupling the frontend from Rails allows you to take full advantage of a complete JS ecosystem (node, ES6 modules, npm, webpack, css-modules etc).

I hope that this article can give you some insights on some of the engineering work we do in GDS. Do check out our QA environment and give us your feedback (the actual site requires a Corppass account).