Django Unchained: How Quorum converted its Django site into a Single Page Application

A few months ago, Quorum launched version 2.0, a single page application built using React, Redux, and several other cutting edge front end technologies to display data served from a REST API. If you were using Quorum at the time, you might have noticed a slight performance increase, which is to say that time spent between pages decreased from the order of seconds to the order of milliseconds (an improvement by several orders of magnitude).

Before:

After:

Notably absent from the update is any trace of Django.

As a data-driven software startup, Quorum relies heavily on its Django-Postgres backend to do all manner of web scraping, machine learning, statistical calculation, and database management using the power of Python and the elegance of the ORM.

Unfortunately, while Django is exceptional for backend development its front end capabilities can be somewhat lacking; its template rendering system is not equipped to provide the kind of experience that users have come to expect from web applications in 2017.

From its launch until the end of 2016, the Quorum platform existed essentially as a series of links strung together. Django mapped each url a user visited to a Python function that would render HTML using any necessary data from the database. The result would be sent to the client to replace the content of their browser.

This is all well and good if, for example, you are a newspaper (the use case Django was originally designed for) and serve unchanging content to every user at every url. Put a simple caching system on top of a Django server to store the results of rendering articles and your site will be so fast the user won’t even notice that the entire page is being reloaded each time they click on anything.

This won’t quite work for Quorum, which offers live-updating statistics computed across the US Congress and all 50 state legislatures, streams of documents pulled from thousands of sources, and a rich interface for users to do everything they could possibly want to with that data.

Between waiting on expensive queries and intentionally avoiding caching to guarantee the most up-to-date results, Quorum was spending a lot of time displaying a spinner.

Well REST-ed

This project began with the the completion of our REST API, a feature made easy by open source projects like Django Rest Framework and Tastypie. The API provided an alternative to the canonical Django methods of sending data to the client.

We began improving core features to fetch data in this way, taking advantage of React to do on the client what Django templates had previously done on the server. Initially, we did this by continuing to render each page as a Django template, but one that contained a single element and a script to fill that element with the React component for that page.

The Search page was successfully rebuilt using this method. What had been thousands of lines of Python, HTML, and jQuery became hundreds of lines of JSX. However, the spinner was still getting too much of the limelight; moving from viewing a list of bills in Search to reading the text of a particular bill and then returning to Search involved unloading and reloading the page at each step, the inconvenience of which overwhelmed the improvements to Search.

The problem was that we were still relying on HTML links to transition between pages. Even if the pages themselves were built with React it was the time between them that was being wasted. The solution to this seemed to be to move our url routing system to the client using React Router. That would require rewriting every feature to be a React Component, as opposed to just drawing them inside a Django template.

As a startup, it is important to maximize the amount of improvement to the product per development hour. To make every feature its own React Component would be to scrap everything and start over, something we couldn’t justify, particularly when it wouldn’t change much from a client perspective beyond speed. For example, a feature such as Explore could be elegantly engineered to procure data from REST and use React to maintain a central visualization. It, like much of the platform, was working fine in its present build, and was mature enough that rebuilding it would add minimal value to the user.

If it ain’t broke…

The competing pressure to ship a single page application as soon as possible and the infeasibility of rewriting the entire platform presented a paradox to the front end team.

What we needed was the ability to take the “legacy” pages that relied on Django templates and jQuery and anchor them, unchanged, within a React Router that could seamlessly transition between them and our new features.

The solution we came up with was a somewhat atypical React component that would, upon being rendered, tell Django which of the old pages it needed, and then inject that HTML into itself.

We could then set up React Router to render this component whenever it encountered a url for one of the legacy features. This freed us to write all of our new features using React and transition between them using router links.

This left one problem: The legacy pages were still rendering HTML links to get to other parts of the site, which would trigger reloads any time they were clicked. If you were reading a 300 page bill and had to click away, you would lose your place and have to reload 300 pages upon returning.

While rewriting all of the features was not realistic, rewriting all of the links was. We simply had to replace <a href="quorum.us/..." /> with <a onClick="someFunction" />.

Here someFunction would become the hook into a system we built on top of React Router. Taking inspiration from iOS nomenclature to describe the stack underlying user navigation, we called our function “segue”, and used it to push and pop pages from a stack that we stored in Redux. We composed a <SegueLink/> from this function and react router to abstract out the management of this internal stack that could intelligently handle legacy pages alongside newfangled ones.

This was the finishing touch that allowed us, in December of 2016, to deploy Quorum 2.0.

Closing Thoughts

Fundamentally, this is a stopgap solution; we are continually swapping out the remaining legacy features with their new and improved iterations. The important thing was getting to skip the ground-up rewrite.

Alongside getting to ship the single greatest performance increase the platform has ever seen, we’ve seen lots of other benefits from the transition. We’ve been able to:

• Build and ship a native desktop application with about 30 lines of code using Electron.
• Significantly enhance the capabilities of our REST API. It’s shaping up to become a product of its own.
• Delete hundreds of thousands of lines of Python, HTML, and jQuery that were rendered obsolete by React and REST.
• Eliminate use of the Bootstrap Modal, no longer feeling compelled to superimpose functionality on top of itself just to avoid reloading the page.
• Integrate Webpack, Jest, and several other tools to dramatically improve our workflow and quality, as well as position us to take full advantage of and contribute to the open source community.

During this process we were surprised by the lack of a canonical method for creating a single page application using Django. Our hope in publishing this is that someone may find inspiration, or, more likely, an example of what not to do. Better still, someone may point out something we missed that can meaningfully improve our software.

In the meantime, we’ll rest easy knowing our users are spending a lot less time waiting.

Quorum is hiring! Learn more here.