Handling Complex Data in Static Site Generators

When presented with a website problem involving complex data, most of the recent time developers have reached for CMS solutions. It seems far easier, even if our editing needs are infrequent, to build data structures and forms in an admin to solve complex data entry. With the robustness of newer static site generators, complex data needs should probably no longer be in the equation when deciding to choose a CMS.

The basic fact is that far fewer sites need to be using a CMS. When you’re building a page that doesn’t have actual complex publishing needs, installing a huge piece of software that’s constantly running is a security and speed hit you don’t really need or want. That not only goes for programs like Drupal and WordPress but also stuff like Ghost.

I will also give you some tooling examples to help you along the decision process with what you might want to use to build this functionality. This is not a tool sermon, however, and I will try to mention other options and pathways you can take.

Dealing with Data

The first thing you need to decide is if your data is simple enough to be edited by hand using something like YAML or TOML. If it is, then use that. An example of data simple enough to be managed by YAML might be a résumé or CV. If you instead have many things that need a single model, you’ll want to build an admin and instead put that data in JSON format.

The easiest way to do this is probably to build a simple Express app to drive a form. You can then write these forms to JSON using lowdb. lowdb is a simple interface for writing and manipulating JSON files using lodash. If you’re writing to flat files that will be stored in your repo, you have no need for anything like authentication or data security as someone could simply overwrite the files manually if they had access to your repo. Simply make a form and write your data using one of the hundreds of UI frameworks out there.

The following is an example Express router that I’m currently using on the company site for Fusion Marketing to write employees to JSON. It is presented without context, but it’s fairly obvious what it’s doing just by reading the code. There are only three routes: /employee/all which displays all employees, /employee/new which creates a new employee with a random unique ID, /employee/ID which controls display, editing, and deletion of existing employees. There are only two templates: edit and showAll. The templates are not really worth showing as edit is simply a form with the fields.

You’ll notice there’s no multipart handling for photos. In this case, since the content is managed by an advanced user, I’m using human power for this by only asking for the filename and for the images to be stored in a specific directory. You can manage the data by using HTML5 input types and patterns, but one could also use a form sanitization plugin for a more robust user interface.

Now all you have to do to create or edit data is to run your little admin app. Save your data sets in your repo. Moving beyond this in terms of functionality is pretty obvious; get it online and instead of lowdb, you’ll be using something like MongoDB or RethinkDB. It requires quite a bit of extra work — setting up authentication, preventing simultaneous edits, and writing a routine to dump the database to JSON — so don’t bother if you don’t need this.

Setting Up Your Generator

Now that you have a way to edit this data, you can pick a site generator. I personally use Hugo because it’s extremely fast, full-featured, has native support for data structures, only transforms content, and is easy to install on a server. I don’t need my site generator to process other things like JavaScript or Sass because I want to control that myself. In fact, with Gulp or Webpack, you don’t really even need to use a different site generator and could send your JSON file to your template using Object.keys() instead. An easy-to-use site generator is still probably a better solution for most.

Next, look up the documentation for your site generator (check out Hugo’s instructions here, Jekyll’s are over here, yours probably has the same functionality or a plugin) for instructions on how to display JSON in the template system. Now you can consume all the data you created earlier! It’s easy! Now you no longer need a CMS simply to display modelled data.