When HTML Can Be Better Than JSON 🔊

Start looking at the Web as a network of computers, not a network of browsers

Fagner Brack
Sep 13, 2018 · 11 min read
The picture of a sleeping lion; his head is RESTing over a stone.
Listen to the audio version!

  • Play around with Javascript to build some interactivity.
  • Later, grab a framework and return some data from a server in a custom JSON structure.

HTML-like content types can provide better foundations to build APIs for the Web

Let's start at the beginning.

There’s a server somewhere, even if you don’t see it.

A long time ago, there was this Search Engine called Google which had a compelling idea to rank the relevance of websites. They did so by writing code that would read the HTML and look for hyperlinks (<a>). Once they indexed how many websites were linking to each other, they could quickly check which ones were more relevant than the others. This way, when the user looked up for a search term, the most relevant websites would stay on top of the search results.

The code example that shows how to parse an HTML file using JavaScript to look for hyperlinks. There’s a function with the name “index for links” that accepts an HTML string. The function parses the HTML and queries all the anchor tags. After that, it uses the “map” and “reduce” functions to return a data structure containing links and the number of occurrences of those links in the page.

Given how HTML defines anchor tags, you can write code to crawl any website and follow links, as long as the author of the website can produce a meaningful message that follows the HTML specification of how to create anchors.

A Web Crawler is not the only type of client that doesn’t care about how the browser renders the HTML. The code for a headless test automation script, such as Selenium or Chrome Headless, only cares about how to identify specific elements of the page. The automation code doesn't care if the underlying test engine has to simulate full browser rendering or not.

The code example that shows a test script submitting a feedback form. The script looks up for the classes “feedback form” to submit the form and “feedback content” to input the feedback as text.
The code example that shows a test script submitting the feedback form in a page where the form is in the navigation bar. The script, represented by the function "send feedback," remains unchanged but still works, even if the browser renders the form in a completely different way. Look at the differences of the HTML from this example to the previous one.

Do not create HTML attributes that are test-specific

Here’s an example of another type of client besides a test automation script:

The code example that shows how the script for the designer's bookmark would be. It has the function “simulate bookmark highlight.” The function queries all the elements with the class “company privacy always anonymous,” iterates over them and adds a solid red border with the size of 3 pixels.

You can enhance the website’s HTML to provide attributes that any script can use, including third-party consumers.

These principles are not just for HTML. You can do the same thing with raw JSON.

  • You design endpoints which can return custom JSON to represent your website’s data and write a test script for it.
The code example that shows a response in JSON containing the data for the latitude of Sydney. The code retrieves the latitude and outputs in an alert using a function called "get latitude from."
The same code example as before, only now the server renamed the property “locations” to “regions.” The code for the function "get latitude from" breaks and the alert doesn’t work. The error is "Cannot read property ‘Sydney’ of undefined." To see it, open the snippet in the JSFiddle website, then observe the Browser Console.
A JavaScript code example that shows a response in HTML containing the data for the latitude of Sydney. The code retrieves the latitude using the browser’s “query selector all” function and outputs in an alert.
The same code example as before, only now the server renamed the unordered list to a description list. Also, the server renamed the header from “Locations” to “Regions.” None of those changes break the code for the function "get latitude from." The alert always works, as long as the element with the class "sydney-latitude" has the correct value.
The same code example as before, only now there's a new latitude in the "description list" for the "Sydney" region. The alert outputs both latitudes separated by a comma.

If the server returns raw JSON as a response, clients have to write code that is tightly coupled to the structure. Any significant change to the structure break the clients.

If you want to use JSON and create a robust integration between two computers using HTTP for many different purposes, you need to use a standard hypermedia specification that can tell you where servers should put identifiers and where clients should look for them. You need to choose a hypermedia specification like Jasonette, Mason, Collection+JSON, HAL or JSON API.

A Tweet from Roy T. Fielding, the author of the REST architectural style. It says: "The reason to make a real REST API is to get evolvability. A “v1” is a middle finger to your API customers, indicating RPC/HTTP (not REST)"

HTML is an API to the browser. What HTML has is a specification that can give you for free a lot of the benefits an API built with JSON without a hypermedia specification doesn’t have.

HTML is not just some magical thing to render a website. The server writes on that language to create an API the browser can consume and provide value to visual clients — the humans. It can also be enriched by the server to provide metadata for non-visual clients — the machines.

server -> HTML ->          browser           -> human brain
server -> HTML metadata -> Some HTML parser -> machine
  • Write test automation scripts that don’t need to change if the elements of the page change.
  • Write bookmark scripts that can highlight certain elements.
  • Expose a Public API for third-party consumers to understand the website's workflow.
A tweet from Matt McClure in response to the previous tweet from Roy Fielding. It says: "And isn’t giving your customers a REST API without client development tooling to complement it also giving them the middle finger?"

Fagner Brack

Written by

I believe ideas should be open and free. This is a non-profit initiative to write about fundamentals you won’t find anywhere else. ~7 min post every few weeks.