Server side rendering and dynamic rendering with Headless chrome

Headless chrome

This article is based on a lecture from Geektime Conf, Tel Aviv, Israel and jsTalks, Sofia, Bulgaria; you can find the slides here: https://slides.com/shotapapiashvili/ssr

What is Server Side Rendering

Traditionally websites were generated on the server and shipped back to the user with all the markup ready to display. There are a lot of things that depend on this kind of behaviour like google bot, social media bots and bots in general, in the last few years since we use client side javascript frameworks to build our sites this behaviour changed, server side rendering is an approach to bridge the gap between the old internet behaviour and modern techniques. Server Side Rendering (SSR) refers to the process of taking a client-side JavaScript Framework website and rendering it to static HTML and CSS on the server.

Server Side Rendering vs Client Side Rendering

To compare SSR and CSR lets talk about 3 things that the browsers does once we request a page, networking, executing JS and rendering.

Difference between SSR and CSR

In this graph you can see the process in both of the approaches, the first step is requesting the index, in CSR the index will return only minimal boilerplate of the page with a script tag that contains the app itself so now the browser needs to request the app js file and analyse it before it can print anything to the user, even after the js bundle is requested and executes the user will need to wait for XHR requests to finish before he can see some meaningful content. in the SSR approach the index request contains ALL the markup built with all meaningful content the user is looking for when visiting our site. after the browser prints the initial content then he requests the js bundle and makes the app “alive” with things like event handlers and checking for new content.

When the content is ready
The lower your First Meaningful Paint score, the faster that the page appears to display its primary content. (google)

In this graph the red lines indicate the moment when the user sees the first meaningful content. First Meaningful Paint is part of the RAIL Model

Why use SSR?

Except the FMP improvement that i mentioned earlier I think that the biggest reason to SSR is SEO.

Every year the percentage of traffic that comes from search engines in general and especially google is increasing, a lot of industries depend on the place in the top of google search. client side rendered sites can be tricky for google to handle, lets look at the process.

First, google bot or the crawler requests the page and get the “index” page markup, the crawler starts to analyse the page, this step called “first wave of indexing”, in this step the crawler will analyse the content of the page, the meta tags (title, canonical, etc.) and look for some structured data like https://schema.org/ data, as much data we will provide to the crawler at this point it will help him to index correctly the page immediately. then the crawler defers the rendering process to the rendering queue and when google will have a resource available for the rendering then the “second wave of indexing” will occur, this can be days or weeks after the first wave of indexing. According to this process we need to provide to the crawler as much of data for the first wave of indexing, if the only thing that the crawler sees is a javascript tag with no right meta tags its impossible for the crawler to understand our content and index it. and the fact that the second wave of indexing can be de delayed in weeks it can be critical to some websites. With SSR the crawler will get all the data he needs to understand the page and rank him, we will ship with the initial request all the appropriate meta tags and relevant structured data.

Another aspect of this is the chrome engine that is used to render the deferred crawler tasks, currently google uses Chrome 41 engine to do this job

https://developers.google.com/search/docs/guides/rendering

Chrome 41 was released 3 years ago, some of the fancy features of the JS frameworks wont work on this version, we don’t want our code to be determined by the version of chrome engine that google uses for this task. if we will do the job of rendering the page for the crawler we can be sure about the result of the render.

SSR Disadvantages

There are some things you'll need to thing about before deciding about SSRing.

The first thing is that you'll need a real web server, web sites that are built with client side frameworks can be shipped pretty easily with services like AWS S3 static website hosting or Firebase hosting, if you want to do SSR you will need a real web server, something like express app. (SSR can be done with firebase functions and without a real web server but in large scale functions can be expensive, here is article about it: https://medium.com/@antonybudianto/server-side-rendering-with-react-and-firebase-functions-cd67fdb2b605), the need for real web server will affect the cost of the infrastructure.

In traditional SSR we face a few more challenges later I will explain how to overcome them with Dynamic Rendering, first of all the server side must be in node.js to use the native SSR in react for example and there is no official boilerplate for SSR applications.

Headless chrome

Headless Chrome is shipping in Chrome 59. It’s a way to run the Chrome browser in a headless environment. Essentially, running Chrome without chrome! It brings all modern web platform features provided by Chromium and the Blink rendering engine to the command line.

Headless chrome is basically the exact same chrome that we all use just without the visual representation. there are currently 2 google open source projects that provide a great APIs to control the headless chrome instances. Puppeteer and Rendertron, puppeteer is more suitable for node.js apps and rendertron can be shipped with any backend technology because its shipped as independent docker container. If you have a lot of different applications you can ship a cluster of rendertron to render all of the applications because it doesn’t depend on the specific app code.

SSR with headless chrome

I will show a little example to how to SSR with puppeteer

const express = require('express');
const app = express();
const puppeteer = require('puppeteer');

app.get('*', async (req, res) => {
const browser = await puppeteer.launch({headless: true});
const page = await browser.newPage();

const local_url = 'http://localhost:8080' + req.originalUrl;
await page.goto(local_url, {
waitUntil: "networkidle0",
});

const html = await page.evaluate(() => {
return document.documentElement.innerHTML;
});

res.send(html);
});

app.listen(80, () => console.log(`Server is up`))

This is a simple express app that takes a request, lunches a headless chrome, load the “CSR” version of our page, waits to network idle (there are a lot of stoping rules like once a CSS selector appears) and returns the rendered markup. this gives us a simple interface to render our apps without handling special routing tables for the server side. this approche enables us to apply this to old apps without changing the code, we can take a 6 years old website that built in angularJs and add a layer of SSR in just few lines of code.

Dynamic rendering

In the disadvantages section I talked about how SSR can increase the cost of the infrastructure, so now lets talk about Dynamic Rendering as a cost optimisation for SSR

In Dynamic Rendering we can decide when we want to SSR and where to not. for example, if you want to do SSR for SEO reasons it makes seance to SSR only for bots.

We can check the users user agent header to determine if we want to SSR or not. here: https://stackoverflow.com/questions/20084513/detect-search-crawlers-via-javascript you can find a good enough regular expression that can help you check if its a bot or not.

Because we control when to SSR and not via the code this can give us the ability to AB test our users for SSR and check if the cost is something that pays back via longer session duration that eventually increase our revenue.

Conclusion

Do SSR definitely for bots, check what SSR does to your business KPIs and then decide if to SSR to users.