How You Render Can Affect Your SEO (CSR vs SSR vs Dynamic Rendering)
Recently I gave a small talk on how rendering can affect SEO. I started with a couple of slides on the dynamic rendering architecture, and tried to work backwards towards first principles. I ended up more or less with a history of the Internet. Along the way, I realized that there are a lot of web fundamentals that are often glossed over. We have great tools like React and Server Side Rendering (SSR), but these offer solutions to problems that some engineers might not understand. Not everyone has been building websites since the 90s, so a lot of the problems that tools like NextJS solve aren’t well understood. Ask any interview candidate to compare CSR and SSR, and you will inevitably get the answer that SSR is better for SEO. But why? Why does SSR make a difference in terms of SEO?
This post is going to try to answer that question starting from the fundamentals of web. We’ll look at:
What’s a website? What’s a crawler?
What is rendering? What’s CSR and SSR?
How Does this impact SEO?
What have I seen used in production?
This is mostly coming from my notes, so expect more of a narrative than something polished and perfect. I’ve linked to helpful resources along the way. Try to keep thinking of things from fundamentals. If you think to yourself while reading “aha, but library xyz solves that problem!” then ask yourself how it solves it. Pulling at the threads can be a great way to learn.
What is a Website?
The simplest website I can imagine is a single, static html file.
HTML is markup — it can contain things like text, images, videos, css, Javascript, etc. You can put tags inside of other tags (like the h3 inside of the body), which makes it hierarchical. By itself, it’s pretty boring to read, so we use web browsers like Chrome or Firefox to view it.
An observation is that browsers turn HTML into pixels on a screen. It takes the HTML, parses it, builds an in-memory data structure to represent and interact with it (a tree — remember that we said HTML is nested/hierarchical?), does some math to figure out layout, then paints it on the screen.
I wrote HTML, now what?
At this point, we have HTML for our content, and a browser to display it. Humans (and robots) and now view our page on the Internet.
However, if you’ve ever published your own blog or website, this visual might be familiar to you.
Sadly, simply putting something on the Internet is like dropping a pebble in the ocean — the ripples just aren’t that noticeable. We need a way to direct traffic to our site. How do people find sites? By using search engines! How do search engines find sites? By crawling.
What is a web crawler?
Systems design interview time — we need to build something to crawl and index the Internet. This exists (Google), so it must be possible. How does one work? Perhaps like this:
- Take a really big list/queue of seed URLs (maybe 1000, 10,000, whatever)
- Pop the fist URL. Make an HTTP GET request and download the HTML for the site.
- Parse the HTML. Look at the <meta> tags in the <head>, the title, and the content. Try to create a small summary or snippet of this site. (Also somehow store and index all this content. Out of scope for now.)
- While parsing the HTML, find all <a> hyperlink tags. Look at their
hrefproperty, and extract the URL that they point to. Add it to the queue in step 1 - Go back to step 1, and loop.
That’s a gross oversimplification, but it is a primitive way of learning to crawling. You could write up a small script to do that if you wanted. Google wrote a more sophisticated crawler called GoogleBot, which will come up throughout this article.
Based on step 3, we can say that (among other things) a crawler turns HTML into search result snippets.
For now, a critical point to remember is that crawlers consume HTML. We have to provide them with HTML if we want our side crawled and indexed. This may seem obvious, but web development has changed over the years. When’s the last time you wrote an HTML file? These days, most of our development is done using Javascript libraries, like React or Angular. It’s important to understand that some additional step(s) will be required to convert that code into HTML that a crawler can read. To better appreciate this fact, let’s take a quick tour of the history of web development.
History of Web Dev: The Static 90s
In the 90s, sites were mostly static, and life was simpler for everyone. Websites usually had the following structure.
Multiple HTML files stored on a server. Keep in mind these were static — no dynamic data . A developer wrote some HTML, uploaded the file, and called it a day.
Bots and humans would see the same site. No worries about cloaking. Bots were being given HTML, so they were happy. Remember, bots like HTML.
History of Web Dev: The Dynamic 2000s
As time went on, the Internet matured, and people were using it for more things (it even had a catastrophic bubble and crash!). The rise of e-commerce and social media like MySpace meant that sites needed to work with dynamic data like usernames, product inventory, profile pictures, etc.
Static HTML is not equipped to handle this. HTML is just markup — it has no executable form, and it’s not even approaching Turing-Completeness. This lead to the rise of languages like PHP. The approach was simple.
- Store data in a database
- Use PHP files on the server (index.php) to respond to requests, pull data from the DB, embed the data in HTML, and return it to whoever requested it.
This process worked quite well, and had the advantage of returning a fully populated HTML file to the user/bot. This meant that bots were still getting HTML, and could go about parsing and snippeting and SEOing.
History of Web Dev: The Javascripting 2010s. Do everything on the client!
As time went on, Javascript became more powerful and more prevalent. Advances such as Google’s V8 Engine meant that Javascript was now more performant. ES6 gave the language a much needed update, and libraries like Angular and React were starting gain traction. JS was in the limelight.
Naturally, people started thinking about moving more of their application logic into Javascript, and executing it on the browser. For one thing, servers aren’t cheap to maintain, so why not execute all those cycles in your users’ browsers instead of on your servers?
Notice that instead of multiple HTML files, the server (or CDN) would now return a single index.html file (hence the term Single Page Application — SPA). This file was largely empty — it would contain little more than a <script> tag pointing to a JS bundle to download and execute. The JS was responsible for pulling data from external APIs, and making the appropriate calls in the browser’s DOM API to generate the HTML structure for the browser to display on screen (remember, browsers turn it into pixels). This is Client Side Rendering (CSR). This was great, except we forgot about someone.
Bots like HTML. We aren’t returning much content in our index.html. We may have broken our site’s SEO (we did).
History of Web Dev: 2016-ish onwards. Back to the server!
For many websites — especially e-commerce — SEO is of vital importance to their business. If your business sells books and someone searches for a book and all of your competitors show up on Google before you do, you’re probably going to be out of business.
This meant that as engineers, pure CSR wasn’t an option in many cases. We needed to get at least some of the content rendered server-side. But we’d also become used to using tools like React and Angular. Giving those up would cause development to regress.
As previously mentioned, Google’s V8 Engine improved the performance of Javascript. It also enabled JS to run outside of the browser through tools like NodeJS. This meant that JS could now comfortably run on the server. If we can run JS on the server, then with a few tweaks, we can also run some React on the server. This means that we could still develop using the tools we had come to like, and not completely break SEO in the process.
There’s a nice series on SSR fundamentals if you’re interested.
Success! We can stop now, right?
Not quite. SSR can help with SEO, as well as some other performance metrics, especially when combined with some other things. It’s worth noting that the entire app doesn’t need to be SSRd — a lot of times you just SSR the critical initial content for the page, and then let the browser take care of the rest/hydrate as needed. But SSR does have a couple of downsides (as of 2022):
- Developer Overhead: There are additional concepts that your dev team needs to be aware of to write React code that will render properly on the server.
- Server Maintenance: You’ll need to setup and maintain some NodeJS environments for executing server side. This will lead to some additional development, monitoring, debugging, etc.
- Cost: You’re going to be executing more cycles on your servers instead of executing everything on your client’s browser — this can come at a co$t.
- Web Components: If your application is using something like web components, these may not render properly using SSR without some additional steps. Things have been improving, but just be aware you may need some shims or polyfills server side.
Dynamic Rendering — Do Everything!
Dynamic rendering was an approach proposed by Google for using a hybrid approach to rendering (it is no longer recommended). You could develop a React app using CSR, and maintain some additional infrastructure on the backend to prerender content using tools like Rendertron. Since this was running a headless version of Chrome, the rendered content would look very similar to what CSR would produce (with the added benefit of web components not breaking). However, running headless Chrome for every incoming request would be expensive, so sites would usually filter requests based on User-Agent to detect bot traffic and only prerender those requests. Again, all for the sake of our bots being able to have some static HTML.
Studious GoogleBot
One final note — GoogleBot has evolved over the years, and can now read JS and index sites using CSR pretty well.
However, there can be a delay when indexing sites that rely heavily on JS, so keep in mind that if data on your site goes out of date quickly (such as an item up for auction), this could lead to users seeing out of date results.
Conclusion
Modern SSR tools give us some really nice solutions for maintaining SEO while still being able to develop everything using JS libraries like React. It’s important to understand how we got here, and what problems those tools solve. Thanks for reading.
