Search Engine Optimization (SEO)
Search engine optimization (SEO) is the practice of increasing the quantity and quality of traffic to your website through organic search engine results.— Moz
Google, Bing, Facebook, Twitter, and other social media sites rely on web crawlers to index your application content and make that content searchable on the web.
A Web Crawler is an automated script used to browse the World Wide Web.
How crawling of major Search Engines like Google work?
Let’s see how traditional HTML is rendered by a Search Engine as compared to Single Page Application
— Traditional HTML based website
- GoogleBot Fetches an HTML page
- Extracts links from the code fetched so that the links can be visited simultaneously.
- It then downloads CSS files
- Downloaded assets and resources are then sent to the Indexer
- And lastly the indexer indexes the page
This whole process is pretty straightforward and thus blazing fast
- GoogleBot Fetches an HTML page
- Downloads all the CSS and JS files
- GoogleBot then has to use Google’s WRS (Web Rendering Services) to parse, compile and execute JS code.
- Then WRS fetches the data from external APIs, from the database, etc.
- Finally, the indexer can index the content.
- Now Google can discover new links and add it to the GoogleBot's crawling queue.
The whole process is much more complicated than HTML crawling.
Since google is the leading Search Engine, we’ll discuss issues in context to Google mostly.
Sometimes things don’t go perfectly during rendering, which may negatively impact search results for your site. — Google
Initially in Angular2 Google used let keyword instead of var which was considered to be erroneous for the version of Chrome that Google uses for rendering i.e. Chrome 41.
So if Google wasn’t able to index their own site just because of a tiny mistake, imagine having those errors in your site and what Google could do to your website’s ranking.
3. Three(3) years old browser
As mentioned above Chrome 41 is being used by Google for indexing and rendering your website. There’s only partial support for ES6 features and some of the keywords might not work like let, const & class etc.
5. Caution! GoogleBot is not a real browser
GoogleBot doesn’t just goes on crawling your content and downloading everything that comes in its way. GoogleBot has to be smart and make decisions on whether to fetch a particular resource or not and whether it’s relevant or not.
A huge JS resource or some JS code that takes too long to execute might be ignored by the Bot completely, which might effect your sites’ ranking if that script contains some critical content of your site.
Changing canonical tags via JS is considered not reliable by Google. So make sure you have your canonical URLs in the HTML and not JS. Chances are Google may have addressed this issue but it’s just a chance, and when it comes to SEO one should not risk it unless dead sure.
7. Hashes in URL
- Bad: google.com/#/search/
- Good: google.com/search/
8. Slow Websites
If downloading, parsing and rendering takes longer than 5 seconds due to a slow API or scripts, chances are google will either defer it or ignore the page completely and move on.
— Bing & Other Search Engines
For a moment let’s consider Google is able to index your site perfectly what about the rest of Search Engines. Bing alone shares 33% of the US internet users searches as compared to Google.
So make sure you test your website for other Search Engines as well, to have better reach.
Server Side Rendering vs Client Side Rendering
Server Side Rendering: with SSR or Server Side Rendering the content is already there when the browser receives it. It means it’s not just easier for the bot to crawl but is extremely fast as it’s classical HTML approach.
Isomorphic Rendering vs Prerendering
Isomorphic Rendering or Server Side Rendering is another approach in which the user and bot both have access to completely rendered content when they visit the site. All the JS features are available which makes it rich for the users and have all things a bot needs to have to index the page. It’s the most recommended option.
Thanks for reading! 😎