Polymer 2 and Googlebot
If you write client-side web apps, you need to be concerned not only with how your app presents to your users, but with how it looks to Googlebot. Thankfully, Polymer 2 works with Googlebot, and this short guide covers some best practices to get your Polymer 2 app rendering on Googlebot.
Googlebot is based on Chrome 41 (see “Rendering on Google Search”) which was released in early 2015. Support for ES6 classes was not added till Chrome 42, arrow functions in Chrome 45, Shadow DOM v1 and Custom Elements v1 not till Chrome 53 & 54 respectively. Here are a few tips to ensuring your application renders correctly.
Do use the polyfill loader
It is recommended that you use webcomponents-loader.js which dynamically loads the right polyfills based on feature detection. Using a naive browser detection that simply checks for Chrome can lead to issues as this is based on an older version of Chrome. Notably, we need the polyfills to run in a browser that supports HTML imports but not class based custom elements v1.
Don’t use an async import
The polyfills must be loaded first before any HTML imports. In many browsers that require the polyfills this is trivial since the HTML import won’t load without the polyfill. However, Chrome 41 loads imports, so we need to guarantee that the polyfills are loaded first or our app may not load correctly. We cannot use an async import as Chrome 41 does not provide a timing guarantee of when the imported document is executed. HTML imports can be placed at the end of the main document to allow content to be painted without being blocked and allow polyfills to be synchronously loaded first. Here is how that ordering looks:
In latest Chrome this incurs a small performance cost as the import could begin downloading earlier. Fortunately the loader itself is small and the performance cost should be minimal. To mitigate this cost entirely, you can utilise server push to start downloading the import earlier. For a server that supports HTTP/2 Push out of the box, see PRPL server.
Do transpile to ES5
Many ES6 features such as classes and arrow functions were not added till later. Here is a full list of what is supported in Chrome 41 and a list of features not supported. Note that Google’s web rendering service (WRS) is based on Chrome 41 but has some differences such as the disabling of service workers. See this page on all available information related to WRS.
Transpiling to ES5 is the easiest way to ensure Chrome 41 is supported. You can achieve this by using Polymer Build which is also available via the Polymer CLI. You will need to specify the compile flag or use the es5-bundled preset.
Differential serving allows you to continue to serve ES6 code to users with modern browsers while still supporting Chrome 41. PRPL server supports differential serving out of the box. The Polymer Shop template app uses PRPL server configured with presets to achieve differential serving.
Don’t forget to test
There are a few official tools by Google to test your application with Googlebot:
- Fetch as Google is a tool that enables domain owners to use “Fetch & Render” to determine how Google sees your page.
- Mobile-Friendly Test is similar but doesn’t require domain verification to run. To test with locally hosted or firewalled pages, you can use ngrok to expose a tunnel to your page.
- Refer to the documentation for all the available tools, including working with structured data or AMP pages.
In cases where rendering is broken, using Chromium 41 will allow you to test locally and debug using DevTools before returning to testing on the official tools described above. Download Chromium 41 with these links:
Please note that these are not official builds of Chrome. If you encounter a sandbox error, you can choose to disable the sandbox by passing the --no-sandbox flag. To avoid corrupting your existing Chrome profile, it is recommended you use a fresh profile by specifying a new, temporary profile directory --user-data-dir=/tmp.
Dealing with other bots? Check out Rendertron. Feedback? Let me know on Twitter @samdotli. Looking for an example? Polymer Shop uses PRPL server to differentially serve ES5.
