Puppeteer: Quick fix for differences between Headless and Headful versions of a webpage
Some webpages behave differently with Chrome and with Chrome Headless. It’s not immediately obvious why that might be the case. The difference mostly has to do with the way Chrome and Chrome Headless set the two header fields — User-Agent and Accept-Language. There could be other reasons — a site could intentionally be detecting and blocking headless browsers. We’ll just keep it simple and focus on the two header fields for now.
User-Agent:
Chrome uses — Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/63.0.3205.0 Safari/537.36
Chrome Headless — Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_6) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/63.0.3205.0 Safari/537.36
Accept-Language:
Chrome uses — en-US,en;q=0.8
Chrome Headless doesn’t set this header.
If you want Chrome Headless to closely emulate Chrome, here’s the code to fix it:
