There are many ways to automate data collection across a variety of subject matter. One of the best ways to do so is with search engine results scraping. Particularly for product information like a GTIN number, the Google Shopping search engine has a deep database of product information and specifications.
SerpApi is the best hassle free way to do search engine scraping on Google Shopping without having to worry about proxies, captchas or bot detection. This tutorial will be using SerpApi so I’ll include some helpful links here and below.
Google Shopping API Documentation: https://serpapi.com/shopping-results
Google Products API Documentation: https://serpapi.com/google-product-api
Search engines like Google are dynamic. They are always changing, updating and improving. One of the consequences of this operation is that scrapers need to be updated and maintained to be sure that they can always handle the most current HTML structure of the results pages.
Unfortunately for us, Google doesn’t give us, or anyone else who scrapes their results pages, a heads up about these changes and how best to plan for them. This means one of the primary ways we learn of an HTML change is when our users report missing JSON elements.
We didn’t always have a…
SerpApi is for getting data from pages like on Google’s search engine. That data is pulled from the webpage and stored into a file which the user can then parse for the specific bits of information they were looking for. Generally, you’d need to know how to code to write an implementation of SerpApi to get the data from the Google results pages. SerpApi now has a plugin for Google Sheets that allows you to do all of this without having to know how to code!
You will also need a SerpApi account and private API key to…