How To Parse Google Local Pack Results

hartator
hartator
Apr 13, 2018 · 3 min read

Parsing Google local pack results — the local results you see being extracted of Google Maps on regular search — is not easy task. However, these local results are more and more numerous.

Here’s a sample of the top result of the request coffee in Austin, TX:

Top results for the search “coffee” inside the regular search

In this tutorial, I will walkthrough the way we build this kind of parser at SerpApi, a Google search engine results API.

If you want to run the code yourself, you’ll need:

Ruby (>= 2.1)
Rails (>= 5.1) # Not necessarily required, but some of its methods are convenient
Nokogiri gem

First, we’ll need to define what’s it’s important to extract. Also, it matters to give to each of these element a descriptive name to make references easier in the future.

Here’s a mockup of what our parser needs to extract:

Second, we need to identify a global selector for this full block. Chrome inspector tool comes very handy:

The CSS class on the div xERobd seems to be good selector for the full local pack. Here’s the Ruby resulting code we’ll use:

def get_local_results
# Parse HTML using Nokogori
doc = Nokogiri::HTML(html)
# Make sure we've a local results block
local_results_top = doc.at_css('.xERobd')
return unless local_results_top
end

Let’s extract now our first data. The Google maps image, and other local map information. We find CSS class selectors the same way:

Interestingly, we notice some GPS looking data inside the Google map link. Structured this way: rllag=30265614,-97744454,182 This look like regular GPS coordinates. Longitude, latitude, and altitude. After double checking that’s this is indeed GPS coordinates that match our request via third party service like gps-coordinates.net, we decide to incorporate this data in our parser. We need a regex to select the part of the Google map link we want, some text cleaning utility to revert back to a more regular GPS coordinates format, and structure it for our JSON output. The final code looks like this:

def extract_gps_coordinates google_maps_link
coords_raw = google_maps_link.scan(/rllag=([0-9\-,]+)&/)[0][0].split(',')
coords = {}
coords[:latitude] = coords_raw[0].insert(-7, '.').to_f
coords[:longitude] = coords_raw[1].insert(-7, '.').to_f
coords[:altitude] = coords_raw[2].to_i
coords
end

We are using the same techniques to find CSS selectors for the title of the place, the type of business (Coffeeshop, restaurant, Supermarket, etc.), its pricing, its hours, its snippet (the small description just after the title), its rating, its reviews, its address, its thumbnail, and the place extensions (list of keywords describing the features of the place.)

The final code iterating through the place results looks like this:

local_results = []local_results_top.css('.uMdZh').each_with_index do |local_result, index|

local_result_hash = Hash.new{ |h,k| h[k] = Hash.new(&h.default_proc) }

local_result_hash[:position] = index + 1
local_result_hash[:title] = local_result.at_css('._rl').text
local_result_hash[:address] = local_result.css('.rllt__details > div')[2].text
local_result_hash[:hours] = local_result.at_css('.rllt__details > .rllt__wrapped').text
local_keywords = []
local_result.css('.UGaK9c').each do |keyword|
local_keywords << keyword.text.gsub('·','').squish
end
unless local_keywords.empty?
local_result_hash[:extensions] = local_keywords
end
if img = local_result.at_css('img')['src'] rescue nil
local_result_hash[:thumbnail] = img
end
local_results << local_result_hash
end

This code needs to be updated every 2–3 weeks as Google keeps changing the way results are displayed. You can keep up the changes by going through the process described here, or you can use our service, SerpApi, that abstracts all of this and more for you.

hartator

Written by

hartator

Passion for beautiful code, lunatic enterprises and shattered dreams

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade