SerpApi
Published in

SerpApi

Reverse engineering Google Finance charts

Scrapping Google finance chart — SerpApi.com

In this article, we will talk about how to reverse engineer Google finance charts to parse them using Ruby on Rails.

Introduction


‌ When you search in Google for something like Bitcoin price or bitcoin vs dollar we will notice a chart and very rich finance data, originally the source of this data is Google finance.‌
‌‌
‌‌
What is Google finance?
‌‌
Google Finance is a website focusing on business news and financial information hosted by Google.

Google finance data & chart “bitcoin vs dollar” — SerpApi

Getting started


We will ignore all the other data and we will focus on parsing the chart only as the extract of the other elements has been covered by other SerpApi blog posts.‌
‌‌
Basically, every chart or graph consists of two important parts (x-axis and y-axis).‌
‌‌
The x-axis is a horizontal line and the y-axis is a vertical line.‌
‌‌

Coordinate grid y-axis and x-axis chart — SerpApi

Now we just need to understand the numbers in the Google finance chart, The y-axis represents the price column, and the x-axis represents the time.

y-axis and x-axis represents the price and time — SerpApi
y-axis and x-axis represents the price and time — SerpApi

It’s obvious now in the screenshot above, the price is 56,854.90 at 8:05.
‌‌
‌Now we will find the chart CSS class:‌

Chart CSS selector — SerpApi

In this example, we will take the attributejsdata but we should note that the input of this attribute is changing every search.‌
‌‌
‌So by using REGEX we will extract the last element inside jsdata="Wplt6c;_;AWRM64" which means the element we want is AWRM64

Using regex to extract chart data — SerpApi

1- This is the Regex that we used to search in the page source for the chart data.‌
2- Is the raw HTML page — to search inside of it.‌
3- The result, which it’s the group of the chart JSON data.

After formatting the JSON, now we need to understand what’s inside the JSON carefully.

JSON chart data — SerpApi

1- This number represents the price.‌
2- This one represents the time in minutes.‌

Putting everything together


‌Now we have to use dig method to extract the JSON data we need, which it consists of the x-axis and y-axis arrays (price and time).‌
‌‌
‌and then we will convert the time from minutes to hours, we will use this formula to get the UNIX time:

unix_time = time * 60

and the last thing to do is convert the unix_time to DateTime ( UTC Y-M-D:H:M )

data[:time] = Time.at(time * 60).utc.strftime("%Y-%m-%d %H:%M %p")


The final result:

Serpapi.com parsing google finance chart

Ending:

You can find the documentation about how to use serpapi and you can follow us on Twitter at @serp_api, to get our latest news and articles.

--

--

--

Fast, complete, and easy API to scrape and extract search results

Recommended from Medium

Five Essential Principles for Developing Lambdas

Interview: Bob Gregory — Chief Architect Cazoo (EventBridge, DDD & Microservices)

TRON SR RANKING May 16, 2018

Deploying my first CLI application through Ruby

Long White Day Candlestick Pattern — How to trade them

How To Create A Popular Mobile App Like Calm in 2020

Cloud Pub/Sub

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Alaa Abdulridha

Alaa Abdulridha

Cybersecurity Engineer and #OSWE certified

More from Medium

Automating Deploy WordPress with MySQL on Kubernetes cluster on AWS.

Lambdatest A Cloud Base Cross Browser Testing Platform

Launching a WordPress and MySQL architecture on K8s cluster on AWS cloud via Ansible

Install VSFTPD, APACHE, PHP 8.1, MARIA DB, Wordpress on DEBIAN 10 (Google Cloud)