Using Google Search Console with Python and a service account

Photo by AltumCode on Unsplash

The Google Search console can provide you many useful informations about your website ranking position. If you are working on a website, you would want to know where are you appearing to the users.

This tutorial suppose you have some basics understanding of the Google Search Console, and know the specific terms around it.

Nevertheless, let’s get started !

1. Create a Service Account with Google Cloud Platform

You may not have yet created a service account and even don’t know what it means. I know it’s in the title of the article and some of the readers will find it pretty obvious, but I want this to cover the subject as much as possible.
You could find a tutorial I made on another article on how to create a service account on the Google Cloud Platform and create a JSON key associated to it on the link just below.

Create a Google Cloud Platform service account in 3 steps

2. Enable Google Search API on your Google Cloud Platform Project

Now that you have a Google Cloud project, and a service account created, it’s time for you to enable the Google Search API.
Indeed, it’s not done automatically, but it’s pretty straightforward.
Just click on the Enabled APIs and services on the left navigation menu as shown on Figure 1.

Figure 1: Enable APIs and services navigation button

And then click on the button at the top named Enable APIs and services as shown on Figure 2.

Figure 2: Enable APIs and services

On the search bar at the center search for google search console api and you will likely have only one result, click on this one and then the Enable button when you arrive on its page.

3. Install and import needed Librairies

You’ll need only two librairies here
pandas
google-api-python-client

As imports just type the following

import pandas as pd
from google.oauth2 import service_account
from googleapiclient.discovery import build

4. Create the connection to the Search console API

Just change of course the service_account_json_key for it to match with your JSON key local path.

5. Accessing Data from the Google Search console

5.1 The payload

First of all, you need to grant access to your service account to your domain on the search console. To do so, just retrieve your service account email which you can find for example on Figure 5 of my previous article on how to create a service account. You could also find it opening your service account JSON key and finding the value associated to the client_email key.
Then, you need to open your google search console, and then go to Settings > Users and permissions.

Google Search Console

Now grant your service account email the required permissions. You can find all of them on this documentation but Full user access should be what you need for this job.

This done, you can start coding.

Just above is the payload of our request. It means it contains all the parameters of it, what we want to query and what filters we want to apply. I tried to include some of the main ones, so let’s try to decompose them.

'dimensions': ["page"]

This means the dimension of the results will be the “pages”, and so you’ll have results for each page. You could have entered also country, device, query, and searchAppearance. More informations about it can be found here.

        "dimensionFilterGroups": [
            {
                "groupType": "and",
                "filters": [
                    {
                        "dimension": "query",
                        "operator": "excludingRegex",
                        "expression": "search term 1|search term 2|search term3"
                    },
                    {
                        "dimension": "page",
                        "operator": "excludingRegex",
                        "expression": "^http://www.mywebsite.com/$|^https://www.mywebsite.com/home/$|"
                    }
                ]
            }
        ]

This is the filter of your query. Here you specify that the groupType is an and clause meaning that the results must match all conditions. If you don’t specify the and clause, the result need to match at least one of the conditions. More informations could be found here.

And then comes the filters themselves where you have to specify the dimension (same options as we previously seen), operators (e.g., contains, equals, excludingRegex, etc… more informations here), and the expression of your filter.

        'rowLimit': 25000,
        'startRow': 0

Finally, you have the rowLimit which indicates the maximum number of rows returned by the query, and startRow, the number associated to the first row. More informations here.

5.2 Query data from Google Search Console

Now here you enter the site url, and the payload that we defined in part 5.1.

for row in response['rows']:
  data = {}

  for i in range(len(payload['dimensions'])):
    data[payload['dimensions'][i]] = row['keys'][i]
  data['clicks'] = row['clicks']
  data['impressions'] = row['impressions']
  data['ctr'] = round(row['ctr'] * 100, 2)
  data['position'] = round(row['position'], 2)
  results.append(data)

dataframe_report = pd.DataFrame.from_dict(results)

This part is only to structure everything into a pandas Dataframe. I retrieved only informations of clicks, impressions, ctr, and position, but I advise you to explore the json in order to check what is realy important to you.

Conclusion

Hope you enjoyed this tutorial, and will enjoy doing some preprocessing with Python directly into your Google Search Console.
The Google Workspace suite allows you to do a lot of things with their APIs, and you can use most of them in Python, so do not hesitate to explore them and find new automation ideas !
And if you want more tutorials, check out my other ones !

Using Spreadsheet with Python and a service account