Level Up Your Keyword Research: Retrieve Google Search Volume automatically from your Ads Account

Online Marketers and Consumer Researchers are using often the keyword ad planner tool through the frontend of Google Adwords. Constant diving into Google AdWords, manually extracting keyword ideas, battling through CSV files, and at times, losing track in the data chaos. It’s time-consuming and tiring. And yet we do it since it gives us important insights on our digital campaigns, consumer trends, and content strategy.

I have created a little software tool designed to empower your keyword research, enabling you to automatically retrieve Google search volumes directly from your Ads Account, without the manual hassle! Whether you’re crafting potent PPC campaigns, sculpting SEO strategies, or curating content, this tool might be a new one in your toolbox.

🛠️ What’s Under the Hood:

• Automated Keyword Generation: Define seed keywords and let the tool generate related keywords, automatically!
• Structured Data at Your Fingertips: Retrieve, clean, format, and store data consistently, all in one place!
• CSV first and Database second: Integrate with PostGres to avoid data loss, manage duplicates, and keep your data organized and accessible!

In this guide, I navigate you through the seamless process of setting up and utilizing this tool, ensuring that your keyword research is not just efficient, but also effectively driving your marketing strategies forward!

Note: You need to be authenticated with the Google Ads API this can be at first difficult. If you have not done it yet — here is my step-by-step guide on how to acquire fast and effortlessly your API key (click here).

Clone the Repo and Set it Up!

Open a new terminal window and clone the repo

git clone https://github.com/floriankuwala/keyword-generator.git
cd googlesearch2postgres

Now, just enable your virtual environment as followed inside the root directory.

python -m venv my_project_env

Activate your virtual environment

• for Mac

source my_project_env/bin/activate

• for Windows

.\my_project_env\Scripts\activate

And lastly, install the needed requirements

pip install -r requirements.txt

Adjust your credentials locally

It is mandatory that you have set up your Google Ads API (see here my guide). Navigate to resources/google-ads.yaml and put in your credentials.

developer_token: "your developer token"
client_id: "your client id"
client_secret: "your client secret"
refresh_token: "your refresh token"
use_proto_plus: True

Go to the generate_keyword_ideas.py in the src folder. Just insert your customer_id:

# Initialize the Google Ads client.
client = GoogleAdsClient.load_from_storage("resources/google-ads.yaml") 
customer_id = '66789990' # Replace with your own customer ID.

Optional: If you want to make use of the PostGres Connector you need also to put your credentials into the .env file in the folder “keyword_generator_env”.

DB_TYPE=postgresql
DB_USER=your username
DB_PASSWORD=your password
DB_HOST=your hostadress
DB_PORT=5432

Tailor Your Research

Within the generate_keyword_ideas.py you can adjust the scope (region and language of your keyword research). By default, it is set to German language and Germany as a region. you find the parameters at the end of the script.

def run():
    location_ids = ['2276']  ##adjust with your location_id
    language_id = '1001'     ## adjust with your language_id
    seed_keywords = read_seed_keywords('data/seed_keywords.txt')
    all_results = []

    fieldnames = ['seed_keyword','keyword', 'avg_monthly_searches', 'competition', 'competition_index', 'month', 'monthly_searches','cpc','low_range_bid','high_range_bid','keyword_annotation','brand_bool','concept_group','concept_name','language_id','location_id']
    write_csv_header('data/keyword_ideas.csv', fieldnames)
    
    for name in seed_keywords:
        result = main(client, customer_id, location_ids, language_id, name, name)
        if result:
            all_results.extend(result)
        time.sleep(1)  
    
    write_to_csv(all_results, 'data/keyword_ideas.csv')

Google has its own ID logic for regions and languages. You can find the most recent list of regions and languages in this folder and the corresponding CSV files: https://github.com/floriankuwala/googlesearch2postgres/tree/main/resources

Define a List of Seed Keywords

Now you can define in data/seed_keywords.txt your keywords that you want to get keyword ideas for. Write just one keyword per line into the text file. … you can do it like this:

Apache207
Kendall Jenner
Fruitbag
Wurstsalat

Launch and Harvest!

Now start from the root directory the main.py with

python3 main.py

You will be asked in the terminal prompt for an action to perform. When you start the repo the first time you need to select a) (enter a) to generate keyword ideas. If you want to push your data to your postgres database choose b. You exit the program with e.

Select an option:
a) Generate Keywords and Search Volume
b) Upload Keywords with Search Volume to Database
e) Exit
Enter your choice: ___

Get the Output

You will be informed of the process through the terminal. Once the process is finished you will find your result csv in the data folder. If you started the Postgres connector you will find a new table in your Postgres.