Geek Culture
Published in

Geek Culture

Scrape DuckDuckGo Ad Results with Python

Contents: intro, imports, what will be scraped, process, code, links.

Intro

This blog post is a continuation of DuckDuckGo web scraping series. Here you’ll see how to scrape Ad Results using Python with selenium library. An alternative API solution will be shown.

Imports

from selenium import webdriver

What will be scraped

Process

Selecting title, link, snippet, source, sitelinks.

What I was doing is basically applying .results--ads selector before additional selector for the title, link, snippet, source, and sitelinks to get the most accurate selector.

A CSS selectors reference.

Code

Using DuckDuckGo Ad Results API

SerpApi is a paid API with a free plan.

The difference is that is much faster, you don’t need to render the page with selenium and you get an immediate structured JSON string.

Links

GitHub Gist DuckDuckGo Ad Results API

Yours,

- D

Originally published at https://serpapi.com on August 11, 2021.

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Dmitriy Zub ☀️

Dmitriy Zub ☀️

117 Followers

Developer Advocate at SerpApi. I help to make structured data from a pile of mess.