Python Web Scraping PART:1

In the next few articles I am going to talk about my web scraping experience with python. I will explain the pro/cons of beautifulsoup, selenium, requests and why in the end I used scrapy.

The reason I got into web scraping is because I needed to find data to answer a digital marketing question about demographic. I first started with a google search web crawling with python and fine turned my google query along the way. The first library I came across was beautiful soup.

The reason I like beautiful soup is because it is an html parser and much more. I could target specific tags by id, class or attribute by using the find_all function. I used beautiful soup to parse veteran owned business information from https://www.veteranownedbusiness.com/ca. The first thing I did was use the google developer tools to inspect the source code. I noticed each listing was inside a class=”top_level_listings.

I noticed the <a href> link, when clicked you are redirected t o a page that provides specific information about the business, such as social media links, business address and contact information