Collect leads For your Business in 5 Minutes Using Web Scraping.
This article describes how to scrape business listing from google places.Which can be used for any further data analysis tasks
Introduction
Purpose of this project.
First step for any successful business is to identify the target customers or potential leads.
This project aims at collecting leads for your business through google business listing and store details into CSV format later it can used for cold messaging ,promotional messages etc. Going forward in this article i will explain the application of Web scraping by taking an example of a business, for which we need to generate leads.
Problem Statement
Mr.XYZ is a wholesaler, he owns a sanitizer business he knew his target customer are all pharmacy retailers in his surrounding areas now he wants way to approach all his customers, for which he needs the basics information like the number of pharmacies in his surroundings in and around some target radius, address of the shops, opening-closing times, phone number of the owner, rating and no. of reviews to shop, etc.
Tackling the Problem…
We all know this information is available in google places, but it is not too handy directly using it. That is where the Web scraping comes handy, we can store all this information into one file, later can be used for various purposes.
What is web scraping?
In simple terms, it is the process of gathering information or data from different webpages (HTML sources). The information or data thus gathered can be used in building datasets or databases for different applications like (Data Analysis, Building a price comparison application,lead generation etc. )
Prerequisite:-
1. Basic understanding of Python 3.0 programming.
2. Python 3.0 or above installed in your system(Don’t forget to ADD python to the path while installing).
3. Make sure Chrome browser is installed in your system.which is essential in this project.
Libraries used :-
- BeautifulSoup :- This library is used for parsing the HTML from the web page.
- Csv :- This library is used to store all our collected data into csv file.
- pandas :- This library can be used for manipulating/cleaning of the collected data .
- Selenium :- selenium is the browser automation tool ,it has several application in automation testing.here i am using it to change webpages automatically.
For initial Steps refer to my another article
Step-1:- Creating the Virtualenv.
Step-2:- Activating the Virtualenv and installing the required libraries.
Step-3 :- Understanding HTML of the page you wanted to scrape by clicking on inspect element.
Here is the page we want to scrape.
The information we require are :-
1. Name of the pharmacy .
2. Rating and no. of reviews.
3. Address.
4. Phone Number if Available.
5. Opening and Closing times.
- Name of the pharmacy.
Note the tag and class of the element. Here tag = span , class =section-result-title-text
2. Rating and no. of reviews.
Here tag = span , class =rs9iHBFJiiu__rating
Here tag = span , class =rs9iHBFJiiu__reviews
3. Address/Location.
Here tag = span , class =section-result-location
4. Phone Number
Here tag = span , class =section-result-info section-result-phone-number
5. Opening and Closing times.
Here tag = span , class =section-result-info section-result-opening-hours
Step 4 :- Proceeding with the program that completely gather the data into a csv file.
Step 4.1 :- Import Required Libraries .
Step 4.2 :- Selenium Settings.
In this step
1. We are providing the URL which we want to scrape.
2. Setting Options for our selenium driver .
This process Requires you to download chrome webdriver as per your Chrome version.
Download link :- https://chromedriver.chromium.org/
NOTE :-Place the downloaded Chrome web driver in your Project folder
Step 4.3 :- Writing a function to collect and store the data into a csv.
Step 4.5 :- Function to Scrape multiple pages.
The entire code is available in my github, feel free to use it
Lets connect at :- linkedin.com/in/saicharan-kr
