Scrape, Clean and Store Zillow Apartment Data (ETL Pipeline)

Use Python to find an apartment on Zillow.

Want to build a data science project like this? Create a job-worthy data portfolio. Learn how with my free project guide.

Photo by Marla Prusik on Unsplash

With many Zillow projects and tutorials focused on home buying, being a current apartment dweller, I thought it would be interesting to obtain Zillow apartment data, since the data returned is slightly less variable than home data and, in my opinion, can be more interesting to examine.

I’ll demonstrate the three main steps involved in getting recent apartment data:

1. Scraping a Zillow web page for apartments in Orlando
2. Cleaning/transforming the resulting data frame
3. Storing the 400+ rows in a BigQuery table for later analysis

I’ll cover methods you may have encountered including: BeautifulSoup, Pandas operations for data frame manipulation, basic SQL and the BigQuery API.

This is part 1 in a 2 part series. Part 2 will be published next week.

Scraping Zillow

Unlike sites that feature heavy text like Wikipedia, Zillow incorporates many visual and dynamic elements like slide…