I was looking for a house, so I built a web scraper in Python!
In a few months, I’ll have to leave my rented apartment and look for a new one. As painful as this experience can be, especially as a real estate bubble looms in the horizon, I decided to use it as yet another incentive to improve my Python skills! In the end, I want to be able to do two things:
· Scrape all the search results from one of the main real estate websites in Portugal (where I live) and build a database with all the listings found
· Use the collected listings to perform some EDA, and ultimately try to find undervalued properties
The website I will be scraping is the real estate portal from Sapo, one of the oldest and most visited websites in Portugal. They have a very large amount of real estate listings for us to scrape. Chances are you are using a different website, but you should be able to adapt the code very easily.
Before we begin with the code snippets, let me just give you a summary of what I will be doing. I will use the results page from a simple search in Sapo website where I can specify some parameters beforehand (like zone, price filters, number of rooms, etc) to reduce the task time, or simply query the whole list of results in Lisbon.
