Scraping Tutorial on Airbnb’s Website (with Scrapy) | Scraping with 🐍
Making your first steps in automating data collection from a raw web page
This article is part of the series “Scraping with Python 🐍 ” where I intend to present and teach from basic to most experienced Scraping concepts.
“The first rule of web crawling is you do not harm the website. The second rule of web crawling is you do NOT harm the website.” — by Zite
Introduction
Web Scraping is the art of collecting data from the Internet. There are two reasons why I decided to go with Airbnb for this from-scratch tutorial. The first one is about the value of the data! Collecting data from Airbnb’s website allows comparing the evolution of prices in a city through times, or compare two cities services. Many use cases could be imagined using those data. The second reason is due to the complexity to scrap such a website. Indeed, Airbnb is a modern website having anti-scraping policies, which makes it (funnier to scrap 🤠 , and) a good way to discover patches. Eventually, it will accelerate your scraping skills development!
