Exploring San Francisco Apartments on Craigslist with R

Learn how to scrape Craigslist posts and visualize housing data in R

Whenever this quarantine situation lifts, I plan to move out to San Francisco and continue my full-time job in person. Since I hate the process of manually filtering through Craigslist for apartments, I figured I could apply some of my recently-acquired R skills to do the heavy lifting.

In this article, we’ll walk through how to use R to scrape Craigslist posts, clean data, and do some basic visualization.

Scraping Craigslist Posts

With some great guidance on web scraping Craigslist in R from Nikhil Kaza at UNC, the process was relatively painless.

Let’s start by loading in some packages:

pacman::p_load(tidyverse, rvest, DataExplorer, data.table)

Constructing the Query

The URL for Craigslist queries has a pretty standard structure. The first step is to replicate that query.