Scraping the Surface of Airtable (Part 1)

  1. this post is not sponsored by airtable or me promoting it. I’m just another user who is excited with certain product, and interested in make the most out of it.
  2. It is not a Tutorial, it is a journey of how I explore web scraping. So it may not be a good reference on how to do things.

Introduction

First let me introduce you to Airtable. In Airtable you can make a base which analogues to worksheet file in Google Sheet. And fortunately Airtable offers lots of template for us to start with.

A lot of template for your personal use to professional project management at work

What I Want

Actually what I want to do is pretty simple. We have multiple team, and each has a base with the same format and template. And every now and then I need to keep track of what’s happened in just one table of each base that every team has. And also I need to make a summary from them. And to be honest checking each of them is such a pain.

Let’s Start!

0. My Plan

Well in short I want to be able to copy the shared view link for a table and have a program to extract the data I want and make an aggregate of the data in a spreadsheet so that I won’t be bothered by the free plan limitation of airtable.

This is what I want to achieve

1. Searching the data

This is the view of what I want to scrape, you also can access it here

The shared view in chrome
the complex structure of HTML
  1. Scrape the data from HTML
  2. Try to get the data from HTTP request

2. Finding the Request

After looking closely on how the page load, I guess I could find out how the data is loaded into the table. So I used Network tab in Chrome DevTools and… There!

There’s time when the content suddenly get populated
THE DATA

3. Pulling the Request ?

So how should I get it, I could just copy pasting it to postman and it will works right?

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Aditya Rachman Putra

Aditya Rachman Putra

Misadventure of programming, doodle, and friends