How to get a room in Berlin using Firestore + Google Functions + Google Scheduler

Photo by 🇨🇭 Claudio Schwarz | @purzlbaum on Unsplash

The first part of this series of posts was about creating a scraper tool in NodeJS that allowed us to send a message to the owner from a template.

The main problem that arises with that approach is to have the script running 24 hours a day. A possibility is to deploy it into a free tier VM or having a dedicated raspberry pi but both alternatives imply monitoring, configuration, deployment, etc.

In this case, I’m using an alternative:

• Firestore for data storage
• Cloud functions for the script
• Cloud scheduler for the repetition

The objective is to have a free alternative as well so I will keep it simple.

Prerequisites

• Have a Google Cloud account. https://console.cloud.google.com
• Have the Firebase CLI installed and set up: https://github.com/firebase/firebase-tools

Let's get started

Create and configure the Firebase Project

1. Go to https://console.firebase.google.com and create a project

2. Give your project a representative name

3. In this case, Google Analytics its not mandatory

4. Once the project is created its time to create a database to store the data. We will use Firestore for this.

5. Go to the Database section

6. Start in test mode to enable quick writing

7. Set the proper location. For Wg-gesucht the best is to have it placed in Europe:

8. Once the Firestore database has been created its time to create a Collection to store the data.

Clone and deploy the function

Now we need to deploy the Functions to start scrapping and posting messages to the owners. The functions are in this repo. In order to deploy it to the project you can:

git clone https://github.com/juancruzmartino/wg-gesucht-scrapper-firebase

Change the FILTER_URL and the credentials inside the index.js before proceeding.

cd functions
npm install
cd ..
firebase deploy —YOUR-PROJECT-ID

Configuring the Google Scheduler

1. Go to https://cloud.google.com/scheduler/ and create a Job

2. Setup the scrapper:

3. Replace the URL so it matches your project ID.

4. Do the same for the autoreserve function:

From now on, the scripts should execute every 5 minutes getting all the new rooms that match your filter and sending the message to the owner using the specified templates.

You can change the execution frequency or disable the functions as need it.