How to web scrape with Puppeteer in Google Cloud Functions

Basic configuration

$ yarn init -y
$ yarn add puppeteer
$ sudo apt-get install google-cloud-sdk
$ yarn global add @google-cloud/functions-emulator --ignore-engines

Interactions with Google Cloud Storage

Handling bad website design

Tips and tricks

Memory management

Debugging

$ functions logs read
$ functions logs clear

DOM interactions

await page.on(‘console’, obj => console.log(obj.text()));

Using headless

Optimization

Sources

--

--

--

Full-stack Web developer

Love podcasts or audiobooks? Learn on the go with our new app.

Recommended from Medium

Minimize The Sum Of Product

Reverse Engineering a Gameboy Advance game: Understanding the Game Physics — Part 3

Solidity Fundamentals — Reference Types

Get started with Laravel Lumen / Introduction

Symfony Station Communique — 22 October 2021. A Look at Symfony and PHP News.

Choosing Elixir over Rust and Golang

PicoCTF — Information

An Inside Look at Deploying Kubernetes on AWS

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Benjamin Morali

Benjamin Morali

Full-stack Web developer

More from Medium

Authentication when using Google Cloud APIs from FileMaker

Authentication

Saleor 3.1: Gift Cards, Click & Collect, improved extensibility

Google App Engine

Low cost development using VS Code SSH remote extension with Google Cloud spot instances