Massively scalable web scraping using decentralized community computers

The following sample code will get you started with scraping websites and collect valuable information in a scalable way.

https://github.com/planetrio/function-examples/tree/master/nodejs/web-scraping

Since planetr can scale on thousands of community nodes on a decentralized network, one can achieve massive scalability. Keep a list of domains or pages to scrape and iterate through the list and invoke the function with one domain at a time. This will parallelize the requests on the planetr.io network and compute nodes will do the real scraping and result will be available at the master node. Cool?

Planetr.io Channel

Decentralized Serverless Computing - Planetary Scale

planetr.io

Written by

Planetary Scale Serverless Computing

Planetr.io Channel

Decentralized Serverless Computing - Planetary Scale