Reducing costs with AWS Lambda and Elasticsearch/Elasticache for scraping
In our business requirement, we need to scrape some web pages. Everyday, we need to send out 300k to 500k requests to get documents from different websites(domains). We used to create three EC2 instances…