Apache Beam: Reading from S3 and writing to BigQuery
In this article we look at how we can use Apache Beam to extract data from AWS S3 (or Google Cloud Storage), run some aggregations over the data and store the result in BigQuery. In most use cases I would recommend storing and loading…