Become a member
Sign in
Asa Harland
Asa Harland

Asa Harland

6 Following
29 Followers
  • Profile
  • Claps
  • Responses

Latest

Asa Harland
Asa Harland
Sep 17, 2018 · 3 min read

Micro-batching with Apache Beam and BigQuery

In the previous article we looked using load jobs to import flat files into BigQuery. However, there are certain limits we have to play by when loading data into BigQuery typically caused by micro-batching data into BigQuery.

4

Asa Harland
Asa Harland
Sep 11, 2018 · 4 min read

Loading Data Into BigQuery With Cloud Functions

34

6 responses
Asa Harland
Asa Harland
Sep 1, 2018 · 4 min read

Apache Beam: Reading from S3 and writing to BigQuery

In this article we look at how we can use Apache Beam to extract data from AWS S3 (or Google Cloud Storage), run some aggregations over the data and store the result in BigQuery. In most use cases I would recommend storing and loading…

60