Serverless: A journey to a no-ops Data Architecture on Google Cloud
Serverless is really a new and fancy term in the data world. A promise of minimum worries in infrastructure (no-ops) and pay only for what you use seems a dream, that invites me to go further and review all the serverless products available on GCP and then build an architecture that covers the basics of Data Orchestration, Data Processing, and Data Warehousing for a simple project.
Use Case
To keep this architecture reproducible I’ll be using the free COVID dataset taken from Google Cloud. My purpose here will be to create some KPIs about cases in Latin America. Remember that is important to have and GCP account if you want to implement the architecture.
Important
Google Cloud offers $300 in credits for first time users.
Architecture
As you can see there are many services involve in this architecture so let’s get some basic information about each one.
Data Warehouse
BigQuery is a fully-managed service, which means that the BigQuery engineering team takes care of updates and maintenance for you. Upgrades shouldn’t require downtime or hinder system performance [BigQuery Docs].
