Gaurang ShahinTowards DevGCP Cloud Composer — Configure Hashicorp Vault to store connection and variables — Big Data…Airflow providers multiple secrets backends to be configured for storing airflow connection and variables. for a long time we were using…Jul 15Jul 15
Gaurang ShahGCP Cloud Composer — Things to consider before implementation — Big Data ProcessingWhen planning to use Google Cloud Composer (Airflow in GCP), there are few essential considerations to address before setup. While these…Jul 5Jul 5
Gaurang ShahGCP Security — Finding Zero Trust Policy issues using IAM policy Recommander — Big Data ProcessingIn our previous blog posts, we explored leveraging Google Recommender for cost optimization. Now, let’s dive into identifying security…Apr 5Apr 5
Gaurang ShahinTowards DevOptimize Your GCP Cloud Costs: Identifying Compute Engine Resources for Scale Down — Big Data…In our ongoing exploration of cloud cost optimization, we’re constantly seeking ways to maximize efficiency and minimize expenditure. In…Mar 21Mar 21
Gaurang ShahinTowards DevUnlocking GCP Cost Optimization Using Recommendation and BigQuery: A FinsOps Guide — Big Data…In recent days, I was working on costs optimization on the Google Cloud Platform (GCP) . Google offers recommendations for cost savings…Mar 5Mar 5
Gaurang ShahinTowards DevData Mesh implementation with GCP BigQuery — Big Data ProcessingIn today’s data-driven landscape, organizations often have multiple teams comprising domain experts who create their own unique products…Feb 8Feb 8
Gaurang ShahinTowards DevGCP Bigquery — Find Partition column for each table — Big Data ProcessingWhat Are Partition Columns?Sep 22, 2023Sep 22, 2023
Gaurang ShahinTowards DevGCP — Create Custom Bigquery Linage using DataCatalog Python API — Big Data ProcessingCreate Custom Data Linage for your bigquery tables using dataplex datalinage python client.Jul 31, 20231Jul 31, 20231
Gaurang ShahinTowards DevGoogle Bigquery — Find Query Cost By User — Big Data ProcessingThe organization I am currently consulting has recently migrated to the Google Cloud Platform (GCP) to leverage its powerful services…Jun 2, 20232Jun 2, 20232
Gaurang ShahinTowards DevAirflow Operational Dashboard using Bigquery and Looker Studio — Big Data ProcessingIn our daily operations, we rely on Airflow (Cloud Composer) to run hundreds of dags. While we have integrated it with ServiceNow and SMTP…May 10, 20231May 10, 20231