Kishore DesettiPyspark job in dataproc to parse json, filter and write to Google cloud bigqueryHi folks , in this article I would like to explain how to setup a pyspark job to run code which will parse an input json file, do one…Aug 15, 2023Aug 15, 2023
Kishore DesettiSetting up subscription pipeline to read message from pubsub and write to bigqury using option…Hi ,in this article lets dive into the steps on reading messages from pubsub topic and writing to bigquery using this relatively new…May 27, 2023May 27, 2023
Kishore DesettiSetting up composer to run complex workflowHello friends, in this article I will explain on how to setup a composer job to run a complex workflow.May 26, 2023May 26, 2023
Kishore DesettiHow to setup a data flow pipeline to read CSV from GCS and write to bigquryHello, in this article I would like to help you with the steps on setting up a data flow to read csv data from google cloud storage bucket…Feb 7, 2023Feb 7, 2023
Kishore DesettiMigrating from hive to bigquery for external tablesOne of the typical use cases came across as part of on job is the migration from hive to bigquery hence thought will share my experiences…Jan 28, 2023Jan 28, 2023
Kishore DesettiHow I passed my GCP Professional Data engineer examHello Friends, considering the recent popularity and prominence of Google Cloud Platform (GCP) I took the certification on GCP Professional…Jan 28, 2023Jan 28, 2023
Kishore DesettiIngesting bulk test data into MongoDB using python multi threadingOff late, I came across a requirement to ingest bulk data into MongoDB as part of testing and I thought this could be little generalized…Jan 20, 20221Jan 20, 20221
Kishore DesettiHow I passed my GCP Professional Cloud Architect examHello Friends, considering the recent popularity and prominence of Google Cloud Platform (GCP) I first took the certification on GCP…Jan 13, 2022Jan 13, 2022
Kishore DesettiHow I passed my GCP Associate cloud engineer examHello Friends, considering the recent popularity and prominence of Google Cloud Platform (GCP) I decided to take the certification on this…Jan 12, 2022Jan 12, 2022
