Amar_KumarMastering Big Data ETL Pipelines: Harnessing Apache Airflow for orchestration and AWS Glue Jobs· Introduction · Use -case: · Components of Airflow: ∘ Directed Acyclic Graphs (DAGs): ∘ Operators: ∘ Sensors: ∘ Connections: ∘…Oct 11, 20231Oct 11, 20231
Amar_KumarDemystifying Delta Lake with AWS EMR : A CDC Use CaseIn the ever-evolving landscape of data storage and processing, three distinct solutions have emerged as game-changers: Data Lakes, Data…Sep 15, 2023Sep 15, 2023
Amar_KumarBuilding a Data Lakehouse with AWS EMR and Apache Hudi: A CDC Use Case — part2/2In the ever-evolving landscape of data storage and processing, three distinct solutions have emerged as game-changers: Data Lakes, Data…Sep 14, 2023Sep 14, 2023
Amar_KumarBuilding a Data Lakehouse with AWS EMR and Apache Hudi: A CDC Use Case — part1/2In the ever-evolving landscape of data storage and processing, three distinct solutions have emerged as game-changers: Data Lakes, Data…Sep 14, 2023Sep 14, 2023
Amar_KumarBuilding a Data Lake with Amazon S3 and EMRImplementing Upsert Operation on top of the data in the data lake.Sep 14, 2023Sep 14, 2023
Amar_KumarChoosing the Right Data Storage Paradigm: Data Lake vs.In the ever-evolving landscape of data storage and processing, three distinct solutions have emerged as game-changers: Data Lakes, Data…Sep 14, 20231Sep 14, 20231