Muhammad Rivaldi IdrisLoad Parquet File From AWS S3 to AWS Redshift Using AWS GlueAmazon Redshift, a powerful data warehousing solution offered by AWS, is a popular choice for organizations looking to store and analyze…Oct 3, 2023Oct 3, 2023
Muhammad Rivaldi IdrisEvent-Driven Data Pipeline with AWS EventBridge, Step Functions, Lambda and GlueIn this article, I will explain the step-by-step process for building an event-driven data pipeline using AWS services, such as…Oct 3, 2023Oct 3, 2023
Muhammad Rivaldi IdrisUsing Apache Airflow to Fetch Weather API Data and Store it in AWS S3In this article, I will explain the steps to retrieve data from a weather API, automate it using Airflow, and store it in an AWS S3 bucket…Oct 3, 2023Oct 3, 2023
Muhammad Rivaldi IdrisHow to Install Custom Image Airflow on Docker (Ubuntu)IntroductionJul 9, 2023Jul 9, 2023
Muhammad Rivaldi IdrisBuild Stock Price Data ETL Pipeline Using Python, Airflow and Google Cloud ServiceIn this article, I will show how to create simple ETL(Extract, Transform, Load) pipeline using Python and automate the process through…Jul 8, 20232Jul 8, 20232