Tutorials to Build Batch ETL Pipelines

Free tutorials and examples for data engineers to build batch pipelines.

Published in

Pipeline: Your Data Engineering Resource

1 min readMar 4, 2022

Building, scaling, maintaining and optimizing data pipelines are core responsibilities of a data engineer. Since datasets are often derived from external sources, it is imperative that you understand the process of extracting, transforming and loading data. Below are three examples from my work on Medium of how to practice and refine ETL skills.

Connect to an API with Python

Use the Reddit API and BigQuery to build a Daily News Pipeline

Despite being known as a thread-based social network, Reddit is one of the leading aggregators of news with communities…

medium.com

Access and Manipulate URL-based Data

How to Use Python to Access Data in Multiple URLs with Rick and Morty

The Rick and Morty REST API presents an interesting challenge: How to access data from multiple URLs at once using…

medium.com

Full-Stack ETL Project with Watch Data

How I Used Python and SQL to Find The Most Expensive Watches in the World

How to Build an End-to-End Data Analysis Project Using Python/SQL

towardsdatascience.com

Create a job-worthy data portfolio. Learn how with my free project guide.