PinnedDeepanshu tyagiinDataEngineering.pyEnd-to-End Data Engineering/ Analytics Project on AWS: IoT Sensors, Kinesis, S3, Athena, and…In this blog, I’ll guide you in building an AWS-based data engineering project that can be applied in various scenarios, from industrial…6 min read·Sep 26, 2023----
PinnedDeepanshu tyagiinDataEngineering.pyPopular Data Engineering StackYou Should Know in 2024As we dive into 2024, let’s take a sneak peek into the exciting world of data engineering. As technology changes every month, this blog…3 min read·Dec 3, 2023--2--2
Deepanshu tyagiinDataEngineering.pyOptimizing Spark Performance: Strategies for Faster Big Data ProcessingApache Spark has emerged as a powerhouse, providing speed, scalability, and flexibility. However, realizing its full potential takes more…4 min read·May 24, 2024----
Deepanshu tyagiinDataEngineering.pyMust-Know No-Code Data Engineering Platforms for Every Data EngineerHistorically, data engineering required good coding abilities, but with the introduction of no-code platforms, the environment is rapidly…4 min read·May 23, 2024--1--1
Deepanshu tyagiTop Data Visualization Best Practices for Data EngineersData visualization is more than just presenting data. It is also about using data to tell a story. As a data engineer, understanding the…4 min read·May 21, 2024----
Deepanshu tyagiIngest Data From Pub/Sub directly to Bigquery - Google Cloud Project with StepsTraditionally, ingesting data from Pub/Sub into BigQuery involved intermediary steps, such as writing and running Dataflow pipelines. While4 min read·May 15, 2024----
Deepanshu tyagiMust-Read Machine Learning Books in 2024Curated List of Machine Learning Beginner’s Books .As we enter 2024, a wealth of informative publications have arisen to meet the unique…3 min read·Jan 25, 2024----
Deepanshu tyagiComplete Apache PySpark Learning Resources with Links — Data EngineeringHandpicked collection of Apache Spark and Data Engineering resources.2 min read·Jan 22, 2024--1--1
Deepanshu tyagiGreat Open Source Data Engineering Projects That Deserve More RecognitionThis blog post provides a curated collection of open source projects from various areas within the data engineering domain.3 min read·Jan 20, 2024----
Deepanshu tyagiinDataEngineering.pyBigQuery vs Redshift: What to Learn in 2024 with Trends ExplainedIn this blog, we’ll look at the ever-evolving terrain of data engineering, with a focus on two giants: Amazon Redshift and Google BigQuery.4 min read·Jan 18, 2024----