Homepage
Open in app
Sign in
Get started
Towards Data Engineering
Navigating the Path to Data Engineering Excellence
About
Follow
Trending
Why Parquet is the Best File Format for Big Data
Why Parquet is the Best File Format for Big Data
When working with big data, you want to store your data in a way that makes it easy to handle, quick to process, and doesn’t take up too…
Vishal Barvaliya
Aug 31
Decodable vs. AWS Managed Service for Apache Flink (MSF): An End-to-End Data Engineering Showdown
Decodable vs. AWS Managed Service for Apache Flink (MSF): An End-to-End Data Engineering Showdown
In recent times, realtime data processing is becoming more and more essential. The ability for the business to proactively react to…
Yusuf Ganiyu
Sep 1
Latest
End-to-End AWS KMS Encryption and Decryption Tutorial
End-to-End AWS KMS Encryption and Decryption Tutorial
We’re excited to share our new tutorial on Keyper. Keyper v0.0.3 now supports AWS (in addition to GCP) for end-to-end data and file…
Lulu Cheng
Sep 10
Troubleshooting Spark Jobs: Overcoming Errors and Performance Challenges — Part 2
Troubleshooting Spark Jobs: Overcoming Errors and Performance Challenges — Part 2
Identifying and Resolving Common Spark Job Failures, Debugging Techniques, and Optimizing Performance for Large-Scale Data Processing
Pritam Deb
Sep 9
Centralized AWS CloudWatch log collection to S3
Centralized AWS CloudWatch log collection to S3
Sebastian Daberdaku
Sep 9
Code and Library Reusability in Microsoft Fabric Notebooks
Code and Library Reusability in Microsoft Fabric Notebooks
How to reuse libraries and methods in your Microsoft Fabric implementations.
Matthew Sayer
Sep 9
Mastering SQL Recursive CTEs: Key Concepts and Interview Questions
Mastering SQL Recursive CTEs: Key Concepts and Interview Questions
Unlock the Power of Recursive CTEs in SQL: Learn Key Concepts, Hierarchical Data Solutions, and Top Interview Questions to Boost Your…
Pritam Deb
Sep 6
Setting up Environments in Microsoft Fabric
Setting up Environments in Microsoft Fabric
What are Fabric Environments, how to configure and use them.
Matthew Sayer
Sep 2
Secure Data Stack: Navigating Adoption Challenges of Data Encryption
Secure Data Stack: Navigating Adoption Challenges of Data Encryption
Encryption is one of the most effective data security strategies, alongside access control, software updates, and network segmentation…
Lulu Cheng
Sep 3
The Generations Of Data Architecture: Past, Present, and Future
The Generations Of Data Architecture: Past, Present, and Future
The Evolution of Data Architecture | Everything You Need to Know About Modern Data Architecture.
RAKESH CHANDA
Sep 1
Introduction to Web3 Data Engineering
Introduction to Web3 Data Engineering
From silos to global data supply chains
Sergii Mikhtoniuk
Aug 29
Data Security Strategy Beyond Access Control: Data Encryption
Data Security Strategy Beyond Access Control: Data Encryption
The cost, frequency, and scale of data breaches continue to rise year over year. According to the IBM 2023 Cost of a Data Breach Report…
Lulu Cheng
Aug 29
About Towards Data Engineering
Latest Stories
Archive
About Medium
Terms
Privacy
Teams