PinnedShashwath ShenoyinDev GeniusChoosing Datahub for Data Engineering: A Comprehensive Comparison with Open MetadataIn today’s data-driven world, effective data management is crucial for businesses to gain insights and make informed decisions. We recently…Jul 1, 20236Jul 1, 20236
PinnedShashwath ShenoyinthedatalchemistMy Journey into the field of Big Data — Data EngineeringAs a Big Data Architect with 12+ years of industry experience, I have had the privilege of witnessing firsthand the tremendous growth and…Mar 29, 2023Mar 29, 2023
PinnedShashwath ShenoyDifferences between Mediocre and Excellent Data engineers!In the ever-evolving world of data engineering, mediocrity is the enemy of progress. After 13 years at the helm of data engineering, I’ve…Nov 6, 20231Nov 6, 20231
PinnedShashwath ShenoyJourney to 10K Followers: A Data Engineering Leader’s Guide to Building an Engaged CommunityEmbarking on a journey of knowledge-sharing has been a cornerstone of my 13-year career in data engineering. LinkedIn, as a professional…Nov 12, 2023Nov 12, 2023
PinnedShashwath ShenoyImplementing Change Data Capture (CDC) in MySQL: A Journey of Data Engineering Leader!Change is the only constant, and in the data-driven world, nowhere is that more true than in the realm of databases. I’ve been working in…Dec 17, 2023Dec 17, 2023
Shashwath ShenoyCommon Challenges When Working with Real-Time Data Pipelines!Real-time data pipelines have become essential for businesses to stay competitive in today’s fast-paced world. As a data engineering leader…1d ago1d ago
Shashwath ShenoyTechniques for Optimizing Distributed Joins in a Big Data Environment!In the world of big data, distributed joins are a critical operation, but they can also be a performance bottleneck if not managed…Sep 1Sep 1
Shashwath ShenoyLessons from My First Interview in 2010 and What Freshers Need to Prepare for Data Engineering in…In 2010, the job market was tough, especially for freshers like me who were trying to break into the industry without campus placements…Aug 31Aug 31
Shashwath ShenoyUnderstanding the Architecture of Databricks Delta Lake and How It Handles ACID Transactions!In today’s data-driven world, the need for reliable, scalable, and consistent data storage solutions is paramount. Enter Databricks Delta…Aug 25Aug 25
Shashwath ShenoyinthedatalchemistDebugging Out of Memory (OOM) Errors in Apache Spark: A Comprehensive Guide!Out of Memory (OOM) errors are one of the most frustrating issues Spark developers encounter, especially when working with large datasets…Aug 17Aug 17