PinnedShashwath ShenoyinDev GeniusChoosing Datahub for Data Engineering: A Comprehensive Comparison with Open MetadataIn today’s data-driven world, effective data management is crucial for businesses to gain insights and make informed decisions. We recently…Jul 1, 20236Jul 1, 20236
PinnedShashwath ShenoyinthedatalchemistMy Journey into the field of Big Data — Data EngineeringAs a Big Data Architect with 12+ years of industry experience, I have had the privilege of witnessing firsthand the tremendous growth and…Mar 29, 2023Mar 29, 2023
PinnedShashwath ShenoyDifferences between Mediocre and Excellent Data engineers!In the ever-evolving world of data engineering, mediocrity is the enemy of progress. After 13 years at the helm of data engineering, I’ve…Nov 6, 20231Nov 6, 20231
PinnedShashwath ShenoyJourney to 10K Followers: A Data Engineering Leader’s Guide to Building an Engaged CommunityEmbarking on a journey of knowledge-sharing has been a cornerstone of my 13-year career in data engineering. LinkedIn, as a professional…Nov 12, 2023Nov 12, 2023
PinnedShashwath ShenoyImplementing Change Data Capture (CDC) in MySQL: A Journey of Data Engineering Leader!Change is the only constant, and in the data-driven world, nowhere is that more true than in the realm of databases. I’ve been working in…Dec 17, 2023Dec 17, 2023
Shashwath ShenoyUnderstanding Referential Integrity and How It Is Enforced in a Database!In database design, ensuring data consistency and accuracy is critical for maintaining a healthy system. One of the key concepts in…1d ago1d ago
Shashwath ShenoyUnderstanding Spark Streaming: Core Components and Optimization Techniques!Apache Spark Streaming is a powerful extension of the core Spark API that enables real-time data processing. It has become widely used for…3d ago3d ago
Shashwath ShenoyCommon Challenges When Working with Real-Time Data Pipelines!Real-time data pipelines have become essential for businesses to stay competitive in today’s fast-paced world. As a data engineering leader…5d ago5d ago
Shashwath ShenoyTechniques for Optimizing Distributed Joins in a Big Data Environment!In the world of big data, distributed joins are a critical operation, but they can also be a performance bottleneck if not managed…Sep 1Sep 1
Shashwath ShenoyLessons from My First Interview in 2010 and What Freshers Need to Prepare for Data Engineering in…In 2010, the job market was tough, especially for freshers like me who were trying to break into the industry without campus placements…Aug 31Aug 31