PinnedShashwath ShenoyinDev GeniusChoosing Datahub for Data Engineering: A Comprehensive Comparison with Open MetadataIn today’s data-driven world, effective data management is crucial for businesses to gain insights and make informed decisions. We recently…Jul 1, 20236Jul 1, 20236
PinnedShashwath ShenoyinthedatalchemistMy Journey into the field of Big Data — Data EngineeringAs a Big Data Architect with 12+ years of industry experience, I have had the privilege of witnessing firsthand the tremendous growth and…Mar 29, 2023Mar 29, 2023
PinnedShashwath ShenoyDifferences between Mediocre and Excellent Data engineers!In the ever-evolving world of data engineering, mediocrity is the enemy of progress. After 13 years at the helm of data engineering, I’ve…Nov 6, 20231Nov 6, 20231
PinnedShashwath ShenoyJourney to 10K Followers: A Data Engineering Leader’s Guide to Building an Engaged CommunityEmbarking on a journey of knowledge-sharing has been a cornerstone of my 13-year career in data engineering. LinkedIn, as a professional…Nov 12, 2023Nov 12, 2023
PinnedShashwath ShenoyImplementing Change Data Capture (CDC) in MySQL: A Journey of Data Engineering Leader!Change is the only constant, and in the data-driven world, nowhere is that more true than in the realm of databases. I’ve been working in…Dec 17, 2023Dec 17, 2023
Shashwath ShenoyTime-Travel feature — Schema where you can query the data at any point in time!Time-travel in data systems enables querying data as it was at a particular point in time. This is a powerful feature for regulatory…20h ago20h ago
Shashwath ShenoyUnderstanding Referential Integrity and How It Is Enforced in a Database!In database design, ensuring data consistency and accuracy is critical for maintaining a healthy system. One of the key concepts in…3d ago3d ago
Shashwath ShenoyUnderstanding Spark Streaming: Core Components and Optimization Techniques!Apache Spark Streaming is a powerful extension of the core Spark API that enables real-time data processing. It has become widely used for…5d ago5d ago
Shashwath ShenoyCommon Challenges When Working with Real-Time Data Pipelines!Real-time data pipelines have become essential for businesses to stay competitive in today’s fast-paced world. As a data engineering leader…Sep 8Sep 8
Shashwath ShenoyTechniques for Optimizing Distributed Joins in a Big Data Environment!In the world of big data, distributed joins are a critical operation, but they can also be a performance bottleneck if not managed…Sep 1Sep 1