PinnedShashwath ShenoyinDev GeniusChoosing Datahub for Data Engineering: A Comprehensive Comparison with Open MetadataIn today’s data-driven world, effective data management is crucial for businesses to gain insights and make informed decisions. We recently…Jul 1, 20236Jul 1, 20236
PinnedShashwath ShenoyinthedatalchemistMy Journey into the field of Big Data — Data EngineeringAs a Big Data Architect with 12+ years of industry experience, I have had the privilege of witnessing firsthand the tremendous growth and…Mar 29, 2023Mar 29, 2023
PinnedShashwath ShenoyDifferences between Mediocre and Excellent Data engineers!In the ever-evolving world of data engineering, mediocrity is the enemy of progress. After 13 years at the helm of data engineering, I’ve…Nov 6, 20231Nov 6, 20231
PinnedShashwath ShenoyJourney to 10K Followers: A Data Engineering Leader’s Guide to Building an Engaged CommunityEmbarking on a journey of knowledge-sharing has been a cornerstone of my 13-year career in data engineering. LinkedIn, as a professional…Nov 12, 2023Nov 12, 2023
PinnedShashwath ShenoyImplementing Change Data Capture (CDC) in MySQL: A Journey of Data Engineering Leader!Change is the only constant, and in the data-driven world, nowhere is that more true than in the realm of databases. I’ve been working in…Dec 17, 2023Dec 17, 2023
Shashwath ShenoyinthedatalchemistDebugging Out of Memory (OOM) Errors in Apache Spark: A Comprehensive Guide!Out of Memory (OOM) errors are one of the most frustrating issues Spark developers encounter, especially when working with large datasets…3d ago3d ago
Shashwath ShenoyinthedatalchemistIs It Right to Compare a CEO’s Salary with a Fresher’s in the Tech industry?In recent years, the debate over income inequality has become increasingly prominent, with many questioning whether the vast gap between a…5d ago5d ago
Shashwath ShenoySharding: The Key to Scaling Your Database for High Performance!As data grows in volume and complexity, traditional database systems often struggle to maintain performance and efficiency. This is where…Aug 10Aug 10
Shashwath ShenoyApache Airflow: The Game Changer for Orchestrating Data Pipelines!In the fast-paced world of data engineering, orchestrating complex workflows and ensuring seamless data processing is no small feat. As a…Aug 9Aug 9
Shashwath ShenoyinDev GeniusUsing Kubernetes as a Resource Manager Instead of YARN!Apache Hadoop YARN (Yet Another Resource Negotiator) has been the go-to resource manager for many big data applications. However, with the…Aug 7Aug 7