PinnedShashwath ShenoyinDev GeniusChoosing Datahub for Data Engineering: A Comprehensive Comparison with Open MetadataIn today’s data-driven world, effective data management is crucial for businesses to gain insights and make informed decisions. We recently…Jul 1, 20236Jul 1, 20236
PinnedShashwath ShenoyinthedatalchemistMy Journey into the field of Big Data — Data EngineeringAs a Big Data Architect with 12+ years of industry experience, I have had the privilege of witnessing firsthand the tremendous growth and…Mar 29, 2023Mar 29, 2023
PinnedShashwath ShenoyDifferences between Mediocre and Excellent Data engineers!In the ever-evolving world of data engineering, mediocrity is the enemy of progress. After 13 years at the helm of data engineering, I’ve…Nov 6, 20231Nov 6, 20231
PinnedShashwath ShenoyJourney to 10K Followers: A Data Engineering Leader’s Guide to Building an Engaged CommunityEmbarking on a journey of knowledge-sharing has been a cornerstone of my 13-year career in data engineering. LinkedIn, as a professional…Nov 12, 2023Nov 12, 2023
PinnedShashwath ShenoyImplementing Change Data Capture (CDC) in MySQL: A Journey of Data Engineering Leader!Change is the only constant, and in the data-driven world, nowhere is that more true than in the realm of databases. I’ve been working in…Dec 17, 2023Dec 17, 2023
Shashwath ShenoyApache Airflow: The Game Changer for Orchestrating Data Pipelines!In the fast-paced world of data engineering, orchestrating complex workflows and ensuring seamless data processing is no small feat. As a…14h ago14h ago
Shashwath ShenoyinDev GeniusUsing Kubernetes as a Resource Manager Instead of YARN!Apache Hadoop YARN (Yet Another Resource Negotiator) has been the go-to resource manager for many big data applications. However, with the…2d ago2d ago
Shashwath ShenoyinthedatalchemistUnderstanding Slowly Changing Dimensions (SCD) in Data Warehousing!Slowly Changing Dimensions (SCD) are crucial in data warehousing for managing and tracking changes in dimension tables. They ensure that…2d ago2d ago
Shashwath ShenoyUnderstanding Resource and Memory Management in Apache Spark!Apache Spark is an open-source, distributed computing system that provides an interface for programming entire clusters with implicit data…Jul 20Jul 20
Shashwath Shenoy7 Database Schema Design Traps and How to Dodge Them: A Guide for Data Engineers!Designing a database schema is a critical task that can significantly impact the performance, scalability, and maintainability of your…Jul 7Jul 7