Abdelbarre ChafikinTowards DevMastering Data Governance with Unity Catalog OSS — with Technical demoIn today’s data-driven world, effective data governance is paramount for ensuring data quality, compliance, and security. Enter Unity…Jun 27Jun 27
Abdelbarre ChafikinTowards Dev🖥️Using Apache Iceberg with Apache Spark and Minio — DockerThis Post explores how to leverage Apache Iceberg, a data table format, in conjunction with Apache Spark, a distributed processing engine…Jun 51Jun 51
Abdelbarre ChafikinTowards DevBuild Data Engineering Pipeline using Spark, Delta , Mage Ai and MinioIn this data engineering post, we’ll explore cutting-edge open-source technologies that can assist us in constructing a resilient data…May 7May 7
Abdelbarre ChafikinTowards Dev🛠️ ETL Mage Ai, the Airflow replacementIn this article, I’ll guide you through a demonstration of Mage.ai, an efficient data pipeline tool tailored for automating your data…Apr 29Apr 29
Abdelbarre ChafikinTowards Dev🚀Leveraging Apache Spark with Delta Lake and Minio for Data Engineering processExplore Apache Spark + Delta Lake + Minio: powerful trio for big data. Efficient, reliable, scalable. 😉Apr 24Apr 24
Abdelbarre ChafikinTowards Dev🛠️CDC — Track the MySQL changes and save it into Delta format using Kafka and Spark StreamingChange Data Capture (CDC) with Delta Lake, Kafka, and Spark StreamingApr 21Apr 21
Abdelbarre ChafikinTowards DevChange Data Capture (CDC) with Docker, Apache Kafka, Debezium, and Apache Spark StreamingApr 171Apr 171
Abdelbarre Chafik🔧Article 2 : Using Models and Tests with dbt and Databricks: Ensuring Data Quality and Accuracy! ✅Now that you’ve mastered the basics of dbt and Databricks integration, let’s take it a step further and explore how to use models and tests…Jul 16, 2023Jul 16, 2023
Abdelbarre Chafik🔧Article 1 : Setting Up dbt with Databricks: Supercharge Your Data Transformations! 💼📊Article 1 : Databricks and Dbt setup and first model tryJul 8, 2023Jul 8, 2023
Abdelbarre ChafikSeamlessly Send Messages from Databricks to Slack, Ideal for Data Quality! 🚀Hey #DataScientists and #DataEngineers,Jun 23, 2023Jun 23, 2023
