PinnedSteve RussoinBetter ProgrammingI Asked ChatGPT to Build a Data Pipeline, and Then I Ran ItYour job might be safe. For now…Apr 5, 202310Apr 5, 202310
PinnedSteve Russo5 Books that Make You a Better Data EngineerAnd one book that you didn’t know you neededJul 1, 202210Jul 1, 202210
Steve RussoUse Rust to Write Spark AppsUntil Spark 3.4, developing and deploying a Spark application was sometimes a big hassle. Getting Spark running locally for development…Jul 3Jul 3
Steve RussoinBetter ProgrammingWriting PySpark UDFs in RustPerformance comparison of different UDF methodsMay 2, 2023May 2, 2023
Steve RussoPassing the Databricks Professional Data Engineer ExamHow to prepare for the Databricks Data Engineer Professional ExamMar 27, 20234Mar 27, 20234
Steve RussoinBetter Programming4 Delta Lake Metadata Queries You Should KnowGet all the metadata you could want about your LakehouseOct 26, 2022Oct 26, 2022
Steve RussoinBetter ProgrammingBuild a FastAPI on the LakehouseCreate a FastAPI on the Databricks Lakehouse with full CRUD capabilitiesJul 18, 20222Jul 18, 20222
Steve RussoinBetter ProgrammingHow to Integrate Great Expectations with DatabricksGet better data quality metrics with one change to Great ExpectationsJul 7, 20223Jul 7, 20223
Steve Russo3 Methods for Dealing with Bad Data QualityDesign to handle bad data before it poisons your platform!Jun 30, 2022Jun 30, 2022
Steve RussoinBetter Programming3 Examples of Motifs with Spark GraphFramesFind specific relational patterns in a graphJun 18, 20221Jun 18, 20221