Daniel StahlMetadata driven development: a new paradigm for data engineeringSoftware engineering is fundamentally a discipline dedicated to abstraction. Rather than writing binary, write assembly; rather than write…Nov 11, 2023Nov 11, 2023
Daniel StahlThe importance of understanding the problem domain in Data ScienceI will show, using a contrived example, the potential pitfalls in using statistics and machine learning without knowledge of the domain for…Aug 6, 2022Aug 6, 2022
Daniel StahlWhy are there so few multi-channel digital pre-processors?I usually write about data, software, and tech…but I thought I’d give writing about my A/V hobby at least one post as well.Jun 20, 2022Jun 20, 2022
Daniel StahlLET: the next evolution of ETL and ELTTo oversimplify, there are two primary use cases for data storage: to store data for atomic consumption by end users, and to store data for…Apr 16, 2022Apr 16, 2022
Daniel StahlA new mindset for a new eraIt was not until 2014 that I first came across a statistical model that had more than a thousand training samples. Not only was the…Mar 13, 2022Mar 13, 2022
Daniel StahlThe ultimate Machine Learning platform: GithubI have used Github for years. My Github page is littered with half-baked libraries, abandoned projects, and in a few cases some genuinely…Feb 25, 2022Feb 25, 2022
Daniel StahlRethinking Lambda Architecture in the age of Streaming DatabasesA “traditional” lambda architecture consumes live events, performs transformations on these events in real time, caches these…Feb 20, 2022Feb 20, 2022
Daniel StahlExperimenting with DaskFor the last 3 years, I have primarily worked with Spark. Spark has a great ML library of common tabular algorithms and some basic NLP…Feb 18, 2022Feb 18, 2022
Daniel StahlGitOps for Machine LearningWhen machine learning algorithms were initially starting to be deployed at scale, they were difficult to manage. Data scientists use code…Sep 20, 2020Sep 20, 2020
Daniel StahlAWS Lambda to GCP Cloud RunI’ve been a long-term AWS user and proponent. I started using Lambdas for simulations and financial models years ago when Lambdas were…Feb 29, 2020Feb 29, 2020