Published inData Engineer’s NotesWhy you should care about VS Code Remote DevelopmentMaybe save a bit on that next expensive laptop upgrade?Nov 12, 2024Nov 12, 2024
Published inData Engineer’s NotesFor quality work, understand your data grain firstThere’s one concept that is central when working with data and goes beyond specific technology or SQL dialects. You just can’t miss it. I’m…Nov 9, 2024Nov 9, 2024
Published inData Engineer’s NotesSometimes, you have to use subqueries!Query without FROM clause cannot have a WHERE clause, goes the old SQL adage.Nov 1, 2024Nov 1, 2024
Published inData Engineer’s NotesNot all NULLS are the sameSo NULLs are definitely beasts of their own and as Data Engineers we come to learn to take them into account.Oct 27, 2024Oct 27, 2024
Published inData Engineer’s NotesExtracting keys from JSON in BigQueryA couple of months ago, I’ve posted about dynamically extracting key-value pairs from JSON in BigQuery SQL which leveraged regex.Oct 27, 2024Oct 27, 2024
Published inData Engineer’s NotesComputing a hash aggregation in BigQuerySo I’ve seen Snowflake has an HASH_AGG function. When would we need it? Every time we’d like to work out if ANY value in a group (or the…Oct 21, 2024Oct 21, 2024
Published inData Engineer’s NotesAnother look at ANY_VALUE in BigQueryA reminder that ANY_VALUE is a pretty interesting aggregation function in BigQuery SQL.Oct 20, 20243Oct 20, 20243
Published inData Engineer’s NotesWhy partitioning tables is not a silver bullet for BigQuery performanceI recently encountered an interesting case that reminded me of a couple of things and taught me a few lessons.Oct 15, 20244Oct 15, 20244
Published inData Engineer’s NotesUsing RANGE_BUCKET in BigQueryI’ve recently had to perform a validation of differences between 2 data sources, so I’ve figured it would be interesting to see the…Sep 23, 2024Sep 23, 2024
Published inData Engineer’s NotesWhy how the data is collected mattersI find that working with data daily significantly influences how you perceive the world around you. You begin to notice patterns and…Aug 6, 2024Aug 6, 2024