Wei JianLessons learned when scaling dbt models quickly.I talked about why we build dbt models quickly rather than building it properly; prioritising working pipelines that are “messier” over…Sep 26, 2022Sep 26, 2022
Wei JianLessons learned after 1 year with dbt.I spoke to some of my fellow data friends about dbt, and how it made our ELT pipelines more manageable. Not everyone agreed with our…Aug 24, 20224Aug 24, 20224
Wei JianHow to effectively deal with too many data requestsUsing ticketing system like JIRA to identify areas where data requests becomes redundant, or when stakeholders aren’t sure what they want.Mar 19, 2022Mar 19, 2022
Wei JianinShopBack Tech BlogDesigning ELT pipelines with dbt (Part 3 — Finale)This article summarises the lessons learned when we migrated our old pipelines to dbt, and the methods we used to optimise its execution…Oct 1, 20211Oct 1, 20211
Wei JianinShopBack Tech BlogDesigning ELT pipelines with dbt (Part 2)This is the second part of a series about my data pipeline experience with dbt. Our focus in this article is on model structures and best…Aug 22, 20211Aug 22, 20211
Wei JianinShopBack Tech BlogDesigning ELT pipelines with dbt. (Part 1)Data transformation (the ‘T’ in ELT) is often regarded as the easy part, but doing so while ensuring data quality — accurate and updated…Aug 6, 2021Aug 6, 2021