Wrong AI

Wrong Data Talent — 8 traits of DQ (Data IQ)

The data landscape is rapidly evolving. The skills and technical expertise required today for data roles such as data engineers, data analysts, data scientists, ML engineers, will be significantly different in 5 years.

The curse of Schema-on-read in Data Lakes

In the past, data warehouses were schema-on-write requiring DBAs and Data Architects to coordinate and interpret the data attributes before persisting within the warehouse. In contrast, Data Lakes today are schema-on-read making it easy to add any data…

Scorecard of a Data Platform for Analytics & ML

Enterprises in every vertical are developing Data Platforms to enable Analytics and ML-based insights. The charter of most Data…

Managing Data Issues as Incidents

Correctness of data pipelines is critical for in-product customer experiences (such as ML-based…

True North of a Modern Data Platform: Any Data, Any Analytics, Any User

At Intuit, we are building a world-class data platform for analytics and ML that is 100% cloud-based — a platform that supports a wide-range of analytics use-cases for our business operations (marketing…

Re-thinking Data Pipeline Patterns

A Data pipeline is a sequence of transformations that converts raw data into actionable insights. In the past, the processing and storage engines were coupled together e.g., a traditional MPP warehouse combines both a processing and storage engine. With decoupled…

Blurring of lines: Is it a Data Warehouse? Is it a Big Data Platform?

Traditionally, data warehouses…

Query Engine looks interesting — my pre-check checklist

Modern Data Platforms decouple compute and storage. Storage is centralized and is typically either S3 or HDFS. There are a plethora of choices for the data query engines namely Spark, Presto, Impala, Drill, Hive, etc.