Analytics

Understanding Big Data workloads from the Storage perspective

De-coupling of Compute & Storage in Modern Databases

Databases started as integrated storage-compute solutions providing both a durable persistence layer (storage) as well as a query processing engine (compute). Given the growing variety of data types, changing analytical processing needs…

Query Engine looks interesting — my pre-check checklist

Modern Data Platforms decouple compute and storage. Storage is centralized and is typically either S3 or HDFS. There are a plethora of choices for the data query engines namely Spark, Presto, Impala, Drill, Hive, etc.

The curse of Schema-on-read in Data Lakes

In the past, data warehouses were schema-on-write requiring DBAs and Data Architects to coordinate and interpret the data attributes before persisting within the warehouse. In contrast, Data Lakes today are schema-on-read making it easy to add any data…

Scorecard of a Data Platform for Analytics & ML

Enterprises in every vertical are developing Data Platforms to enable Analytics and ML-based insights. The charter of most Data…

True North of a Modern Data Platform: Any Data, Any Analytics, Any User

At Intuit, we are building a world-class data platform for analytics and ML that is 100% cloud-based — a platform that supports a wide-range of analytics use-cases for our business operations (marketing…

Re-thinking Data Pipeline Patterns

A Data pipeline is a sequence of transformations that converts raw data into actionable insights. In the past, the processing and storage engines were coupled together e.g., a traditional MPP warehouse combines both a processing and storage engine. With decoupled…

Blurring of lines: Is it a Data Warehouse? Is it a Big Data Platform?

Traditionally, data warehouses…