Separation of metadata and data: Building a cloud native warehouse, Part 3

Dong Jiang
Mar 16, 2018 · 15 min read

What is metadata?

ACID and transaction are difficult

Schema-on-read, Pattern: Validate-Insert-Create

DDL is the new DML

Temporary View vs External Table

Schema Evolution

Removal of DDL or DML locks and data fragmentation

Test with real production data

Boosting performance by separating metadata and data

dataxu.technology

Notes and ideas from dataxu’s technology team

Dong Jiang

Written by

Software Architect @DataXu

dataxu.technology

Notes and ideas from dataxu’s technology team