Unistore -> An evolutionary workload of Snowflake
Context :: In Snowsummit-2022 out of all the various features which were announced by Snowflake one of them was “Unistore” a completely new workload primarily designed to solve OLTP & OLAP needs.
What is Unistore ?
Unistore is a new workload of Snowflake which delivers modern approach to work with transactional and analytical needs. With this feature teams can build transactional applications within Snowflake. This simplifies architecture and satandardize security/governance on a single platform eliminating the need of copying or moving data for OLTP/OLAP needs. All this is powered by an introduction of the new kind of table i.e., Hybrid tables.
What is hybrid table ?
Hybrid tables are a new Snowflake table type powering Unistore. A key design principle is to have this table support all the transactional capabilities need. These are highly performant which is a need of any transactional application & support fast single row operations. They work on entirely new row-based storage engine. This is unlike other tables in Snowflake where data is stored in columnar way.
Why Hybrid tables ?
Below picture depicts why hybrid tables (unistore workload) is important which is primarily because of 3 reasons i.e., to go away from
- Too many disparate systems
- Data movement challenges for OLTP and OLAP needs.
- Delayed access
Properties of HYBRID tables are as follows:
Other facts::
- Support of these new tables :: As of today it is supported for AWS cloud tenant and with limited locations but down the line it would be made available to all cloud providers and regions.
- Storage :: These hybrid tables data is stored as 2 copies one in the row storage & other in column storage. Hence a trade-off is that it would be a bit costly.
- Locking concept :: Locking happens at a row level for hybrid tables. This would enable higher concurrency execution of single row updates.
- Indexes :: Hybrid table uses B-Tree indexes both for primary key and secondary indexes. A good example is shown below see how table with secondary index defined skips the full table scan part from query profile.
5. Compatible with connectors :: Hybrid tables are compatible with all connectors just like others.
6. Composite key :: There are can be more than 1 column defined as primary key in hybrid table which would make it as composite key.
7. Scalability :: Hybrid tables would have some size limit and it would be in some terabytes. This is unlike other tables in Snowflake.
8. Pricing model :: It would be a bit costly considering the data would be stored in two forms(row and columnar) but as of today the pricing model is not yet finalized.
P.S. → Please note this is still under “PRIVATE PREVIEW” as of today as per Snowflake.
More reads & references:
