Highlights of Snowflake Summit 2021

Snowflake Summit 2021

What is Snowflake Summit?

Snowflake Summit is a virtual user conference from Snowflake’s technology experts, partners, and customers, presenting on every facet of the data cloud. Data is everywhere and it holds the key to unlocking your organization’s success now and in the future. By attending Snowflake Summit, you will learn how to use the Data Cloud to unify, analyze, and share data previously out of your reach for the impact you have ever only imagined.

Don’t worry if you have missed this year’s Summit, I have tried to summarize the important announcements. Look out for next year's summit and it's going to be really exciting to see an emerging Data Cloud platform that changes the way we deal with data.

Theme — Data Together Now

Snowflake Summit 2021 was driven by the theme Data Together Now which promised to inspire business and technical leaders, data scientists and engineers, data analysts, and application developers, to lead their organizations to a data-driven future. The first-ever summit started in 2019 and the 3rd annual summit on June 8th -10th this year came with a variety of announcements and new features as follows.

What's new in Snowflake Data Cloud?

1.Connected Industries

Snowflake powers a new type of collaboration across industries by simplifying access to high-value data. Snowflake Data Marketplace is planning to expand its data providers and consumers in a considerable number. This initiative from Snowflake will allow customers to seamlessly secure and share data across clouds and regions. Some of the new features announced in this category are

1. Try-Before-You-Buy allows data consumers to try a sample data before they buy and access it. This feature in Snowflake’s Data Market place will enable self Service data offering to reach greater customers with lesser cost and time to market.
2. Buy Data Instantly enables the purchase of any data available in Snowflake Marketplace instantly and pay for the data only you use.
3. ServiceNow Integration

Snowflake is now providing a native connector to ServiceNow which enables customers to replicate full-fidelity data from ServiceNow virtually at any scale. Customers could set up, configure and execute the connector using simple SQL.

2.Global Governance

Global Governance of Snowflake promises data organizations with robust control over their data with policies applied across all data and roles. It ensures trust and protects data while maintaining its value for better business results. The two major updates in this category are,

1. Data Classification provides the capability to automatically detect personally identifiable information (PII) in a given dataset. This feature is currently in Private preview.
2. Object Tagging leverages Snowflake’s tagging framework to annotate the data. Assign custom tags to resources like compute clusters, snowpipe, and any objects like Columns, Tables, Views, External Tables & Materialized views. This will enable us to easily track 1000s of objects for reporting and access controls.
3. Anonymized views is another interesting feature which can be used to protect privacy and identity in a dataset, while still retaining its analytical value. This enables us to secure sensitive data by policy-based access controls and by setting a level of protection to meet your internal data policies.

Global Governance is a massive boost for the users in leveraging the power of data while ensuring the protection of personal data.

3.Platform Optimization

Snowflake’s Platform Optimization has ensured the current performance engine to adapt more efficiently to different workloads like Data Engineering, Data Science, Ad-Hoc Analysis, BI/Dashboards, Data Sharing etc.

1. Changed Storage Economics: Recent changes to Snowflake’s data storage resulted in better compression, and reduced storage costs. This enhanced storage technology is available transparently, with no user action, no configuration changes, and no application or query changes, and this is already rolled out to all Snowflake customers and will apply to newly written data.
2. Support for Interactive Use Cases: This Snowflake’s new set of updates improves concurrency, latency, and throughput. By caching metadata and parallelizing compilation QPS throughput is increased by 6x. By enabling in-memory query scheduling, the average query duration is decreased by 8x. Clusters are executed in parallel which decreases the query duration by 16x. This feature is currently in Private Preview.
3. Usage Dashboard: This new dashboard feature in Snowflake helps customers to better understand usage and costs across the platform, making it easy to manage all accounts across the entire organization. This feature is in Public Preview.
4. Query acceleration Service: This update in Snowflake accelerates large-scale exploratory analysis, data science, and other heavy workloads. This feature helps long-running queries that scan a huge amount of data by speeding up the performance without changing the warehouse size. It helps to scale workloads more than the largest warehouse size. This feature is in Private Preview.

4.Data Programmability

Data Programmability features introduced by Snowflake will simplify and automate pipelines with a focus on data rather than infrastructure. Gives you ultimate flexibility in preference of Programming languages and model. The exciting new features introduced are,

1. Snowpark: This enhances Snowflake’s developer experience by allowing data engineers, data scientists, and developers to build data workloads using their preferred language and familiar programming concepts, and then execute them directly within Snowflake. Easily complete and debug data pipelines with familiar constructs such as DataFrames and third-party libraries. With initial support for Java and Scala, this feature is currently in Private Preview.
2. Java UDF: With Java user-defined functions, data engineers can bring their custom code and business logic to Snowflake for better performance and expanded use case capabilities while reducing complexity. It allows building functionality into Snowflake using popular Java libraries. This feature is currently in Private Preview.
3. Unstructured data Management: Snowflake provides a whole new capability to store, access, govern, process and share unstructured data with fine-grained governance of files as well as metadata. This unlocks the ability to retrieve any dataset with simple SQL commands. This feature is currently in Private Preview.
4. SQL API: This enables custom applications to call Snowflake directly through a REST API, without the need for client-side drivers, thus reducing the complexity and administration overheads. This feature is currently in Public Preview.
5. Automatic Schema Detection: This is an interesting feature that enables fast and automatic data onboarding with Snowflake. When loading new semi-structured files to internal or external stages, snowflake can automatically create a new table schema by pointing it to the files in the stage both internal and external (Azure Blob, AWS S3, Google Storage). This enables data programmers to directly query schematized tables which increases their productivity and saves a huge amount of time in onboarding new files containing new columns to existing tables. This feature is supported for all major files types like Parquet, Avro & ORC.
6. Snowflake Scripting: This enables you to create and save complex SQL flows including SQL Stored Procedures. This provides the ability to write standalone SQL codes with parameters and expressions.

5.Powered By Snowflake

Powered by Snowflake is a new Snowflake partner program launched to help data companies and data programmers build, market, and operate applications in the Snowflake data cloud. Customers will have the below benefits by enrolling in this program.

1. Get access to Snowflake resources & technical expert guidance
2. Access to workshops to design the right architecture
3. Grow your business partnering with Snowflake
4. Get professional support to optimize data performance.

Final Thoughts

I hope the above content gives you the most important updates of the Snowflake Summit 2021. I tried to cover the high-level overview of updates and features announced.

What next in Snowflake Data Cloud?

There are lots more exciting features coming up before the next year. Getting Python into Snowpark is the one feature of a high priority now and I listened that we can expect it to be available in the 2nd half of 2021.

References

Summit 2021