Snowflake Summit ’23 ❄️
New features and powerful toolset!
Lot of innovations announced at Summit, including advancements with generative AI and LLMs, Apache Iceberg, flexible programmability, application development, and much more.
Dynamic Tables
Dynamic Tables are the declarative components designed to orchestrate the data transformation pipeline. They significantly simplify data engineering at Snowflake and provide a reliable, cost-effective, and automated way to transform your data for consumption. Instead of defining the data transformation steps as a series of tasks and having to monitor dependencies and scheduling, you can simply define the end state of the transformation using PivotTables and leave complex pipeline management to Snowflake.
https://docs.snowflake.com/en/user-guide/dynamic-tables-about
Infer Schema and Schema Evolution
Infer Schema — Automatically detects the file metadata schema in a set of prepared data files containing semi-structured data and retrieves the column definitions, allowing the creation of tables based on the input files.
Schema Evolution: Semi-structured data tends to evolve over time. Systems that generate data add new columns to accommodate additional information, requiring subsequent tables to evolve accordingly. The structure of tables in Snowflake can automatically evolve to accommodate the structure of new data received from data sources. Snowflake supports the following:
- Automatically add new columns
- Automatically removes the NOT NULL constraint from missing columns in new data files.
https://docs.snowflake.com/pt/sql-reference/functions/infer_schema
Snowflake Iceberg Table
Iceberg Tables combines unique features in Snowflake with the open source projects Apache Iceberg and Apache Parquet to support the architecture of your choice. As part of the latest version of Iceberg, we added catalog support to the Iceberg project to ensure that engines outside of Snowflake can interoperate with Iceberg tables.
Snowflake Native Apps
A new way to build apps with the Snowflake native app framework. Today, we’re excited to bring the power of the Snowflake Native App Framework to developers around the world with a public preview on AWS. Developers can now start building and testing native Snowflake apps in their AWS account. Distribution and monetization features will be available in public preview on AWS later this year. Snowflake Native App Framework continues to be available in private preview on Google Cloud Platform and Azure.
https://quickstarts.snowflake.com/guide/getting_started_with_native_apps/index.html
Snowflake and LLM / Generative AI
The potential of Generative AI and the Large Language Model (LLM) for businesses is enormous. We’ve talked about this opportunity before, and at Summit’23 we announced several capabilities coming together to help our clients bring generative AI and LLM’s directly to their property data, all delivered through a single, secure platform.
https://quickstarts.snowflake.com/guide/frosty_llm_chatbot_on_streamlit_snowflake/#0
Snowpark Container
Snowpark Container Services is a fully managed container offering that helps you easily deploy, manage, and scale containerized applications without having to move data out of Snowflake. As a fully managed service, it comes with Snowflake security, configuration, and operational best practices built in.
https://docs.snowflake.com/en/LIMITEDACCESS/snowpark-containers/overview
