Ollion’s Notes from Snowflake Summit Part 2: Much Improved Monitoring with Trail
At the recent Snowflake Summit, one of the most interesting sessions was on Snowflake Trail, a service that provides insights into application and pipeline performance without the need for agent installation, setup, or data export tasks. Here’s a detailed look at what Snowflake Trail offers and why it’s good for Snowflake users.
What is Snowflake Trail?
Snowflake Trail is a monitoring service that enhances your Snowflake environment by providing detailed telemetry and observability. This tool significantly reduces the time to detect issues and the time to resolution, making it an invaluable tool for data teams.
Key Features of Snowflake Trail
- Effortless Telemetry: Activating telemetry in Snowflake Trail is straightforward. Users just need to turn it on and start benefiting from its capabilities.
- Event Tables, Alerts, and Notifications: Stay informed about critical events and receive timely alerts and notifications.
- Pipeline Observability: Gain insights into your data pipelines with comprehensive observability features. Monitor and debug task histories, task graphs, and dynamic tables with ease even through nested Snowpark stored procedures and functions
- Application Observability: Monitor and troubleshoot applications built on Snowflake including Snowpark stored procedures, UDFs, Streamlit, Container Services, Native Apps, and Notebooks. You can setup Event Tables, turn on auto-instrumented telemetry, and just turn on logging, tracing and metrics.
- Data Quality Monitoring: Ensure data observability with robust data quality monitoring capabilities.
How to Enable Snowflake Trail
To enable Snowflake Trail for your database, you need to adjust the log and trace levels. Execute the following commands:
ALTER DATABASE SET LOG_LEVEL = ERROR;
ALTER DATABASE SET TRACE_LEVEL = ALWAYS;
ALTER DATABASE SET METRIC_LEVEL = ALL;Once enabled, a new “Query Telemetry” tab will appear in the query history, in addition to the existing details and profile tabs. This new tab includes distributed trace telemetry, providing deeper insights into query performance and issues.
Tracing Support with OpenTelemetry
Snowflake Trail supports tracing in Snowsight with OpenTelemetry compatibility. This feature is crucial for troubleshooting Snowpark jobs more efficiently. Users can leverage auto-instrumented spans for stored procedures, functions, and notebooks. Additional benefits include:
- Trace Context Propagation: Propagate trace contexts seamlessly across different services.
- Related Logs and Metrics: Access related logs and metrics for comprehensive monitoring.
- Parallelized Execution of UDFs: Visualize the parallel execution of UDFs in Snowpark.
Debugging and Bottleneck Identification
Snowflake Trail excels in debugging by allowing users to pinpoint specific child objects that are failing. This capability is particularly useful for finding bottlenecks in monolithic code and nested stored procedures. Users can search through trace logs and integrate with various downstream tools like Datadog, Grafana, Monte Carlo, and Observe.
Conclusion
Snowflake Trail is a powerful addition to the Snowflake ecosystem, providing users with effortless telemetry and advanced observability. Whether you’re looking to monitor data quality, troubleshoot applications, or gain insights into your data pipelines, Snowflake Trail offers the tools and features to make it happen.
Also, check out Part 1 of the series focused on the new Summit Announcements!
Or, even better, go to Part 3 to read about the services for Snowflake developers.
About Ollion
At Ollion, we have been a proud Snowflake Service Partner for almost a decade. Our mission is to connect companies and capabilities worldwide, helping ambitious organizations achieve game-changing breakthroughs without losing sight of the people impacted. We offer a unique point of view as an independent, straightforward partner backed by a global team of client partners, sales, engineering, delivery, and more.
Let me know if you attended and want to talk more about Snowflake Summit 2024!