Snowflake Trail for Snowpark is Now Generally Available

We are excited to announce the General Availability (GA) of Snowflake Trail for Snowpark, a powerful suite of tools designed to help developers monitor, debug, and optimize their Snowpark workloads with enhanced logs, tracing, and metrics.

Back in October we launched a public preview of Snowpark’s integration with Trail, enabling developers to gain critical visibility into their Snowpark stored procedures, UDFs, and DataFrames. Now, with its GA release, Snowflake Trail is production-ready for Snowpark — bringing proven observability capabilities to Python, Java, and Scala workflows at scale.

Snowflake Trail a set of Snowflake capabilities for developers to better monitor, troubleshoot, debug and take actions on pipelines, apps, user code and compute utilizations.

What Snowflake Trail Offers

With Snowflake Trail, developers can leverage:

Enhanced Logs: Access detailed, structured logs specific to Snowpark workloads to diagnose and resolve issues quickly. These logs provide granular visibility into execution details, helping you pinpoint areas of improvement.

Explore logs in Snowsight. Navigate from Monitoring > Traces & Logs to find logs emitted in your Snowflake account.

Tracing Capabilities: Track the lifecycle of your Snowpark workloads from start to finish. Tracing allows you to correlate activities, uncover bottlenecks, and understand dependencies within your pipelines.

The Query Telemetry tab shows how data flows through your pipelines. In this example, a Python Stored Procedure is invoking a DataFrame query (save_as_table), which in turn calls two UDFs: compute and get_user_info.

Metrics for Optimization: Monitor performance metrics like execution time, memory usage, and input/output statistics to fine-tune your Snowpark applications. These insights empower data teams to deliver high-performing, efficient pipelines.

Click on a span to see the CPU and Memory consumed by the Python/Java process. In this example, we’re seeing the memory consumed by the two UDFs during the query.

Why It Matters

Snowflake Trail for Snowpark reflects our commitment to making Snowpark a comprehensive development platform. By incorporating observability as a first-class citizen in your Snowflake workflows, Snowflake Trail enables teams to:

• Minimize Downtime: Faster issue resolution leads to smoother operations and uninterrupted productivity. Consider setting up Alerts & Notifications on your logs, traces, or metrics to be notified for errors or warnings thrown from your code.
• Optimize Performance: Data-driven insights allow you to enhance the efficiency of your applications. Check out this deep-dive blog post from Snowflake’s Engineering team for guidance on using tracing to remove redundant queries, use asynchronous child procedures, parallelize Python processing, and more.
• Boost Developer Confidence: Developers can build with the assurance that Snowflake provides the tools they need to succeed at every stage of the development lifecycle. Snowflake’s other product areas will be onboarding to Trail soon — stay tuned for updates!

Try It Today

As of today, Snowflake Trail for Snowpark is generally available in all Snowflake regions. Every Snowflake account has a default Event Table already created so if you’re a developer, reach out to your account admin to get access. After that, follow the quickstart to learn how to add logs and traces to your Python code!

A demo of Snowpark’s logging, tracing, and metric functionality from the public preview announcement.

Integrate with Popular Developer Tools

In our ongoing commitment to enhance your data management capabilities, we are thrilled to showcase expanded integrations with leading monitoring and observability tools. Since its inception, the Event Table has used OpenTelemetry standards to allow seamless integration with popular developer tools.

Observe, Inc is built on Snowflake and natively integrates with Snowflake Trail to continuously analyze telemetry data spanning logs, metrics, traces, and events. Observe for Snowflake provides unified observability into Snowpark user-defined functions and containerized applications deployed in Snowpark Container Services. Learn more about Observe for Snowflake.

For those using Datadog, their integration supports comprehensive Snowpark monitoring, as detailed in their recent blog post. The power of OpenTelemetry means you can easily integrate Snowflake with a variety of partner tools, ensuring that your monitoring strategy can adapt to your evolving needs.

Learn more about Datadog’s integration with Snowflake.

You can now utilize Snowflake’s Grafana dashboard templates for an intuitive and customizable visualization of your telemetry. These templates are designed to enhance your monitoring experience, providing you with real-time insights into log exploration and Snowpark telemetry.

Additional resources

• A guide to Snowflake observability for Builders
• Learn more about Snowflake Event Tables
• Learn more about enabling logs, traces, and Snowpark metrics
• Learn more about Snowflake Trail
• Browse the documentation

Happy hacking!