Snowpark Container Services — A Tech Primer
Updated 12/20/2023
Introduction
At our annual Summit 2023 user conference the last week of June, Snowflake announced a new product feature, Snowpark Container Services. Snowpark Container Services is a fully managed container offering that allows you to easily deploy, manage, and scale containerized services, jobs, and functions, all within the security and governance boundaries of Snowflake, and requiring zero data movement. As a fully managed service, Snowpark Container Services comes with Snowflake’s native security, RBAC support, and built-in configuration and operational best-practices.
Snowpark Container Services are fully integrated with both Snowflake features and third-party tools, such as Snowflake Virtual Warehouses and Docker, allowing teams to focus on building data applications, and not building or managing infrastructure. Just like all things Snowflake, this managed service allows you to run and scale your container workloads across regions and clouds without the additional complexity of managing a control plane, worker nodes, and also while having quick and easy access to your Snowflake data.
How are customers and partners using Snowpark Container Services today? Containerized services on Snowflake open up the opportunity to host and run long-running services, like front-end web applications, all natively within your Snowflake environment. Customers are now running GPU-enabled machine learning and AI workloads, such as GPU-accelerated model training and open-source Large Language Models (LLMs) as jobs and as service functions, including fine-tuning of these LLMs on your own Snowflake data, without having to move the data to external compute infrastructure. Snowpark Container Services are an excellent path for deploying applications and services that are tightly coupled to the Data Cloud.
With all of the excitement and buzz around Snowpark Container Services, we want to provide a detailed technical explainer that outlines the different objects and components that make up a Snowpark Container Service, as well as outline the functional models that are supported and common patterns of usage.
Snowpark Container Services: Object Model
The introduction of Snowpark Container Services on Snowflake includes the incorporation of new object types and constructs to the Snowflake platform, namely: images, image registry, image repositories, compute pools, specification files, services, and jobs.
Images
The notion of an image in Snowpark Container Services is equivalent to that of an OCI-compliant image; that is, an image is a file used to execute code in an OCI-compliant container runtime such as Docker. Its build process and execution commands are specified in a file such as a Dockerfile, and images are built into containers and then ultimately pushed to an image repository for execution in Snowpark Container Services.
Image Registry
Image registry in Snowflake is an OCIv2 compliant service for storing OCI-compliant container images. The image registry has a unique hostname which allows OCI clients to access an image registry using REST API calls. Within a registry, there is a notion of image repositories which is a storage unit within that image registry service.
Image Repositories
Image repositories are the landing zone for built containers. In Snowflake, an image repository is effectively a storage mechanism for built images which is mapped inside of a Snowflake database and schema with a corresponding FQDN:
CREATE IMAGE REPOSITORY <db>.<schema>.<image_repo_name>;The corresponding tagged images in a Snowflake image repository are used in the specification file for a service to define the source image of each container comprising the service:
image: my-org-my-account.registry.snowflakecomputing.com/<db>/<schema>/<image_repo_name>/my_image:latestCompute Pools
A service in Snowpark Container Services needs compute resources to actually run, and a compute pool is a collection of virtual machines or nodes on which those services run. Compute pools are created and managed similarly to virtual warehouses on Snowflake, including suspension/resumption, but give you the flexibility to specify different machine types and min/max node counts to scale to on any one compute pool. Snowflake handles this auto-scaling of the compute pool based on the configured node counts, the number of instances of a particular service running on a compute pool, and the number of different services running on that compute pool. For services with multiple instances running across one or several nodes in a compute pool, Snowflake automatically load balances incoming traffic to that service across instances.
The Public Preview of Snowpark Container Services supports standard CPU, high-memory, and GPU instance types in a variety of sizes. Like Snowflake virtual warehouses, compute pools are billed to the customer account on a per-second basis, using a standard Snowflake credits/hour rate depending on the compute pool’s instance type, size, and number of nodes in the pool. For more detail about the exact specifications and sizes of node types available to compute pools, contact your Snowflake representative.
Specification Files
Specification files are YAML files that provide the definition for a service. The specification file defines the container(s) that comprise the service, the corresponding image(s) in the image registry that back them, and relevant environment variables, endpoint definitions, volume mounts, and more. For Kubernetes users, Snowpark Container Services specification files are similar to a Kubernetes deployment manifest. The specification file for a particular service is then uploaded to a Snowflake stage and then the corresponding staged file is referenced in the CREATE SERVICE statement.
Services
Services are the actual applications, container(s), etc. that run in Snowpark Container Services and are created from specification files to execute on compute pools. Users can perform standard CRUD operations on services, and system functions allow users to retrieve information about the service runtime status, local logs from the container service, and more. Services themselves do not incur any cost, as Snowflake only bills customers for active compute pools and corresponding storage of images. Services and the corresponding containers that comprise them support three different design models for deployment, which are described in detail in the subsequent section.
Snowpark Container Services Design Models
The initial release of Snowpark Container Services supports three core service design models, on top of which complex data intensive applications can be built: jobs, functions, and services:
- Jobs are containerized tasks that are executed, perform some sort of operation, and run to completion (e.g. GPU-accelerated machine learning model training).
- Services are long-running applications that are accessible via combinations of internal and external endpoints (e.g. web UI, REST API, Kafka Connect cluster).
- Service Functions are callable bits of computation to which data is passed, a response is produced and returned (e.g. prompting OSS LLMs). Importantly, with service functions that are called on Snowflake data, the data never actually leaves Snowflake.
We will look at each of these service models in more detail:
Jobs
Jobs in Snowpark Container Services are containerized tasks that are executed, and run to completion over some bounded time domain. At invocation via an EXECUTE SERVICE command (triggered by a user command, or through an orchestration framework), the service spins up on the compute pool, executes, and spins down. They are in some ways similar to Snowflake tasks, with of course the exception that they execute on compute pools and give you the flexibility to run jobs that could not otherwise be supported in a SQL or Snowpark Python/Java/Javascript task.
Example: GPU-accelerated model training
Snowpark Container Services can run on GPU-powered compute pools, including both A10 and A100 NVIDIA GPUs, which enable a variety of ML and LLM-specific workloads. In the context of GPU-enabled machine learning, model training is a great example of a Job service. This is a script to perform model training using GPU-acceleration, CUDA drivers, and compatible open source libraries. It is triggered for execution (automatically, through a Snowflake task, or manually via a user EXECUTE SERVICE statement), and runs to completion.
Services
Services in Snowpark Container Services are exactly that- long-running services which exist to provide some sort of accessible functionality, potentially through an externally accessible UI, programmatic access, or to perform some sort of other persistently running task. This model mirrors a common cloud microservices architecture, and supports hosting of services such as Jupyter notebooks, REST APIs, or even a Kafka connect cluster that reaches outside of Snowflake to stream/ingest data into Snowflake.
Examples: Hosted Jupyter notebooks, REST APIs, Kafka Connect
There are two sub-models within the notion of long-running Services as Snowpark Container Services. These are (1) services that users reach into from outside of snowflake (e.g. a User Interface such as Hosted Jupyter Notebooks), and (2) services that reach out of Snowflake (e.g. Kafka Connect). Of course, certain services might fit both of these models. In either case, these are persistent services that are “up and running”, and potentially made accessible through public endpoints. These long-running services additionally support both UI and programmatic authentication and access.
As an example, to host Jupyterlab in Snowflake running as a Snowpark Container Service, you need the following specification yaml:
spec:
container:
- name: jupyterlab
image: <org>-<account>.registry.snowflakecomputing.com/<my-db>/<my-schema>/<my-image-repo>/jupyter/datascience-notebook:latest
env:
JUPYTER_ENABLE_LAB: "yes"
JUPYTER_TOKEN: "docker"
endpoints:
- name: "ui"
port: 8888
public: trueThe service is then created via a simple SQL statement:
create compute pool if not exists JUPYTERLAB_STANDARD_2_CPU
min_nodes = 1
max_nodes = 1
instance_family = standard_2
auto_resume = true;
create service jupyterlab
min_instances=1
max_instances=1
compute_pool=JUPYTERLAB_STANDARD_2_CPU
spec=@<MY-DB>.<MY-SCHEMA>.service_yaml/jupyterlab_spec.yaml;From there, we have Jupyterlab running securely inside of Snowflake, accessible to users via browser:
Service Functions
Service Functions in Snowpark Container Services are services that take in data as an input parameter, for example in the context of table columns in a SQL query, and run some custom processing against that data, returning a result. The framework for service functions follow the same structure as Snowflake external functions in terms of data patterns, with of course the difference that with Snowpark Container Services, the data never actually leaves Snowflake. A function-as-a-service requires two components: the service, which specifies the logic of how your function processes input data, implemented via a REST interface that responds to POST requests, and then a configured SQL function which references the specific service and endpoint. For example, an open-source LLM can be loaded into a service on GPU compute with a configured REST endpoint. A function can then be defined which specifies input string (such as a user prompt), calls the service at the endpoint, and then returns the generated result. This can also support running arbitrary functional code that may not be otherwise supported in Snowflake using SQL or Snowpark User-Defined Functions (UDFs).
Example: OSS LLMs (e.g. Dolly, LLaMa, E5)
Service Functions provide a mechanism to host arbitrary functional code to process data in a Snowpark Container Service. These service functions use the Snowflake external function structures to expose named SQL functions that can be called in a query, table definition, and more. Importantly, even though service functions follow the external function ICD in terms of how the virtual warehouse structures and passes data to the Snowpark Container Service, this all occurs within the Snowflake network boundary (i.e. no data movement- maintaining complete control, governance, and security of your data). Examples of this are hosting open-source LLMs to accept user prompts about your Snowflake data, as well as arbitrary data processing code (in any language).
Check out Reka running as a Native App inside of Snowpark Container Services as an example.
Communicating with Services
In thinking about a full application stack running as a service(s) on Snowpark Container Services, an obvious question is: how do I communicate with different containers and/or services? Web applications need to be accessible by users (potentially over the internet); those same web UIs may need to communicate with REST API containers running inside the same service; functions may need to reach outside of Snowflake, too. Service communication is supported in three different ways: ingress (reaching into a service from outside), egress (reaching outside of Snowflake from inside a service), and inter-service communication (containers communicating with each other without traversing the open internet).
Ingress
Services running in Snowpark Container Services can optionally support ingress, that is, the ability for users, applications, and services from outside of Snowflake reaching into and interacting with the service running inside of Snowflake. This is done by optionally configuring endpoints (i.e. ports) as public in the service specification file. For example, if you want to run Jupyter notebooks as a hosted service that can be accessed by users, you simply specify port 8888 as public: true in your specification file. This will generate a publicly accessible URL for the corresponding endpoint which users can use to reach into the service, login, and connect to it. These public endpoints can also be accessed programmatically through oauth tokens, which supports hosting REST API services that can be accessed by other services running outside of Snowflake.
Egress
Services running in Snowpark Container Services can also reach out to other applications, endpoints, or the public internet (egress outside of Snowflake). This is configurable via network policies and External Access Integrations, providing fine-grained security controls over what external resources services can communicate with.
Inter-service Communication
Individual endpoints (even that are non-public) in a service can be communicated with by other services (or between containers in the same service) using a generated DNS name and the corresponding port number without having to traverse the public internet. When the service is created, Snowflake generates a DNS name of the format:
<service-name>.<schema>.<db>.snowflakecomputing.internalwhich can be used for inter-service communication. Note that this is a standardized DNS name, whereas public endpoints for ingress are randomly generated and must be published; the result is that programmatic discovery of services for interservice communication is not necessary, since service endpoints will be exposed at known DNS locations.
Partner Applications
During the product keynote at Snowflake Summit 2023, we unveiled Snowpark Container Services by showing more than 10 Snowflake partner tools running natively inside of Snowflake. For more info about our Design and Launch partners, check out the official Snowflake blog announcement on Snowpark Container Services.
You can also check out a number of these partner tools running in Snowpark Container Services on our YouTube channel:
You can always find more on the Snowflake Developers YouTube channel.
Conclusion
With Snowpark Container Services, Snowflake customers are now able to host and run arbitrary containerized data-intensive applications, colocated with their data, and utilizing a fully managed service.
To get started with Snowpark Container Services, check out our Intro to Snowpark Container Services Quickstart. For more information about Snowpark Container Services, check out the documentation and contact your Snowflake account team or sales representative.
Watch the full announcement from the Opening Keynote at Snowflake Summit 2023!
