Turning a Streamlit in Snowflake into a Native App — My April Month Goal

Published in

Snowflake Builders Blog: Data Engineers, App Developers, AI/ML, & Data Science

7 min readApr 29, 2024

As a part of my April Month Goal, I challenged myself to build something. What’s a better way than to try building a Snowflake Native App!

Journalling for my April Month Goal (generated with Playground.com)

Quite a few of my customers have taken the step from standalone Streamlit applications to Streamlit-in-Snowflake for internal apps since Streamlit in Snowflake went GA. Taking that Streamlit-in-Snowflake application and turning it into a Native App is an opportunity for those customers to manage those Streamlit applications centrally and distribute them across Snowflake accounts in their organization, as Brian Hess outlined in his article ‘Snowflake Native Apps — 3 Less Obvious Use Cases’ .

And then, once you are ready to embark on the journey of monetization (a little side hustle never hurt anyone) you can think of publishing your Native App to the Marketplace too.

Goal

I’ve used these QuickStart materials from Snowflake before to get familiar with the foundations of Native Apps, which are a good starting point for anyone starting their Native App development journey:

Now, with my April Month Goal, I decided to use that foundation to build something, more specifically to create a Native App from an existing Streamlit application. My goal is to:

Create a Native App from an existing Streamlit application
Share that Native App within my Snowflake Organization (and not on the Marketplace)

So I picked up my dream team’s previous Streamlit creation, FrostyGen, and turned it into a Native App so that other users in our Snowflake Sales Engineering Organization can generate synthetic data for demos or POCs too. If you are a Provider of a Native App, you can share your Native Application well beyond your own Organization by creating a Marketplace Listing. For development purposes or for sharing internal tools, you don’t necessarily need to publish your Native App to the Marketplace.

Creating my Native App

Below, I’ll walk you through the three main conceptual differences between building an app with Streamlit in Snowflake and Native App with Streamlit UI.

Create an Application Package with setup.sql and manifest.yml
Using reference(‘object’)
Defining privileges for the app_role and consumer

1️⃣ Create an Application Package with Setup.sql and Manifest.yml

Next to your streamlit.py file, you’ll need to create at least a setup.sql and manifest.yml file in order to create an Application Package for your Native App.

The setup script contains the creation of an application role, any tables that are used by the application itself (and hidden to the consumer unless explicit access is granted to the consumer), the Streamlit object based on your streamlit.py, procedures such as one to set, remove or clear references to tables by the consumer. Remember that your setup.sql is going to be rerun every time an application is upgraded, so use IF NOT EXISTS and OR REPLACE where needed.

The manifest file contains at least the version of your application, as well as artifacts that are distributed from this version of the package (such as readme, setup script and default streamlit) and runtime configuration for this version. Finally, any references and privileges required are defined in the manifest file too.

The Application Package is a Snowflake object that can be created using SQL or using the Snowsight UI. The package encapsulates all the data content, application logic, metadata and setup script required by your application. In the Application Package object, you will create a Stage in which you store all the necessary materials for your application (but at least the setup.sql and the manifest.yml).

2️⃣ Using reference(‘object’)

In a Native App you can use objects that you created in the setup.sql or objects from the consumer in the consumer account. The consumer must give the Native App access to any objects that it requires. This is done using References: in the Native App logic, you’ll use reference(‘object’) and the consumer is asked which object they want to give access to for ‘object’.

A reference can contain various object types, such as tables, views, but also warehouses and API integrations. The following list describes the object types that a reference can include and the privileges allowed for each object (from Snowflake Documentation):

TABLE : SELECT, INSERT, UPDATE, DELETE, TRUNCATE, REFERENCES
VIEW: SELECT, REFERENCES
EXTERNAL TABLE: SELECT, REFERENCES
FUNCTION: USAGE
PROCEDURE: USAGE
WAREHOUSE: MODIFY, MONITOR, USAGE, OPERATE
API INTEGRATION: USAGE

Snowflake provides an example Stored Procedure for handling a call back for the reference, as can be seen below, which calls a system function to perform a specific operation (add, remove or clear) on a reference that is passed as an argument to the stored procedure which was helpful.

CREATE APPLICATION ROLE app_admin;

CREATE OR ALTER VERSIONED SCHEMA config;
GRANT USAGE ON SCHEMA config TO APPLICATION ROLE app_admin;

CREATE PROCEDURE CONFIG.REGISTER_SINGLE_REFERENCE(ref_name STRING, operation STRING, ref_or_alias STRING)
  RETURNS STRING
  LANGUAGE SQL
  AS $$
    BEGIN
      CASE (operation)
        WHEN 'ADD' THEN
          SELECT SYSTEM$SET_REFERENCE(:ref_name, :ref_or_alias);
        WHEN 'REMOVE' THEN
          SELECT SYSTEM$REMOVE_REFERENCE(:ref_name);
        WHEN 'CLEAR' THEN
          SELECT SYSTEM$REMOVE_REFERENCE(:ref_name);
      ELSE
        RETURN 'unknown operation: ' || operation;
      END CASE;
      RETURN NULL;
    END;
  $$;

GRANT USAGE ON PROCEDURE CONFIG.REGISTER_SINGLE_REFERENCE(STRING, STRING, STRING)
  TO APPLICATION ROLE app_admin;

For example, in FrostyGen, a user can select a table to save the generated data to. The app cannot determine the name of the schema and object in the consumer account. Using Native App concepts, one can create the reference by calling the SYSTEM$REFERENCE system function and running the callback stored procedure passing the id of the reference.

The Snowflake Native App Framework provides a Python Permission SDK that allows providers to embed requests for the consumer in the user interface. Snowsight displays the access requests in the Security tab of the installed Snowflake Native App.

With this possibility of using reference(‘object’), you might find that you have to get used to thinking about objects as references to objects of which you don’t know the details during your Native App development.

3️⃣ Defining privileges

The last brain trainer for me was to think about all privileges needed in your (Streamlit) application.

Should the application be able to create objects outside of the Native App in the consumer account?
Should databases be created in the application itself that are visible to the consumer of the app?

For example, in FrostyGen I used an intermediary table to store the generated data before it was saved into a consumer table of choice. I wanted that intermediary table visible to the consumer in Snowsight. For this, you can create a table in your application and grant access to the app_role. If the application role is not granted permissions onto an object then you will not see the object in Snowsight. The application itself can still use the objects regardless.

An application can request global privileges (aside from references to objects) to a consumer. You can request the following global privileges in the consumer account (from Snowflake Documentation):

EXECUTE TASK
EXECUTE MANAGED TASK
CREATE WAREHOUSE
MANAGE WAREHOUSES
CREATE DATABASE

I didn’t use this in FrostyGen but I can imagine using this for creating a separate warehouse to run the data generation scripts, which as an added benefit gives the consumer complete transparency over how many credits running this Native App is consuming.

Sharing my Native App

Finally, my Native App was coming together, and I had my setup script, manifest and Streamlit file — it’s time to actually create the Application Package. I’m asked to choose a distribution option:

Distribute to accounts outside your organization
Distribute to accounts in your organization.

Choosing a Distribution Option for your App Package (from Snowflake UI)

For my project, I’m selecting Distribute to accounts in your organization, as I’m not intending to share this on the Marketplace. It also means that there’s no required security scan, which would be part of distributing the Native App outside your organization.
At this point you can share your Application Package through a listing, sharing it with Only Specified Consumers, or if you’d like to install the Application yourself (in your own provider account), you can install the application like this:

— ################################################################
— INSTALL THE APP IN THE ACCOUNT
— ################################################################

USE DATABASE NATIVE_APP_DB;
USE SCHEMA NATIVE_APP_SCHEMA;
USE WAREHOUSE NATIVE_APP_WH;

— This executes “setup.sql”; This is also what gets executed when installing the app

CREATE APPLICATION NATIVE_APP FROM application package NATIVE_APP_PACKAGE using version V1 patch 0;

You can now see your Native App listed under Apps in the UI. What’s a better reward to my April Challenge than to see my creation come to life like that?

Conclusion

In summary, the transition from standalone Streamlit apps to Native Apps in Snowflake offers centralized management and distribution across organizational accounts. By encapsulating logic in Application Packages and carefully defining privileges, you can foster collaboration and innovation in data analytics and application development.

Interested in building apps in Snowflake and going to 2024 Data Cloud Summit? Don’t miss out on these session:

Best Practices: How to Build Enterprise-Grade Snowflake Native Apps
Hands-On-Lab: Build a Snowflake Native App with Snowpark Container Services
Optimizing Efficiency in Autonomy Zones with the Snowflake Native App Framework (Bayer)
Don’t forget to register for Snowflake Dev Day on June 6th (Wednesday) too!

Snowflake Data Cloud Summit 2024 (https://www.snowflake.com/summit/)

Special thanks to Ranya Bellouki (Sales Engineer @ Snowflake) for reviewing before publication!