Snowflake CLI Tutorial: Upload and Use Non-Snowflake Anaconda Channel Packages in Snowflake

Learn how to upload and use non-Snowflake Anaconda Channel packages in Snowflake.

Assumptions

• At the time of this article’s publication, Snowflake CLI is currently in public preview (PuPr), which means things are subject to change between public preview (PuPr) and general availability (GA).
• User uses an IDE (e.g. Visual Studio Code) of their choice.
• User has access to a Snowflake account. In the Snowflake account, the user has a role with the proper permissions to create and/or use a database, schema, and stage and can create objects like a UDF, UDTF, or SPROC inside of a schema.

Part 1: Virtual Environment Setup

1. Create a virtual environment in Python using venv:

python -m venv venv

2. Activate the virtual environment:

• Windows

venv\Scripts\activate.bat

• Mac

source venv/bin/activate

3. Install the latest version of the Snowflake Developer CLI:

pip install snowflake-cli-labs

Part 2: View the Snowflake CLI Menu

1. Execute the following command:

snow --help

CLI Output:

Part 3: Manage Connections to Snowflake

1. Execute the following command:

snow connection --help

CLI Output:

2. Then, add a connection to the configuration file:

snow connection add

CLI Output:

3. Now, test the Snowflake account connection:

snow connection test --connection trial

CLI Output:

Part 4: Use Snowflake CLI to search for, create, and upload a PyPi package to a Snowflake account

See menu options for `snow snowpark`

1. Execute the following command:

snow snowpark --help

CLI Output:

See menu options for `snow snowpark package`

1. Execute the following command:

snow snowpark package --help

CLI Output:

See if a package is available on the Snowflake Anaconda channel

1. Execute the following command:

snow snowpark package lookup faker-biology

CLI Output:

See if a package is available on the PyPi channel

1. Execute the following command:

snow snowpark package lookup faker-biology --pypi-download

CLI Output:

Create a ZIP file for a Python package available in the PyPi channel

1. Execute the following command:

snow snowpark package create faker-biology --pypi-download

CLI Output:

Upload the faker-biology ZIP file to a Snowflake stage

1. Execute the following command:

• If the default connection has been set up, execute the following command:

snow snowpark package upload -f faker-biology.zip -s <stage_name>

• If the default connection has not been set up, execute the following command:

snow snowpark package upload -f faker-biology.zip --connection <connection_name> -s <stage_name>

CLI Output:

Check stage to see if the ZIP file exists in the Snowflake stage

1. Execute the following command:

• If the default connection has been set up, execute the following command:

snow object stage list <stage_name>

• If the default connection has not been set up, execute the following command:

snow object stage list <stage_name> --connection <connection_name>

2. To double check, using the Snowsight UI, navigate to the DEMO.DEMO.PYTHON_PACKAGES stage and check if the faker-biology.zipexists in the stage.

Part 5: Snowpark Object Creation

SQL

1. In Snowsight, open a new SQL worksheet and set the proper contexts.
2. In the SQL worksheet, create a Python user-defined function (UDF) called faker-biology.

CREATE OR REPLACE FUNCTION faker_biology()
RETURNS string
LANGUAGE python
RUNTIME_VERSION = '3.11'
HANDLER = 'faker_biology'
PACKAGES = ('six', 'faker')
IMPORTS = ('@PYTHON_PACKAGES/faker-biology.zip')
as
$$
from faker import Faker
from faker_biology.physiology import CellType, Organ, Organelle

def faker_biology():

    fake = Faker()
    
    fake.add_provider(Organ)
    
    fake.organ()
    # Sublingual glands

    return f"Sublingual glands: {fake.organ()}"
$$;

3. In the same SQL worksheet, call the Python UDF.

SELECT faker_biology();

Python UDF Output:

Summary

Thank you for reading my post! To recap, I provided a step-by-step walkthrough of how I used the Snowflake CLI to search for, create, and upload a Python package that’s in the PyPi channel but not in the Snowflake Anaconda channel. From there, I created a Python UDF that imports the non-Snowflake Anaconda channel Python package with IMPORTSand sets up dependencies available in the Snowflake Anaconda channel with PACKAGES.

While Snowflake CLI enables the creation of ZIP files from the PyPi channel, the CLI does not always stage everything properly, does not currently support Python packages with C dependencies, and the CLI output message is not always accurate in terms of what dependencies are needed. For Python packages that involve API endpoints, external access will need to be configured for the object, but this is not necessarily the most intuitive because the CLI output does not account for this. During object creation, should any errors occur, review the Python package’s documentation and the errors carefully to identify additional dependencies that need to be included.

In this specific context where faker-biologyis being used, six was the only dependency that the CLI identified, but, while building the Python UDF, I identified faker as another dependency I needed to include in my PACKAGES.

If you have any questions, please post them in the comments below or contact me on LinkedIn.