Open in app

Sign in

Write

Sign in

SiS Application Development: CI/CD Setup with GitHub Actions

Preface

Streamlit in Snowflake (SiS) applications can be created either through programmatic SQL commands or the Snowsight UI’s Streamlit section/tab. To manage edits or updates to the SiS applications, developers are often tied to the Snowsight UI’s edit mode capability where only one developer can make changes at a time. Exiting the edit mode capability after making changes also makes it difficult to retrieve past code snippets that may be useful for reverting the code to a previous version.

Problem

Many customers who are building or have built Streamlit in Snowflake (SiS) applications share similar sentiments about developer productivity and code maintainability that can be summed up in three words — manual, siloed, and difficult.

Prerequisites

To follow along, make sure the following has been set up:

  • A Snowflake account that has been enabled with the SiS feature
  • A service account user with a reasonable level of access (recommended)
  • A GitHub repository where the SiS application(s), SiS environment setup file (i.e. environment.yml), and the dependencies file (i.e. requirements.txt) can be stored (Note: This can either be a separate repository per app or a monorepo of apps — That’s a design decision to make)
  • The proper access role in the GitHub repository to create GitHub Secrets

Solution

To enable customers’ teams to automate part of the SiS application development process, reduce siloes, and simplify code maintainability, here is a solution:

# This workflow will build and push a Streamlit application to Snowflake when a commit is pushed to your default branch.

name: Create an SiS App from a file using GitHub actions/checkout

# Controls when the workflow will run
on:
  # Triggers the workflow on any branch or tag commit
  pull_request:
    branches:
      - "main"

  # Allows you to run this workflow manually from the Actions tab
  workflow_dispatch:

jobs:
  # This workflow contains a single job called "deploy"
  deploy:
    runs-on: ubuntu-latest

    env:
      SNOWSQL_ACCOUNT: ${{ secrets.SNOWFLAKE_ACCOUNT }}
      SNOWSQL_USER: ${{ secrets.SNOWFLAKE_USERNAME }}
      SNOWSQL_PWD: ${{ secrets.SNOWFLAKE_PASSWORD }}
      SNOWSQL_DATABASE: ${{ secrets.SNOWFLAKE_DATABASE }}
      SNOWSQL_SCHEMA: ${{ secrets.SNOWFLAKE_SCHEMA }}
      SNOWSQL_TABLE: ${{ secrets.SNOWFLAKE_TABLE }}
      SNOWSQL_ROLE: ${{ secrets.SNOWFLAKE_ROLE }}
      SNOWSQL_WAREHOUSE: ${{ secrets.SNOWFLAKE_WAREHOUSE }}
      SNOWSQL_STAGE: ${{ secrets.SNOWFLAKE_STAGE }}
      SIS_APP: ${{ secrets.SIS_APP }}
      GRANTEE_ROLE: ${{ secrets.GRANTEE_ROLE }}

    # Steps represent a sequence of tasks that will be executed as part of the job
    steps:
      # Checks-out your repository under $GITHUB_WORKSPACE, so your job can access it
      - uses: actions/checkout@v3

      - name: Install SnowSQL
        run: |
          curl -O https://sfc-repo.snowflakecomputing.com/snowsql/bootstrap/1.2/linux_x86_64/snowsql-1.2.30-linux_x86_64.bash
          SNOWSQL_DEST=~/bin SNOWSQL_LOGIN_SHELL=~/.profile bash snowsql-1.2.30-linux_x86_64.bash
      - name: Upload artifacts to Snowflake via SnowSQL put file command
        run: |
          ~/bin/snowsql -q "put file://$(pwd)/streamlit_app.py '@$SNOWSQL_DATABASE.$SNOWSQL_SCHEMA.$SNOWSQL_STAGE/' overwrite=true auto_compress=false" -o friendly=false
          ~/bin/snowsql -q "put file://$(pwd)/environment.yml '@$SNOWSQL_DATABASE.$SNOWSQL_SCHEMA.$SNOWSQL_STAGE/' overwrite=true auto_compress=false" -o friendly=false
      - name: Create Streamlit in Snowflake (SiS) App
        run: |
          ~/bin/snowsql -q "CREATE OR REPLACE STREAMLIT $SIS_APP ROOT_LOCATION='@$SNOWSQL_DATABASE.$SNOWSQL_SCHEMA.$SNOWSQL_STAGE' MAIN_FILE='streamlit_app.py' QUERY_WAREHOUSE=$SNOWSQL_WAREHOUSE COMMENT='Create SiS App From File Via GitHub Actions'" -o friendly=false
          # Add SQL commands that grant usage to the objects that are part of your SiS app
          ~/bin/snowsql -q "GRANT USAGE ON DATABASE $SNOWSQL_DATABASE TO ROLE $GRANTEE_ROLE" -o friendly=false
          ~/bin/snowsql -q "GRANT USAGE ON SCHEMA $SNOWSQL_SCHEMA TO ROLE $GRANTEE_ROLE
          ~/bin/snowsql -q "GRANT USAGE ON TABLE $SNOWSQL_TABLE TO ROLE $GRANTEE_ROLE
        
          # Add SQL commands that grant usage of the SiS app to a role or multiple roles
          ~/bin/snowsql -q "GRANT USAGE ON STREAMLIT $SIS_APP TO ROLE $GRANTEE_ROLE" -o friendly=false

The solution comprises of the following components:

  • GitHub Secrets — to securely store the service account user’s credentials
  • GitHub Actions — to set up an automated CI/CD workflow that overwrites/recreates the SiS application with the most recent code
  • SnowSQL — to connect the service account user to Snowflake and enable SQL query executions
  • Snowflake’s Snowsight UI — to test and validate that the SiS app was overwritten/recreated with the most recent code and the user experiences behaves as expected.

Solution Walkthrough

Part 1: GitHub and VS Code

  1. Clone the GitHub repository for your SiS application to your local desktop.
  2. Open the GitHub repository in either Visual Studio Code or an IDE of your choice.

Part 2: GitHub Secrets

  1. Using GitHub, navigate to the GitHub repository for the SiS application.
  2. In the GitHub repository, click on the Settings tab.
  3. Then, navigate to and click on the Secrets and variables section.
  4. From there, click on the Actions subsection.
  5. On the Actions secrets and variables page, click the New repository secret (green) button.
  6. Create and save a repository secret for each of the following: SNOWFLAKE_ACCOUNT, SNOWFLAKE_USERNAME, SNOWFLAKE_PASSWORD, SNOWFLAKE_DATABASE, SNOWFLAKE_SCHEMA, SNOWFLAKE_ROLE, SNOWFLAKE_WAREHOUSE, SNOWFLAKE_STAGE, SNOWFLAKE_TABLE, SIS_APP, and GRANTEE_ROLE. (Note: The secret value must be in quotes)

Part 3: GitHub Actions and VS Code

  1. In the GitHub repository, branch off main by creating a branch called gh-action.
  2. Picking up where we left off from Part 1, using your VS Code editor where you have opened the GitHub repository, create a .github folder.
  3. Then, right-click the .github folder to create a subfolder called workflows.
  4. In the workflows subfolder, create a create_sis_app YAML file.
  5. Copy and paste the following code to the YAML file:
# This workflow will build and push a Streamlit application to Snowflake when a commit is pushed to your default branch.

name: Create an SiS App from a file using GitHub actions/checkout

# Controls when the workflow will run
on:
  # Triggers the workflow on any branch or tag commit
  pull_request:
    branches:
      - "main"

  # Allows you to run this workflow manually from the Actions tab
  workflow_dispatch:

jobs:
  # This workflow contains a single job called "deploy"
  deploy:
    runs-on: ubuntu-latest

    env:
      SNOWSQL_ACCOUNT: ${{ secrets.SNOWFLAKE_ACCOUNT }}
      SNOWSQL_USER: ${{ secrets.SNOWFLAKE_USERNAME }}
      SNOWSQL_PWD: ${{ secrets.SNOWFLAKE_PASSWORD }}
      SNOWSQL_DATABASE: ${{ secrets.SNOWFLAKE_DATABASE }}
      SNOWSQL_SCHEMA: ${{ secrets.SNOWFLAKE_SCHEMA }}
      SNOWSQL_TABLE: ${{ secrets.SNOWFLAKE_TABLE }}
      SNOWSQL_ROLE: ${{ secrets.SNOWFLAKE_ROLE }}
      SNOWSQL_WAREHOUSE: ${{ secrets.SNOWFLAKE_WAREHOUSE }}
      SNOWSQL_STAGE: ${{ secrets.SNOWFLAKE_STAGE }}
      SIS_APP: ${{ secrets.SIS_APP }}
      GRANTEE_ROLE: ${{ secrets.GRANTEE_ROLE }}

    # Steps represent a sequence of tasks that will be executed as part of the job
    steps:
      # Checks-out your repository under $GITHUB_WORKSPACE, so your job can access it
      - uses: actions/checkout@v3

      - name: Install SnowSQL
        run: |
          curl -O https://sfc-repo.snowflakecomputing.com/snowsql/bootstrap/1.2/linux_x86_64/snowsql-1.2.30-linux_x86_64.bash
          SNOWSQL_DEST=~/bin SNOWSQL_LOGIN_SHELL=~/.profile bash snowsql-1.2.30-linux_x86_64.bash
      - name: Upload artifacts to Snowflake via SnowSQL put file command
        run: |
          ~/bin/snowsql -q "put file://$(pwd)/streamlit_app.py '@$SNOWSQL_DATABASE.$SNOWSQL_SCHEMA.$SNOWSQL_STAGE/' overwrite=true auto_compress=false" -o friendly=false
          ~/bin/snowsql -q "put file://$(pwd)/environment.yml '@$SNOWSQL_DATABASE.$SNOWSQL_SCHEMA.$SNOWSQL_STAGE/' overwrite=true auto_compress=false" -o friendly=false
      - name: Create Streamlit in Snowflake (SiS) App
        run: |
          ~/bin/snowsql -q "CREATE OR REPLACE STREAMLIT $SIS_APP ROOT_LOCATION='@$SNOWSQL_DATABASE.$SNOWSQL_SCHEMA.$SNOWSQL_STAGE' MAIN_FILE='streamlit_app.py' QUERY_WAREHOUSE=$SNOWSQL_WAREHOUSE COMMENT='Create SiS App From File Via GitHub Actions'" -o friendly=false
          # Add SQL commands that grant usage to the objects that are part of your SiS app
          ~/bin/snowsql -q "GRANT USAGE ON DATABASE $SNOWSQL_DATABASE TO ROLE $GRANTEE_ROLE" -o friendly=false
          ~/bin/snowsql -q "GRANT USAGE ON SCHEMA $SNOWSQL_SCHEMA TO ROLE $GRANTEE_ROLE
          ~/bin/snowsql -q "GRANT USAGE ON TABLE $SNOWSQL_TABLE TO ROLE $GRANTEE_ROLE
        
          # Add SQL commands that grant usage of the SiS app to a role or multiple roles
          ~/bin/snowsql -q "GRANT USAGE ON STREAMLIT $SIS_APP TO ROLE $GRANTEE_ROLE" -o friendly=false

6. Save the YAML file.

7. Commit and push the file to the GitHub repository.

Part 4: GitHub

  1. Create a pull request from gh-action into main.
  2. After review and approval from a peer (or peers), merge the pull request and delete the gh-action branch.

Part 5: Developer Flow with VS Code, GitHub, and Snowsight

  1. Using the already open GitHub repository in the VS Code window, branch off main by creating a branch called dev.
  2. In the dev branch, create or add an existing streamlit_app.py file.
  3. For a new streamlit_app.py file, developers can use the following boilerplate and alter as needed:
import streamlit as st

##############################################################################
# Local Snowpark Python session
# Instructions: Use this code block to perform local Streamlit app development

from snowflake.snowpark import Session

connection_parameters = {
    "account": st.secrets["snowflake"]["account"],
    "user": st.secrets["snowflake"]["username"],
    "password": st.secrets["snowflake"]["password"],
    "role": st.secrets["snowflake"]["role"],
    "warehouse": st.secrets["snowflake"]["warehouse"],
    "database": st.secrets["snowflake"]["database"],
    "schema": st.secrets["snowflake"]["schema"]
}

session = Session.builder.configs(connection_parameters).create()

##############################################################################
# SiS Snowpark Python session
# Instructions: Use this code block prior to creating/publishing the SiS app

# from snowflake.snowpark.context import get_active_session
# session = get_active_session()

4. Add additional code to the streamlit_app.py file and save the file.

5. Commit and push changes to the GitHub repository.

6. In GitHub, create a pull request from dev into main.

7. After review and approval from a peer (or peers), merge the pull request and delete the dev branch.

8. On your GitHub repository page, navigate to and click on the Actions tab.

9. In the Actions tab, due to the recently merged pull request, the create_sis_app GitHub Action should have been kicked off.

10. Click on the current GitHub Action run.

11. Click on the deploy button.

12. Follow along as the GitHub Action workflow processes each step and its result.

13. Once the GitHub Action workflow has finished, navigate to the Snowflake account and login with the service account user’s credentials.

14. After logging in, navigate to your name and role and ensure the service account user is using the right role to see the SiS app.

15. In Snowsight, navigate to and click on the Streamlit section.

16. In the Streamlit section, using the right role, the service account user should see the recently created SiS app.

17. At this point in time, feel free to test out the user interactions with the SiS app and continue the application development process as needed.

Summary

Thank you for reading my post! To recap, I provided a comprehensive walkthrough on how to set up a uni-directional, automated CI/CD approach that pushes application code changes from the GitHub repository to the SiS app(s) in Snowflake’s Snowsight UI.

As I have been working with SiS apps, I find that I prefer to do most of my application development via a local machine and then, once I’m ready, I will go into Snowflake’s Snowsight UI to interact with the published SiS app and test the functionality and user experience. For those who have been working with SiS apps the past few months, what is your preferred application development process?

If you have any questions, please post them in the comments below or contact me on LinkedIn.

Happy Streamlit-ing in Snowflake! 🎈

Snowflake
Streamlit
Github
DevOps
Python

--

--

Emily Nguyen
Snowflake Builders Blog: Data Engineers, App Developers, AI/ML, & Data Science

Written by Emily Nguyen

85 Followers
Writer for

Senior Cloud Solution Architect at Microsoft

Help

Status

About

Careers

Press

Blog

Privacy

Terms

Text to speech

Teams