Using Cortex LLM with Native Apps for building Gen AI Applications

Published in

Snowflake Builders Blog: Data Engineers, App Developers, AI/ML, & Data Science

5 min readJun 28, 2024

Date : 20-June-2024

Introduction

In today’s data-driven world, organisations are constantly seeking innovative ways to extract insights from their data and make informed decisions. Snowflake has recently introduced two game-changing features: Snowflake Native App and Cortex LLM functions. In this blog post, we’ll explore how these two technologies can be combined to build a simple native application which leverages the power of Snowflake Cortex LLM functions. Before we look into the code let’s understand on a very high level what is Snowflake Native App and Cortex LLM functions.

What is Snowflake Native App Framework?

Snowflake Native App is a new way to build applications on top of Snowflake’s AI Data Cloud Platform. It allows developers to create custom data applications or containerized applications all running natively with in Snowflake AI Data Cloud Platform.

You can learn more about the Native App Framework from the following documentation.

About the Snowflake Native App Framework | Snowflake Documentation

Edit description

docs.snowflake.com

What are Cortex LLM Functions?

Snowflake Cortex gives you instant access to industry-leading large language models (LLMs) trained by researchers at companies like Mistral, Reka, Meta, and Google, including Snowflake Arctic, an open enterprise-grade model developed by Snowflake. Since these LLMs are fully hosted and managed by Snowflake, using them requires no additional setup. Your data stays within Snowflake, giving you the performance, scalability, and governance you expect. You can learn more on this from https://docs.snowflake.com/user-guide/snowflake-cortex/llm-functions

Using these two features you can build your Gen AI application and can easily distribute and monetise on them.

Using Cortex LLM Functions in Native Apps

The native app we are building will be reading a csv file using Snowpark Python APIs, load that data which has the raw conversations from a health insurance call centre into a Snowflake table. Later we will be using Snowflake Cortex LLM functions to extract various information from the raw conversation and also do call diarization. This is code is part of the solution we have build for Call Centre Analytics.

Native App Setup

I am not going to talk about the basics of what is native app and how do we create a native app. You can find some great documentations and details on creating a native app from the link mentioned in the above sections.

Github repo for the demo : — https://github.com/sfc-gh-praj/snowflake-na-cortex

Below are the artifacts required for this demo:

We need to include import privileges on snowflake DB in the manifest.yml file for cortex LLM functions to work in a native app.

Script to create the native app


use role accountadmin;

-- Create a role SPCS_PSE_ROLE 

GRANT CREATE APPLICATION PACKAGE ON ACCOUNT TO ROLE SPCS_PSE_ROLE;
GRANT CREATE APPLICATION ON ACCOUNT TO ROLE SPCS_PSE_ROLE;
GRANT CREATE INTEGRATION ON ACCOUNT TO ROLE SPCS_PSE_ROLE;

USE ROLE SPCS_PSE_ROLE;

-- SECURITY INTEGRATION is needed as we will create a streamlit app based on the data generated in this demo

CREATE SECURITY INTEGRATION IF NOT EXISTS streamlit_integration
  TYPE=oauth
  OAUTH_CLIENT=snowservices_ingress
  ENABLED=true;


CREATE DATABASE pr_call_centre_analytics_db;

CREATE STAGE pr_call_centre_analytics_db.PUBLIC.demo_stg
  ENCRYPTION = (TYPE = 'SNOWFLAKE_SSE');


CREATE APPLICATION PACKAGE pr_cortex_na_package;

USE DATABASE pr_cortex_na_package;

-- Upload all the files required for the native app in following folder
-- While uploading to na_stage, ensure you upload it to a folder with path streamlit/v01
LIST @pr_call_centre_analytics_db.public.na_stage/streamlit/v01/;

Below are the artifacts for the app which should be uploaded to the stage created above.

You can find artifacts required for the app here. After uploading the artifacts run the below commands.

-- Check the package version after uploading the files
 SHOW VERSIONS IN APPLICATION PACKAGE pr_cortex_na_package;

-- Adding version to the package
ALTER APPLICATION PACKAGE pr_cortex_na_package
  ADD VERSION "v1_0"
  USING @pr_call_centre_analytics_db.public.na_stage/streamlit/v01;

-- Creating App from the package
CREATE APPLICATION pr_call_centre_analytics_app
  FROM APPLICATION PACKAGE pr_cortex_na_package
  USING VERSION "v1_0";

DESC APPLICATION pr_call_centre_analytics_app;

-- Grant the below permission to the app. This is required to use cortex LLM functions
GRANT IMPORTED PRIVILEGES ON DATABASE SNOWFLAKE TO APPLICATION pr_call_centre_analytics_app

Without granting the IMPORTED PRIVILEGES ON DATABASE SNOWFLAKE , when you call the cortex LLM functions you will encounter the below error from the app:

Unknown user-defined function SNOWFLAKE.CORTEX.COMPLETE
in function LOAD_DATA with handler audioanalytics.init_data

After the app is installed and granted the required privileges, run the below command to load the data from csv and populate various tables.

 CALL pr_call_centre_analytics_app.code_schema.load_data();

Below is extract of the code which is doing the heavy lifting of extraction the information from raw conversations using the cortex LLM functions as seen below:

 # Referring the UDTF created in setup.sql
  udtf_table_function = table_function("code_schema.get_qa_prompt_oneshort")

  # Joining UDTF with ALL_CLAIMS_RAW DF
  oneshort_prompted_snowdf=generated_transcripts_df.join_table_function(udtf_table_function(F.col("conversation")).over(partition_by=["datetime","AUDIOFILE","CONVERSATION","PRESIGNED_URL_PATH","DURATION"]).alias('Oneshort_PROMPT'))
 
  # Pass the LLAMA prompt column to the COMPLETE function
  oneshort_info_extracted_snowdf = oneshort_prompted_snowdf.with_column("EXTRACTED_INFO", F.call_builtin("SNOWFLAKE.CORTEX.COMPLETE", F.lit('llama2-70b-chat'), F.col("ONESHORT_PROMPT")))

  oneshort_info_extracted_snowdf.write.mode('append').save_as_table("app_public.TRANSCRIPT_INFO_EXTRACTED_QA")

Source data which is a csv which has just the conversation :

After executing the SP , below is the output in the final table.

SELECT * FROM  pr_call_centre_analytics_app.app_public.AUDIO_CLAIMS_EXTRACTED_INFO;

Below are the other two tables which are populated as part the SP call.

SELECT * FROM  pr_call_centre_analytics_app.app_public.All_Claims_raw;

SELECT * FROM pr_call_centre_analytics_app.app_public.TRANSCRIPT_INFO_EXTRACTED_QA;

Once you have tested your application locally you can distribute this application with other accounts using private listing or put it in the market place.

Conclusion

This demo will show you how to utilize Cortex LLM functions within a native app. By combining these powerful features, you can create highly useful Gen-AI applications for your in-house teams or for your customers.

We will further enhance this app to include the Streamlit app which is part of the call centre analytics app. Watch out for the second part.