SiS Logging Framework: A Manual Approach

Credit: This image was created with the assistance of DALL·E 2.

Are you looking to log Streamlit-in-Snowflake application errors for your Snowflake account? Here’s how I approached solving the problem with a manual approach.

Assumptions

For this example walkthrough to work, the following assumptions have been made:

• The Snowflake account is hosted on either AWS or Azure and is not PrivateLink.
• The user has access to a role with the right privileges to create SiS applications.
• The user has access to a role that is the owner of one or more SiS applications.

How to create an SIS_LOGGING table

▹ 1. In Snowsight, create a new SQL worksheet and set the appropriate role, database, schema, and warehouse context.

▹ 2. In the SQL worksheet, create the SIS_LOGGING table:

Create a table object called SIS_LOGGING with 8 columns (1 timestamp, 7 string).

How to add logging to your SiS applications

▹ 1. In the Snowsight UI, switch the role context to the SiS app owner (SYSADMIN in this example) and navigate to the Streamlit Apps section.

▹ 2. In the Streamlit Apps section, create a new SiS app or click on an existing SiS app.

▹ 3. Set up the SiS app’s Python worksheet with the following code:

▹ 4. In the SiS app’s Python worksheet, add a get_logs(query, msg) function that executes an INSERT INTO statement for the SIS_LOGGING table:

Query text and message variables are passed into the get_logs() function to then be inserted into logs.

(Note: In this case, ‘EXAMPLE_SIS_APP’ would be manually changed to the SiS app’s name.)

▹ 5. Then, add an output_error(query, message) function to call the get_logs(query, message) function and then return an error flag in the SiS app.

▹ 6. Next, add a log_query(query) function that contains a try/except block to log successful and unsuccessful queries.

▹6a. Example where the base Snowpark exception class is used:

▹ 6b. Example where each Snowpark exception class is used (minus any that happen to be subclasses of one):

• snowflake.snowpark.exceptions.SnowparkDataframeReaderException is a subclass of “snowflake.snowpark.exceptions.SnowparkDataframeException
• snowflake.snowpark.exceptions.SnowparkInvalidObjectNameException is a subclass of “snowflake.snowpark.exceptions.SnowparkGeneralException
• snowflake.snowpark.exceptions.SnowparkInvalidObjectNameException is a subclass of “snowflake.snowpark.exceptions.SnowparkGeneralException”
• snowflake.snowpark.exceptions.SnowparkJoinException is a subclass of “snowflake.snowpark.exceptions.SnowparkDataframeException”
• snowflake.snowpark.exceptions.SnowparkPandasException is a subclass of “snowflake.snowpark.exceptions.SnowparkDataframeException”
• snowflake.snowpark.exceptions.SnowparkSQLInvalidIdException is a subclass of “snowflake.snowpark.exceptions.SnowparkSQLException
• snowflake.snowpark.exceptions.SnowparkSQLUnexpectedAliasException is a subclass of “snowflake.snowpark.exceptions.SnowparkSQLException”
• snowflake.snowpark.exceptions.SnowparkSessionException is a subclass of “snowflake.snowpark.exceptions.SnowparkServerException
• snowflake.snowpark.exceptions.SnowparkTableException is a subclass of “snowflake.snowpark.exceptions.SnowparkDataframeException”
• snowflake.snowpark.exceptions.SnowparkUploadFileException is a subclass of “snowflake.snowpark.exceptions.SnowparkServerException”
• snowflake.snowpark.exceptions.SnowparkUploadUdfFileException is a subclass of “snowflake.snowpark.exceptions.SnowparkServerException”

▹ 7. Now, come up with sample queries that you will pass into the log_query(query) function:

Sample query that I know will execute and return results

Sample query that I know will fail and return an error message

▹ 8. Build your main() function to include the queries you created during step 5 and the log_query() function where queries are being passed into the function.

The main() function contains two queries and calls the log_query() function for each of them.

▹ 9. Repeat step 6 until you have structured all of your queries like the above screenshot.

▹ 10. After putting it altogether, call the main() function.

The main() function is executed, which then executes the log_query() that kicks off other function calls.

How to leverage the SIS_LOGGING table

▹ 1. Assuming you have an SiS application with Snowpark Python queries that will execute as the SiS app runs top-bottom, in your current role, open and load the SiS application.

▹ 2. Then, switch between SiS applications where this approach has been set up.

▹ 3. After clicking through one or more SiS applications that your current role has access to, return to the SQL worksheet and run the following query:

SELECT * FROM STREAMLIT_DB.SIS.SIS_LOGGING;

Expectant Output:

There are 4 records/results, 2 of which are successful queries and 2 of which are queries that errored out.

▹ 4. To find out how many queries have errored out for the ANONYMOUS user, type out and execute the following query:

SELECT * FROM STREAMLIT_DB.SIS.SIS_LOGGING WHERE USER_NAME = ‘ANONYMOUS’ AND LOG_MSG NOT ILIKE ‘Success’;

Expectant Output:

There are 4 records, all of which are Snowpark SQL Exceptions.

Wrapping up

Thank you for reading my post! To recap, I shared my manual approach for SiS application logging. Since SiS has been generally available (GA) since December 2023, please feel free to share application logging approaches you’ve come up with in the comments below.

If you have any questions, please post them in the comments below or contact me on LinkedIn.

Code Repo: emmynguyen/sis-logging-framework (github.com)

Happy Streamlit-ing in Snowflake! ❄️