Snowflake Native App To Estimate The Price Of Your Property — Using a ML model trained With Consumer Provided Data.

Snowflake Native Applications have revolutionised the way application providers build, distribute and deploy applications for their customers. Providers are not only being able to implement state of the art applications while protecting their IP, but they are able to help consumers use their application to generate actionable insights while driving business growth.

In the first blog I have demonstrated how the providers can build ML models and ship them to their consumers for performing online and batch inferencing.

However during several discussions with partners it was clear they are not happy with just building ML models as a provider and shipping them to their consumers. The reason being every consumer is unique with different ways of managing their business. Hence it calls for a requirement to be able to create application that can train and build ML models at the consumer end with consumer provided data.

This blog is a continuation to the first blog. Most of the code units that are used in this part are updates to the previous version. Hence it is highly recommended to review the first part of the blog before continuing with this one.

The app in the first blog has been updated to build the ML model at the consumer end using the consumer provided dataset. To allow the Snowflake Native App to access existing objects outside the APPLICATION object, the Snowflake Native App Framework provides references and callback stored procedures that allow the customer to specify the name and schema for an object and enable access to the object. Here is the link to the documentation for your to deep dive.

Before we dive into the code snippets let us review the revised application flow. The provider is responsible for the application logic alone. Rest of the steps like model training and inferencing are all getting exercised in the consumer account.

The revised project structure shows the program files and highlights their functionalities. Points to note, the model is not prebuilt and hence not a part of the application package. It is built in the consumer account.

• “deploy_app_to_provider_account.py” -> If you follow the native app documentation or the developer guide, you will see that there are certain number steps to be followed during the process of application deployment. Now, as a developer we love automation and reduce our checkpoints. This is one such initiative where I have automated the application deployment every time there is a change in the app logic or the model is rebuilt by the provider when the source data is refreshed.

Here is an output snapshot of the deployment script that makes your life easy. :

One of the key code unit is the manifest.yml file. This file has the reference to the consumer data object that is used during the model training.

manifest_version: 1
artifacts:
  setup_script: scripts/setup.sql
  readme: readme.md


references:
  - historical_housing_data:
      label: "historical_housing_data" 
      description: "Select table"
      privileges:
        - SELECT
      object_type: Table 
      multi_valued: false 
      register_callback: core.update_reference

The updated application when installed as two modules:
1. The model build. &
2. The model inference

Disclaimer on the choice of the ML model and its efficiency : Please note that the objective of this exercise is not to build a model that is super efficient to estimate the price of the property. Hence the choice of the variables or the choice of the model or the model efficiency may not have been perfect. This objective was to showcase how a native application can build a ML model at the consumer account using consumer data.

You can use this application to build multiple models trained on different datasets. Once the model building is done, you can use the most efficient model for online as well as batch inferencing. The app demonstrates how online inferencing can be implemented. This can also be used for a batch process as the final output is stored in a database table in the application schema of the consumer account.

For the purpose of this demo I have installed it in debug mode in the same snowflake account. However you can always add a create a marketplace listing from the package and share it with your consumers.

A quick look at the model building page of the application. The inferencing page remains the same as it was in the first version of the app.

Here is a quick video for you to watch the installed app in action.

Demo App In Action

Future scope of upgrade for you to explore :

1. Upgrade the app to create a module to check the quality of the data shared by the consumer.
2. Add few data preprocessing steps based on the quality of input data.
3. Add few more ML algorithms that the consumer can experiment with, before choosing the final one for inferencing.

By now you must be curious to dive into the code and test if for yourself. All you need is a Snowflake Account. Here is the google drive link for you to get access to the code zip file and start playing with it.

Link to the first blog : https://medium.com/snowflake/build-your-first-snowflake-native-app-to-estimate-the-price-of-your-property-e4b39f3b184b

Please note opinions expressed in this article are solely my own and do not represent the views or opinions of my employer.

~Cheers