Build Your First Snowflake Native App To Estimate The Price Of Your Property
Do you frequently build “Data Applications”? Are you finding it difficult to manage and distribute your application with your consumers and protect your application IP at the same time ? Are you looking for an opportunity to monetise your application ? If these questions haunt you frequently, stay on this page. I might have a solution for you.
Lets start with defining a Data Application ?
One of the best definitions that I could find over the internet is from PURESTORAGE.COM : Data applications (not to be confused with database applications) are a mix between data visualizations and web applications in that they allow end users (decision-makers, subject matter experts, and even consumers) to visualize and effectively manipulate large sets of data.
Data Developers have been building “Data Applications” for some time now. Packing your application logic and your user interface in a deployment package before shipping to your customer is something that the community has been doing for quite sometime. However there are several challenges that are prevalent today.
- How do you package your provider managed data along with your application ?
- Even if you bundle your data in the deployment package how do you do you ensure that the provider data is not stale as the application ages ?
- Packaging data along with your app, may work in scenarios where the data volume is less and staleness of the data does not impact the application insight. Unfortunately that is not the case with majority of the data applications. Most data applications are supported by a huge volume of dynamic data.
- If you want your application to work on consumer data, how can you make that data available to your application with all the governance controls in place ?
Snowflake has released the Native Application Framework to developers around the world with the public preview in AWS to address all the problems highlighted above. If you are interested to know all the benefits of this framework and understand them in detail, I would request you to go through the Developer Guide.
Here is quick diagram that illustrates the working of a Native Application Framework.
To understand this framework in action I have created a simple application to estimate the price of your property in selected zipcodes in United States. Lets look at sample the application flow :
The project structure shows the program files and highlights their functionalities.
This is the structure of the app repository. There are two program files to focus upon before you go ahead with deployment.
- “houseprice_prediction-preparation.ipynb” -> This notebook uses the data file /sourcedata/house_data.csv packaged within the repository to build the ML model. This dataset is publicly available [in kaggle.com] and can be managed by the provider as and when it is updated. The provider can also retrain the model and update the application whenever required without the consumer having to perform or worry about any of these steps. This notebook loads the data file from the repository to an internal stage. There after, pushes the data into a Snowflake table. The linear regression model is built on the data that is loaded in the snowflake table.
- “deploy_app_to_provider_account.py” -> If you follow the native app documentation or the developer guide, you will see that there are certain number steps to be followed during the process of application deployment. Now, as a developer we love automation and reduce our checkpoints. This is one such initiative where I have automated the application deployment every time there is a change in the app logic or the model is rebuilt by the provider when the source data is refreshed.
Disclaimer on the choice of the ML model and its efficiency : Please note that the objective of this exercise is not to build a model that is super efficient to estimate the price of the property. Hence the choice of the variables or the choice of the model or the model efficiency may not have been perfect. This objective was to showcase how a native application supported by a provider managed ML model can be deployed on a consumer snowflake account.
Once the model is built using the notebook : “houseprice_prediction-preparation.ipynb”, the Snowflake application package stage location looks like the image in the next section. The automated deployment script does all the hard work of moving all the required files from the repository to the stage in the background before building and deployment of the application.
A successful execution of the application deployment script will show you an output similar to the image below where all files in this stage location mentioned above are bundled to create and deploy the application.
At this point of time you are good to test your application in your provider account. If you are good with the application, its time to share the application with a specific consumer account. How do you do that ? You need to create a “Listing” from the “Provider Studio” in Snowsight. Below are the steps to create the listing.
Lets login to the snowflake consumer account. We need to install the application that the provider just shared.
If you have come so far , its time for you to take a look at the application in action. You can go the “Apps” menu of your Snowsight and view the application in action in both the provider and consumer accounts.
By now you must be curious to dive into the code and test if for yourself. All you need is a Snowflake Account. Here is the google drive link for you to get access to the code zip file and start playing with it.
Please note opinions expressed in this article are solely my own and do not represent the views or opinions of my employer.
~Cheers
