Advanced LLM Operations in Snowflake: A Case Study

Streamlining AI with Snowpark ML Model Registry: A Case Study in Automotive Retail Industry.

In today’s rapidly evolving tech landscape, seamlessly integrating and deploying machine learning models is crucial for maintaining a competitive edge. As a Senior Data Architect at Infostrux Solutions, I have had the opportunity to lead AI/ML initiatives and implement innovative solutions for various industries. One such project involved leveraging Snowflake’s cutting-edge Snowpark ML Model Registry library to perform advanced LLM operations for a client in the automotive retail sector.

This article will walk you through the steps we took to develop a production-ready automatic customer classification system, highlighting key aspects such as ensuring reproducibility, continuous integration, and democratizing model access. By sharing our approach and insights, I hope to provide a valuable resource for those looking to enhance their machine-learning capabilities within the Snowflake ecosystem.

Project Overview

Our client wanted to create an automatic customer classification system that they could use to understand areas of improvement within their services quickly. But more than just a proof of concept, we aimed to implement a production-ready system that would allow the company to:

• Ensure reproducibility by managing different features and model versions.
• Ensure continuous integration and deployment by automatically serving new model versions and keeping the sources up to date.
• Democratize model access by lowering the technical skills required to interact with the model.

The image below illustrates the architecture we implemented for the client to accommodate the mentioned characteristics and show data flow from various sources through different processing stages in Snowflake, culminating in data analytics and review categorization.

Customer Review Classification Architecture with Snowpark ML Model Registry and Snowflake Feature Store.

• Data Sources: Salesforce, SAP, Customer Reviews: Data collected from these sources.
• Processing Stages: Ingest: Data ingestion into Snowflake. Clean: Data cleaning to remove errors. Normalize: Data normalization for consistency. Integrate: Data integration from multiple sources. Analyze: Data analysis for insights.
• Data Management: Feature Store: Stores categorized review data. Dynamic Tables: Manage dynamic, updated data. Model Registry: Maintains models for review categorization.
• Review Categorization LLM: Large Language Model for categorizing reviews.
• dbt Core, Terraform, Apache Airflow, GitHub: Tools for data transformation, infrastructure management, workflow automation, and version control.

Key Aspects of Machine Learning

If you’ve been working in Machine Learning for some time, you'll understand two fundamental aspects:

1. There is no such thing as the best model or Machine Learning technology. Sometimes, a linear regression can be sufficient for the task at hand. This implies that a good model registry library should be able to store your model regardless of the underlying technology.
2. Building Machine Learning is an iterative process, and you should aim to keep your iteration process short. Delivering the first version of your model as quickly as possible and iterating after that is essential.

The Snowflake team behind the Snowpark ML Model Registry library understood this well and built the library to accommodate any underlying ML technology you use. You can use Sklearn, Tensorflow, Pytorch, or any Python library to ensure that the library can accommodate your requirements.

Implementing the Model Registry CustomModel Library

For our customer, we use the Model Registry CustomModel library to pack an LLM for zero-shot classification of customer reviews. The library is quite simple to use. First, you implement a CustomModel class where you define the specific prediction behavior you need. In our case, we implemented the use of the DebertaV3Small model and the tokenizer to do the classification.

After that, you only need to load your model artifacts and instantiate your class, producing a fully contained object providing a standardized interface for obtaining results. The image below illustrates the process.

Model Logging and Deployment

Finally, we again use the Model Registry library to log and serve the first version of the model. To log your model, you just need to provide the instance of your custom class and its library dependencies. To deploy the model, we implemented a Snowflake stored procedure that loads the latest model version as a UDF, making the functionality available to SQL users. This effectively lowers the entry point for people to interact with the model. The following image also illustrates the process.

Conclusion

And that’s it. With these simple steps, we established an automated and iterative process for delivering progressive improvements to the model’s results without impacting its usability for non-technical users, regardless of the underlying technology supporting the customer classification process.

I’m Fabian Hernandez, Senior Data Architect at Infostrux Solutions. Thanks for reading my blog post, and I hope you enjoy the content! You can follow me on LinkedIn and subscribe to Infostrux Medium Blogs for the most interesting Data Engineering and Snowflake news. Please let us know your thoughts about this approach in the comment section.