Introduction to Snowflake Cortex: FAQs
5 min readMar 27, 2024
This week I was joined in an AMA (Ask Me Anything) episode by David Taylor, Product Manager for Snowflake Cortex. In this session we showed a quick overview and short demo of this new feature set that brings generative AI to Snowflake via serverless SQL and Python functions.
In case you missed it you can:
In case you missed it, here is a list of some of the great questions asked during the event.
Is Snowflake running LLMs on an external platform?
- No. It is all running inside Snowflake’s perimeter so all of your data maintains your assigned security and governance policies.
- Because there are no separate environments, there are no APIs, integrations or separate governance to set up and manage.
Which LLMs are supported in Snowflake Cortex?
Do the LLMs learn from my data?
- No. Snowflake does not use your data to train any LLM to be used across customers and does not use your data to fine-tune any LLM.
- You can learn more about our commitment to safeguard your data in the AI Trust & Safety FAQ
What is the main difference between the complete() function and some of the specialized functions like summarize() ?
- With the complete function you get two things: the option to choose the LLM you want to use and also run requests against your data using custom prompts. With specialized functions like summarize, translate, or sentiment there is no need to manually select an LLM or do any prompting. The function takes care of all that to give you the result for the task at hand.
- To see what LLM options are available in the complete function check out the documentation.
When I choose a model like Mistral-large or Llama-7b in the complete function? Am I downloading the model into my account to run inference?
- No. As part of the serverless experience, Snowflake hosts all of the models and makes them available on-demand. When you select the model as part of your SQL or Python function, you are just picking which endpoint to hit with your request.
How do I know which LLM I should choose in the complete function?
- The documentation offers a few metrics you can use to help you decide which LLM to use.
- If you’re not sure where to start, try mistral-large first. This will give you an idea of what the most broadly capable model can do. Use this as a baseline to evaluate other models.
What are some of the use cases or jobs I can do with LLMs on my enterprise data?
- Pretty much anything is possible with the magic of prompts, here’s a few examples:
- Generate custom summaries in json format (go to bonus section)
- Turn emails into rich prospect / customer databases
- Classification of form fills
Are all the LLMs available only for text, or does Snowflake Cortex support models for other types of data such as images?
- Snowflake Cortex has support for both commercial and open-source language models from Mistral AI, Meta and Google and soon also will have multi-modal LLMs Reka. For any commercial model such as mistral-large, we take care of all the partnership and commercial agreements so you can use them just like any open-source model with a license that permits commercial use such as Llama2.
- You can read more about our new partnership with Mistral and Reka.
Do I need to purchase a separate product / license to use Snowflake Cortex? How is it priced?
- The LLM functions do not require any additional purchase. All the functions are available for you to use without any separate procurement or agreements, including free trial accounts.
- To pay for the use of these functions, customers can use Snowflake credits. LLM functions are charged based on tokens processed.
- Documentation has a table that shows the cost in Snowflake credits per 1 million tokens for each function.
How do I get Cortex functions enabled in my account?
- To ensure use of LLMs within your organization is governed, by default only the ACCOUNTADMIN role has access. This role can propagate this access to other user roles.
- By using a Snowflake Free Trial, anyone can use the ACCOUNTADMIN role to try these functions.
What coding languages are supported to use Cortex functions?
- SQL and Python are both supported. Guide to Python here.
Does warehouse size impact time to completion, if at all?
- Larger warehouses do not increase performance of LLM execution. This is why we recommend executing queries that call a Snowflake Cortex LLM Function with a smaller warehouse (no larger than MEDIUM).
If the model is not caching or learning anything from my data, how can I have a chatbot that remembers the conversation?
- To give short-term memory to your model, you can keep track of the previous answers and append that to your context before adding the latest question.
- Check out how to do that in this blog.
How do you build Retrieval Augmented Generation (RAG) based applications with Snowflake Cortex?
- Everything you need is available as an integrated experience inside Snowflake. In addition to LLMs for generation, users can currently access vector embedding and distance functions for retrieval, VECTOR as a native data type in Snowflake (in private preview).
- You can try building your own RAG-based document chatbot using this quickstart.
Are there any native SQL functions in Snowflake to convert pdfs to text and perform chunking?
- Using third-party python libraries such as PyPDF2 and langchain inside a Snowpark Python function it’s easy to make execution of conversion and chunking as part of a SQL pipeline.
- Sample code on how to do this in step 2 of this quickstart.
Can I fine-tune models in Snowflake?
- Yes. Using Snowpark Container Services, you can fine-tune any open-source model and run it inside Snowflake. We’ll continue to work to make that experience even easier to securely customize models with your own data.
Snowflake Developer Community
If you have more questions, or if you run into errors while building your project, feel free to reach out to the Snowflake developer community on reddit, stackoverflow, or comment on this post, I’d be happy to help.
