Exploring LLM AI at Enterprise Level with Data Privacy in Mind
I am thrilled about the new era of AI that we are entering. I believe that as we continue to learn and evolve, it is important to share our knowledge and experiences with one another. Today, I am going to share what I have learned so far about exploring LLMs (large language models) in the enterprise and how to get started.
First, let’s clarify a few concepts that may be confusing to some.
ChatGPT, technically, when we talk about it, it’s two things: both the AI model (GPT) and the ways to interact/access (Chat) with the model- the name ChatGPT is derived from this combination.
LLM, stands for Large Language Model:
- Language Model refers to a type of artificial intelligence (AI) model that is designed to understand and generate human-like text based on the input it receives.
- The label “Large” refers to the size of the language model. It is measured in terms of the number of parameters it has. Parameters are the internal variables that the model adjusts during training to capture patterns in the data. Most of the popular LLMs have billions of parameters.
When it comes to exploring LLM at an enterprise level, data privacy is a crucial concern. ChatGPT has already taken steps to ensure the privacy of their users by disabling chat history, as mentioned in this blog post. However, at the enterprise level, we must be extremely careful to avoid any data leaks.
Let’s take a look at the following picture that I’ve summarized.
First, it depends on which model you want to use, you have two options:
- Commercial AI models. Here I’ll use OpenAI models, such as GPT-3.5, GPT-4 and etc
- Open source models, such as LLaMA, GPT-NeoX, YaLM, etc
There are other commercial models out there such as Jurassic-2 by AI21, Claude by Anthropic, etc. but I suggest starting from the most known model, GPT by OpenAI.
Regarding open-source models, they are continually updated. For a more comprehensive list, I suggest checking out this resource: https://github.com/Hannibal046/Awesome-LLM#llm-leaderboard
Then you will need a way to host (also known as serve) the model. We probably already know that OpenAI hosted their models on their website, but it’s less known that Azure has an Azure OpenAI service that also hosts OpenAI models. When it comes to open source models, you have the luxury to host on your own computer, with or without a fancy GPU. There are platform and toolkit available already, such as FastChat, GPT4All, etc.
Once the model is hosted, there are usually two ways to access it:
- Chat UI, a chat interface where you can type messages back and forth with it, as seen in ChatGPT. Please do note that unless you are using ChatGPT Business subscription (not released yet), the messages you typed would be used for training at OpenAI, so I suggest avoiding using ChatGPT UI to explore. If you run models locally, then of course no data (prompts or any data) is leaked into anything that is outside of your computer.
- API access, where you can “chat” with the model programmatically. Commonly used tools for this purpose include command line, Python apps with SDKs, Jupyter Notebook, as well as application frameworks such as LangChain or LlamaIndex. It is also worth noting that Microsoft Azure has a web tool called “Azure OpenAI Studio” for interacting with the model.
I would recommend utilizing API for accessing models, either from OpenAI or Azure OpenAI service:
- It is quite safe (compared to via UI, aka, ChatGPT) to access GTP models hosted by OpenAI via API, as you can see from their API data privacy policy “By default, OpenAI will not use API data to train OpenAI models or improve OpenAI’s service offering.”
- If you decide to use Azure OpenAI service, Microsoft has taken extensive measures to ensure the security of your data. As stated here: “no prompts or generations are stored in the model. Additionally, prompts and generations are not used to train, retrain, or improve the base models.” You can find more information on their data privacy FAQ page as well.
Overall, I would suggest exploring GPT models (for best performance, compared to open source models), hosted by either OpenAI or Azure, via API.
I hope you find this article helpful to get you started. Please follow me on Medium or LinkedIn, and stay tuned for more articles on exploring AI models.
