Open Source LLMs are Enterprise Ready [FREE REPORT]
Amateurs discuss the latest AI breakthroughs at Google and OpenAI. The pros? They’re looking at open source LLMs. Here’s why you should be a pro.
Welcome to Episode 33 in Prolego’s Generative AI Series.
Recently, we released a LLM Optimization Playbook to help teams optimize LLM application performance.
We expanded our efforts with a study assessing various optimization impacts. You can read the full study at prolego.com/ragstudy, but here is our key discovery: providing correct context (e.g., RAG) trumps choosing the best LLM. Even smaller open source LLMs like Mistral-7B matched GPT-4’s performance with the right context.
These results surprised us because our study has the “real world” challenges we encounter in industries like financial services, healthcare, defense, pharma, and manufacturing. We used the Formula 1 rulebooks, documents with arcane terminology and complex structure. We also evaluated the model with human-generated questions similar to those asked by business users.
Nevertheless, smaller models achieved the same results when provided the right context. Before discussing the implications, here is a quick primer on open vs proprietary LLMs.
Companies like Google, OpenAI, and Anthropic are building and selling proprietary LLMs. Most customers pay a per-token cost and access them through an API. These models get lots of media attention.
Alternatively, companies like Meta are building and releasing LLMs as free, open source software, just like the Linux and Android operating systems. Other developers can use and improve them. Now Meta isn’t being generous. It uses these LLMs to sell ads, and it benefits from the improvements by others.
Open source LLMs are not hobby projects. This serious business, with billions of dollars invested globally, and thousands available on HuggingFace, continually enhanced by developers.
Here are a few reasons why you should consider them.
- Cost. Small, open source LLMs are significantly cheaper than large, proprietary ones. Mistral7B c osts about 10 cents per million tokens, effectively free for most business use cases. Meanwhile, Anthropic’s largest model costs about $50 per million tokens, a price that could be more expensive than human labor.
- Security. Many companies have policies precluding them from sending data outside of their environment, and thus they cannot use hosted LLM solutions like OpenAI’s GPT-4. By hosting LLMs within your environment you can leverage the technology without violating policies.
- Control. OpenAI can change its models anytime, and many developers report unexpected, undocumented performance changes that suddenly break their applications. You control updates to open source LLMs you host.
While there’s no one-size-fits-all solution, and a mix of models is inevitable, open source LLMs offer compelling benefits and are rapidly advancing. Consider integrating them into your AI strategy.
