What’s Hard about Large Language Models in the Enterprise?
Large Language Models (LLMs) are the rapidly evolving technologies behind products like OpenAI’s ChatGPT and Google’s Bard. LLMs and related Generative AI (Gen AI) technologies can allow people to interact with applications in natural ways using words, sounds and imagery.
Consider the terrible customer support chat experience that we’ve all suffered through. Full of hope, we type in our detailed question only to be given a suggestion that we visit the support web page, or we’re offered an irrelevant answer that seems like it was based on just one word in our question. Sometimes there is a overworked multi-tasking human on the other end who eventually gives us a good answer, but they take a long time and seem to lose the context easily. LLMs have the potential to radically improve this experience: they can be accurate, nuanced, fast, and cost-effective. But LLMs can be unruly and need a lot of help to be useful and reliable in a business context. Here are just a few reasons why:
- LLMs are trained on public data up to a certain point in history, so they have no knowledge of a businesses’ private data nor of public data that has not been included in training. To answer questions about these data sources, LLMs need to be given extra context. Management of this context is a complex problem that requires a raft of sophisticated data systems and algorithms to work well.
- Out-of-the-box LLMs will try to solve problems or answer questions that are beyond the scope of a given business application. To be used effectively, LLMs must be constrained. Prompt engineering is a tool that can be used to provide constraints but it is not the only way to reduce the chance that the LLM will run amok.
- There is a growing stable of LLMs with different capabilities, different security and privacy guarantees, and a range of licensing terms from open source to proprietary. A business application could employ several specialized LLMs that are optimized for different tasks. In this fast moving marketplace, developing the right application architecture and choosing the best LLMs for quality, speed and cost requires vigilance and expertise.
- The concept of Agents has emerged as a helpful way to think about using using LLMs. Think of an Agent as an LLM that is combined with context and constraints to obediently handle requests within a well-defined scope. It’s a great concept, but then an enterprise needs a way to define, build, test, deploy, share, secure, manage, and monitor their library of Agents.
Fixie was founded to overcome these challenges, and many more, for enterprise application developers. We make it easy for developers to create LLM-backed agents that can answer questions about private data sources, draw from historical customer support conversations, and interact with existing enterprise resources via traditional APIs.
All Fixie Agents are managed and monitored within the Fixie cloud service. Each Fixie Agent has a simple REST API for prompt-response interaction, making it easy to add flexible natural-language capabilities that delight and astound your users.
If you are imagining a natural-language user experience that could be amazing for your business, give Fixie.ai a try or talk with us.
