A Strategic Framework for Building LLM Applications
When ChatGPT went public over a year ago, it gave the world unfettered access to the most advanced AI models. We were able to realize firsthand what AI can do for us. We began imagining how to use it to unleash our creativity and boost productivity.
Large language models (LLMs) have dismantled all the barriers to entry, empowering businesses of all sizes to unlock the transformative power of AI, regardless of budget or technical expertise. Many of you have built compelling LLM applications.
But with great power comes great complexity — choosing the right path to build and deploy your LLM application can feel like navigating a maze. Based on my experience guiding LLM implementations, I present a strategic framework to help you choose the right path. This framework categorizes LLM applications into three distinct approaches, each tailored to your specific needs, skillset, and time-to-market goals.
Off-the-Shelf Gen AI
One way to simplify the process is by adopting a “plug-and-play” approach. You can achieve this by utilizing pre-trained and hosted models like OpenAI or Palm 2. These models offer developer-friendly APIs that allow you to build advanced applications with minimal technical expertise. Frameworks such as LangChain and LLamaIndex make this even more easier.
By understanding the complementary strengths of the following three fundamental techniques — prompt engineering, functions & agents, and RAG — you can unlock LLMs’ full potential and build truly transformative applications.
The Secret Sauce: Prompt Engineering
For this approach to be successful, it is essential to provide the right instructions. That’s where prompt engineering comes in. Your prompts must be clear, with detailed commands telling the model what you want it to do and should not do. Remember to include the context and data to generate the correct response and personalize it by telling it what style and tone.
If you feel the responses are generic or verbose, employ few-shot learning to teach the LLM by example.
Supercharge Your Model with Superpowers: Functions and Agents
If you need to go beyond text generation, look into employing functions and agents. Imagine your LLM App ordering pizza based on your preferences — that’s the magic of functions and agents. Functions are tiny extensions, enabling your model to interact with APIs, databases, and other systems.
You can employ agents if you build applications beyond a simple question and answer or a simple task. Agents can understand the entire context, break down multi-step objectives into smaller steps, and adapt their responses across multiple conversation turns. Imagine an agent seamlessly navigating a complex customer service interaction, analyzing sentiment, reasoning through the issue, and making informed decisions across multiple exchanges.
Combining the flexibility of functions and agents with the power of prompt engineering will unlock a broader range of possibilities for your LLM applications.
Knowledge is Power: Boosting Accuracy with Retrieval-Augmented Generation (RAG)
Prompt engineering relies on crafting instructions for the model, but it can’t guarantee factual accuracy or real-world grounding. RAG solves this by retrieving relevant information from a knowledge base before generating a response. Imagine asking your LLM app for historically accurate creative content or a chatbot confidently answering policy questions based on internal knowledge. That’s the magic of RAG.
You can combat hallucinations by verifying information and preventing fabricated details. Plus, you can ask the LLMs to explain their responses by citing your sources. Finally, RAG excels at understanding context, leading to nuanced and relevant responses in complex situations.
By leveraging one or more of these three techniques, you can develop complex LLM applications capable of summarizing support conversations, searching through thousands of documents, and creating task-oriented chatbots.
The best part is that you don’t need to hire AI engineers for this; full-stack engineers would suffice. And, since you are utilizing proprietary models, you don’t need to worry about the complexities of hosting these models.
You can launch your applications swiftly and with minimal expenses.
Customized Gen AI Models
While the Off-the-shelf approach offers plug-and-play convenience, some tasks demand a custom approach. Enter Customized Gen AI models, where you take pre-trained models and fine-tune them like a race car for optimal performance.
If you want the model to understand your domain, data, and brand voice without including that with every prompt, fine-tune the model. For example, a travel company specializing in travel recommendations can fine-tune a pre-trained model to understand its unique data and brand voice. This model could generate personalized itineraries that consider their customers’ past travel preferences, current weather conditions, and ongoing events — making them feel like VIPs.
Level up your control:
With fine-tuned models, you can write highly converting product descriptions for your eCommerce site, generate medical reports on your specific clinical data, or understand the sentiment of customer reviews.
Though fine-tuning a model is slightly more expensive and time-consuming than the off-the-shelf approach, it is still cheaper than training a model from scratch, as you are only tweaking a smaller subset of the model’s parameters to suit your needs.
This path also does not require AI expertise. Full-stack engineers with data science skills can effectively fine-tune and integrate models into your infrastructure.
If you are using proprietary models, you can save time and money on DevOps.
Train Your Own Models on Proprietary Knowledge
There comes a point when you need a Gen AI solution tailor-made to your unique requirements — something that off-the-shelf or even fine-tuned models can’t fully address. That’s where training your own models on proprietary knowledge enters the picture.
Imagine a hospital developing a custom AI model trained on their vast patient data. This model could analyze medical scans and predict disease risk with unprecedented accuracy, potentially saving lives and revolutionizing healthcare.
Mastering Your Domain:
Training custom models are compelling when dealing with niche domains or sensitive data requiring more specificity. Whether it’s creating predictive models for financial markets or diagnostic tools for rare diseases, this path allows you to leverage your proprietary datasets to develop models that can provide insights and accuracy that generic models cannot match.
Innovating on Your Terms:
When you train your own models, you innovate on your terms. You choose the data, training methods, and performance metrics. This approach is for forward-thinking businesses building unique AI capabilities.
Navigating the Challenges:
Training machine learning models from scratch is challenging and resource-intensive. With careful planning, you can gain complete control over the AI’s capabilities, and the potential for competitive advantage and innovation is vast.
Big investments:
This route demands more data, expertise, and time. You’ll need a team of ML engineers, data scientists, and engineers for model building, training, and lifecycle management. It’s a significant investment, but the ROI can be substantial for high-stakes industries.
Choosing the right path
We’ve explored three paths to building your LLM applications, each with unique advantages and challenges. To help you navigate this golden triangle of Time, Cost, and Control, I summarized the three paths in this handy comparison table:
Conclusion
So, which path will you take? Whether you’re a budding entrepreneur or a seasoned leader, there’s a generative AI approach that can empower your business. Start exploring today, unleash your creativity, and let AI be the wind beneath your wings.
