The most comprehensive AI strategy from a vendor so far
A few years ago, I remembered asking a good friend, an expert in the field, “What is AI?”. I am not even sure we agreed on the form the answer should take. Fast forward a few years, and I am not sure we have a clear answer to the question, but we better understand the use cases where AI can be successful in the enterprise.
I just returned from Las Vegas, NV, where IBM held its first-ever TechXchange conference, designed for a very technical audience, with content from (mostly) a very technical audience. Here are a few of the insights I wanted to share with you.
In this article, I will briefly cover use cases and models, define RAG, remind you that AI is not only about generative AI, touch on basic optimization, share some basics about models, and reinforce the role of data. Ready? It's going to be fast.
Six families of use cases and their models
I will focus on six families of use cases:
- Q&A: This is a very common use case where a user can ask questions to a machine using natural languages. The machine will leverage its large (or not, see later) language model (LLM) as well as retrieval-augmented generation (RAG) to give answers that matter.
- Generation: In this context, it is about text generation based on identified sources. We are in an enterprise context where sources matter. Not leveraging or identifying them will have corporate responsibility consequences, such as patent or copyright infringements, privacy violations, bias-based decisions, or worse.
- Extraction: extract information from unstructured documents, like an insurance claim. By extracting data, you also expect the model not to create information (hallucinate) when it does not find the right information.
- Summarization: create a summary from larger documents.
- Classification: classifying documents, dogs & cats pictures, and more.
- Code generation: yes, we have all seen that AI can generate bad dad jokes, but do you want the same model to generate enterprise-grade code? This is where you should use models like Starcoder.
In the previous photo, you can see that all models are not suited for all use cases.
Your rag keeps your AI under control
Retrieval-augmented generation (RAG) is an AI framework for retrieving facts from an external knowledge base to ground large language models (LLMs) on the most accurate, up-to-date information.
Thanks to RAG, you reduce (or even remove) the hallucination a model can bring into your system. It also allows your AI system to say, "I don't know," instead of making something up. Learn more about RAG here or watch Marina Danilevsky's video.
AI did not start in November 2022
On November 30th, 2022, OpenAI released ChatGPT and put generative AI (GenAI) in the front seat. McKinsey predicts that Generative AI is the path to a 45% increase in productivity. Whether you or I like predictions and the institutions behind them, the general availability to consumers of GenAI has created an inflection point.
However, it is important that pioneers were there before and created some foundations, like the AI ladder. Tim O'Reilly remembers when, in 2019,Rob Thomas first told him about this AI ladder idea: it was at a crowded cocktail party during IBM’s Think event. I thought this is one of those ideas, like Open Source Software, Web 2.0, big data, and the Maker movement, that isn’t just a label but a map to help guide people into what had previously been Terra Incognita.
Let’s remind ourselves of the four steps:
- Collect: Make data simple and accessible.
- Organize: Create a business-ready analytics foundation.
- Analyze: Build and scale AI with trust and transparency.
- Infuse: Operationalize AI throughout the business.
Generative AI is one of the enablers of the infuse stage. Accessing the results of the AI workloads via OpenAI's or watsonx's APIs enables this last stage of the maturity model. However, do not forget that you will still need to collect, organize, and analyze.
Modern data management and data integration tools & methodology, like data contracts, will simplify your way there. ProfitOptics can coach you along the way of the four steps.
Optimization means $$$
Don't get me wrong, I stand by Knuth's most famous quote:
Premature optimization is the root of all evil.
Donald Knuth, December 1974
Nevertheless, it does not mean it gives engineers a carte blanche to do anything they want, as Knuth stated in his famous 1974 ACM article.
As I just mentioned, there are a variety of models for different use cases. One criterion is the number of parameters included in the model. If I compare granite.13b to llama2, we are comparing 13 billion parameters to 70 billion, which means that operating llama2 will cost significantly more than granite. I may want to use llama2, particularly if I want its personality, but I also want to be cost-conscious.
I hear your question (and probably objection) coming. Can I achieve the same operations with a 153 million-parameter model as with a 20 billion-parameter model? As often in our industry, the answer is "it depends," but working with a smaller model, like slate.153m, is possible and will avoid potentially out-of-control costs. This is where the experience of a trusted partner comes in handy. Note that slate.153m cannot be used in generative AI, which means, once more, that the model is critical to the use case.
Let's be true about what's in a model
Welcome to the model soup! If you were thinking that each vendor had its own model and it would end there, you are grossly mistaken. OpenAI has a few based on their products (GPT 3.5, GPT base, DALL E) and the evolution of their products (gpt-3.5-turbo and gpt-3.5-turbo-16k for GPT 3.5), for example.
In watsonx.ai, IBM offers several radically different foundation models, with their research labs actively publishing on the topics and new models.
Hugging Face has even created a marketplace of models; you can see them as the GitHub of AI models.
So, how do you know which model to use? This is one of the values IBM wants to bring you with watsonx.ai. You have access to curated models like Red Hat gives you access to a curated Linux in RHEL. Curated models are trained on commercial data, are cleaned from (excessive) bias, reveal their sources, and are compatible with the explainability many legislators require. IBM also provides industry-specific models.
Use the right data
Of course, your AI will need your data. IBM is investing a lot to have great models that each and everyone can use in conjunction with their own data. As much as the model helps, your data is still crucial.
When it comes to bringing data to your LLM, there are several ways we will study in an upcoming article. In the meantime, identify your domains & build your contracts. This will still be useful!
Next steps
AI is not easy. However, AI's promises are within reach, and they are not as difficult as long you use the right partners to help you on this journey.
Should you follow IBM on the Enterprise AI journey? They demonstrated a mature understanding of the technology, the business requirements, and the legal environment. As often with such endeavors, a partner in the middle will help.
Oh, and if you wonder what my answer was to the question at the top of the article, my answer at the time was that AI is all about natural language understanding (NLU) and computer vision (CV). Did it change much?
Interesting additional resources:
