Use Case Milestones on the Path from Data Foundations to GenAI Applications

By: Elisabeth Strenger

RTInsights attended the Gartner Data & Analytics Summit 2024 earlier this year to hear about the most recent research the firm had done, the insights and recommendations it had distilled from hundreds of inquiries with the companies across industries and technology maturity levels, and both adoption and innovation trends.

We also had an investigative agenda–find examples of real-time data serving to advance AI, especially GenAI use cases, and take the pulse of the rate of adoption, especially given the many discussions around the risk and cost involved. Were companies actually implementing GenAI solutions? And what were they using them for?

Interviewing industry-leading software providers about what they were seeing their customers do resulted in an interesting aggregation of use cases–interesting especially because these software providers viewed the AI and GenAI use cases through the lens of their products’ capabilities and the business problem they focus on solving. These different perspectives result in a composite view of the opportunities organizations see in AI, the ones they are investing in first, and the technology strategies they rely on.

While we focused the interviews on the actual, the present, conversations with the industry’s leading innovators strayed into what use cases were emerging among early adopters and what solutions were still over the horizon but already being conceptualized by these software providers.

A peek into our findings: yes, real-time data processing and analytics are integral parts of some of today’s AI solutions and are essential to many of the solutions in the proof of concept or piloting phase, that is, the final phases on the path to deployment.

See also: Gartner Data & Analytics Summit Delivers on GenAI

Extremely fast data ingestion

Madhukar Kumar, CMO of SingleStore, described his customers’ AI and GenAI use cases that rely on having extremely fast data ingestion to support real-time analytics. At least, that’s the starting point. Surge pricing, fraud detection, and investments management involve millions, sometimes billions, of transactions that must be analyzed in real time. LLMs and Transformers allow AI to be part of those analytics so that users can conduct semantic searches against structured and unstructured data through the addition of a vector database, as requested by an innovative customer several years ago. Now, customers are eager to do meaning-based searches that require finding data clusters, using algorithms to find next-nearest neighbors, and working with contextual data facilitated by retrieval-augmented systems (RAGs).

Beyond this new breadth of search capabilities, customers’ use cases demand extremely fast results. As Madhukar described, SingleStore responded by adding new algorithms: hierarchical, small habitable world, product quantization, and inverted index. Using all of these, the data is compressed by 60 or 70%, and the search can be run in memory, so the search is 200 times faster. Customers can now search across both unstructured and structured data in a few milliseconds and in one SQL statement, so many real-time AI use cases have now become a reality. For example, live video or live conversations can be vectorized in real time and then handed over to an LLM. Financial services, supply chain, ride-sharing, and ad tech have the most appetite for this scenario.

SingleStore does not specifically see GenAI applications deployed yet. Customers are building two types of applications: information synthesis and agentic. They can work together, for example, is a customer agent (human) uses a chatbot that curates all the data and helps the agent answer a question. Agentic takes it further: Based on the curated data, the chatbot can create, close, or update a ticket. Several customers are in the process of building this kind of application.

What’s next? Combining different agentic applications or, more accurately, a mixture of different expert models that work in coordination.

See also: Smart Talk Episode 2: The Rise of GenAI Applications with Data-in-Motion

Unifying analytics and AI for a 360-view

Customer 360 and customer experience use cases still reign as the primary objective of enterprise data initiatives. However, the goal of having a comprehensive view of a customer, let alone a customer base, is still beyond our reach. The more the technology for this use case develops, the more data and analytics methods we uncover, AI and GenAI being the most recent advances that will help us fill in the gaps.

Nik Acheson, Dremio’s Field CTO, identifies supply-chain analytics as another significant area where companies are investing heavily to bring together disparate types of data, historical and real-time, structured and unstructured, to build a holistic view of a complex process that can predict events and recommend options based on current inventory or suppliers’ logistics. What underpins these use cases is query acceleration and unif…

Continued on RTInsights.com…