Building with AI (not for AI)
By: Branavan Selvasingham, VP, Head of AI; Jake Irwin, AI — Software Engineer; Melissa Valdez, AI Lead Consultant; Shahir A. Daya, Chief Technology Officer; Charbel Safadi, Group President
In this article, we will discuss Zafin’s approach to AI, using Zafin Studio’s Product & Pricing Index (PPI) as a focal point. At its core, PPI is a global banking data aggregation tool, with the aim to provide our clients with easy access to the global banking landscape and assist them in formulating new product propositions.
To achieve this ambitious goal, we’re leveraging AI in both the preparation of the index (a back-end data transformation pipeline); and as a new medium for interaction with the index (a channel-agnostic Natural User Interface or NUI experience).
The Effects of Democratized AI (or A Primer on AI Data Transformation Pipelines)
A by-product of the overall industry’s mission to democratize AI is the increased accessibility to capabilities that historically took entire organizations to achieve. We’re at a point where these capabilities can be used as foundational elements — that is, they enable us to supercharge our tools without getting distracted from our core focus of banking modernization.
Index preparation and maintenance is a particularly nuanced challenge that requires an understanding of the domain space at a micro and macro level. There are many variations to how banking products are structured; how features are packaged; how geographies are categorized; and yet, there is a connective consistency throughout all of them. The challenge is in recognizing and giving weight to these subtle variations in structure and values; and simultaneously, standardizing and normalizing down to a consistent index. Such an index will allow for comparisons across products, generalizations on trends, and the distillation of meaningful insights.
Large Language Models (LLMs) are well suited for this challenge. The AI powered data transformation pipeline (shown in Figure 1) starts with the scraping of 200 financial institutions which are stored into a blob store. This raw text is cleansed, resized, and chunked as needed to fit into GPT-4’s context window and to improve the performance of the pipeline by feeding GPT-4 only relevant information. The blobs of text are processed by a chain of LLM prompts. The outputs are essentially database insert suggestions consisting of all the features, rates and fees for a particular product.
The validation process of each suggestion is a bit of an art. First we filter for hallucinations and volatility by checking for consistency in the database insert suggestions across identical runs (the exact number of identical runs is something that fluctuates). Then, we look for variations in column values across suggestions using multiple sources for the same product (i.e. webpage vs. PDF). During this multi-source merge process, we automatically fill in any non-conflicting but missing fields that were not available from one source but were observable in the other. This gives us a final unvalidated product profile merged from multiple runs per available source.
Then, an automated flagging process creates tasks for human review that highlight conflicts between sources in the column value suggestions. These conflicts are placed higher in priority while other due diligence validation tasks are placed lower in priority. The human review process takes time and may seem contrary to an AI-driven approach, but it is in fact a core pillar of our approach to AI: human in the loop.
Once the validation tasks are complete, the unvalidated data is moved to a validated database. We create an intentional gap between this database and the actual production database which serves more as a host for static tables rather than any real-time index preparation activities. Thus, Zafin Copilot and any user accessing the Product & Pricing Index application can easily browse a standardized schema that compresses the global banking landscape.
Widening the Accessibility Aperture with NUI
The technology space is undergoing a tectonic shift in how users interact with platforms. Historically, with the entry of each new paradigm in user interface, we’ve seen a massive increase in the accessibility of capabilities and the diversity of users that the platform serves. Consider the shift from terminal / command-line user interfaces (TUIs) to interactive point-and-click graphical user interfaces (GUIs). This shift ignited a universal transformation across industries where the power of computing transcended from a niche skill set to a digital revolution.
Similarly, another massive increase in accessibility to capabilities is happening with the advent of mainstream natural user interface (NUI), primarily ushered in by OpenAI. It is in alignment with this coming shift in user behaviour, experience expectation, and greater accessibility to capabilities, we have embarked on the journey of Zafin Copilot.
Zafin Copilot is both a knowledge assistant and an agent of action to support our banking users. We want to enable a conversational interface that answers relevant banking product questions such as:
- What are the current trends in banking products, and what are the demands in the market?
- What products and features are offered by our competitors?
- How can we adjust pricing models to maximize profitability — without sacrificing competitiveness?
Zafin Copilot can understand and decompose high-level intent from the user and turn questions into actionable steps. It can execute those tasks by forming queries on the Product & Pricing Index and interpreting the results.
It is designed with specific instruction sets on understanding the Index’s schemas, including the meaning behind various data and relevant validation steps. It generates dynamic context based on the query, enabling it to respond in a meaningful way to a wide array of intents. We’ve ensured that it stays focused on the domain and does not stray too far away from its enterprise context. Though we’ve put in place definitive constraints on it, we’re noticing it can come up with approaches which do not have any supporting tangential or related examples given by us. This is because our design lets the copilot first figure out ‘how should I approach this’ and keeps the human in the flow for confirmation of approach. This is the most exciting part. Lastly, it has access to full-functionality REST APIs, ensuring Zafin Copilot has scalable, flexible, and lightweight access to up-to-date information across the platform. It is through this that Zafin Copilot can become a cross-product engine that permeates across our product platform.
Our Guiding Principles for Deploying AI
As Zafin deploys AI capabilities, three key principles guide our development activities:
- Consistency and accuracy — Responses must be consistent, accurate, derived, and explainable based on an explainable process.
- Acceleration and expansion — We want to thoughtfully expand the landscape of information, knowledge, and capabilities available to more users while meeting them where they are.
- Human in the loop — always. We are committed to always have a human review and validate activities that ensure quality and reliability.
With these principles in mind, we can move forward with speed and determination into exciting and unfamiliar territory.
Challenges of Today
A feature of working on this frontier is constantly solving for novel technical challenges.
For example, initial token limits prevented sending all available context to the generative AI model with each call, so we had to intelligently select and supply access to the right information at the right time via our Task Manager (Figure 2). Another example is the fact that large language models struggle to break down complex problems on their own. Zafin Copilot uses Conversation Manager to deliver single, well-defined tasks to a set of purpose-specific agents. Lastly, for most of 2023, OpenAI’s APIs were stateless, meaning conversational AI applications such as Zafin Copilot had to preserve and manage conversation context on its own using runtime caches and conversation histories.
Even as we build out these innovative solutions for our application stack, we are hyper-aware that the generative AI models we use are improving at an unprecedented rate.
Opportunities for Innovation Ahead
Last month’s OpenAI Dev Day brought the announcement of game-changing updates to their API (though not yet available for consumption in full on Azure OpenAI at the time of writing).
The forthcoming release of OpenAI’s GPT-4 Turbo with a larger 128K context window will open up a wide range of opportunities for Zafin Copilot and the AI Data Transformation pipeline. Our platform’s use of chunking can be reduced and we won’t have to worry about leaving relevant information out of our prompts.
GPT-4 Turbo’s JSON mode will enable lighter prompting and decreased variability in responses. This will reduce much of the burden of formatting responses and in turn, free up more resources for responding to the user’s inquiry. The seed parameter setting will further allow us to reduce the randomness of the AI models. Greater predictability improves the conversational experience and increases confidence that the tool can complete the tasks it was designed for.
The new Assistants API will accelerate integration with in-house models and tools through new capabilities like code interpreter, knowledge retrieval, and function calling. GPT-4 Turbo’s improved speed will allow us to reduce our average answer time. Our less computationally intensive tasks can be sent to Turbo to help free up resources for the main GPT-4 model. Finally, the team is also excited to incorporate the new multimodal capabilities, such as text-to-speech to enrich user experience further.
Our architecture is well-positioned to pivot quickly and take advantage of these powerful updates. We take a modular approach so we can rapidly upgrade elements to the state-of-the-art with minimal re-work. We expect to fully adopt these new features into Zafin Copilot in Q1 2024.
As we do, we will share deep dives into several key aspects of our approach for building with AI in the age of rapid disruption. Check back with us soon — we’ll have some exciting updates to share along the way.
References
- Zafin Studio: Your gateway to innovation in banking (2023) Zafin. Available at: https://zafin.com/studio/ (Accessed: 22 December 2023).
- Natural user interface (2023) Wikipedia. Available at: https://en.wikipedia.org/wiki/Natural_user_interface (Accessed: 22 December 2023).
- Large language model (2023) Wikipedia. Available at: https://en.wikipedia.org/wiki/Large_language_model (Accessed: 22 December 2023).
- Boyd, E. (2023) Introducing GPT-4 in azure openai service, Microsoft Azure Blog. Available at: https://azure.microsoft.com/en-us/blog/introducing-gpt4-in-azure-openai-service/ (Accessed: 22 December 2023).
- OpenAI. Available at: https://openai.com/ (Accessed: 22 December 2023).
- OpenAI devday. Available at: https://devday.openai.com/ (Accessed: 22 December 2023).
- OpenAI platform (no date) GPT-4 and GPT-4 Turbo. Available at: https://platform.openai.com/docs/models/gpt-4 (Accessed: 22 December 2023).
- Assistants Overview — OpenAI API — platform.openai.com. Available at: https://platform.openai.com/docs/assistants/overview (Accessed: 22 December 2023).
