Approaches in Using Generative AI for Business Automation: The Path to Comprehensive Decision Automation

9 min readAug 4, 2023

Co-authors: Allen Chan, Luigi Pichett, Yazan Obeidi

Generative AI has been rapidly evolving, enabling more sophisticated interactions with Large Language Models (LLMs) to pass Turing tests. In a series of articles, we will be looking at how we can combine existing IBM Business Automation technologies, whether it is Workflow, Content or Decisions, together with LLMs to address various business scenarios.

After a first article about GenAI and content management with FileNet, we delve into the world of automating decisions using LLMs. For decades, enterprises have relied on software solutions to determine claim eligibility, approve loans, and set service prices.

Among these solutions, Business Rule Management Systems have been widely embraced by industries, including financial services and insurances since the 1980s. These systems enable the development, testing, simulation, deployment, and maintenance of business policies through logical rules. Domain experts define a business ontology with logical predicates that apply to their decision concepts, resulting in structured outcomes. These rules, often presented in the form of IF-THEN statements or decision tables, are evaluated by a causal inference engine capable of instantiating and chaining them within a data context, thus automating a reasoning path. This technology has matured over the years, enabling the capture of complex decision models with thousands of rules and decision tables. IBM Operational Decision Manager (ODM) [1] has proven to be scalable, handling over a billion decisions daily while spending only a few milliseconds to evaluate a complex set of rules for each decision.

Given the recent progress in Generative AI, a question arises: Can LLMs alone suffice to automate business decisions and efficiently process claims, determine eligibility for social services, or validate banking transactions?

Considerations

What is an LLM?

In a nutshell, a Large Language Model compresses information read from a training text corpus to generate new text from a given prompt. Based on a neural network architecture, its behavior is entirely statistical. It takes a sequence of tokens (groups of characters) expressed in the prompt and produces a sequence of tokens that are highly probable based on its training data. It is not a reasoning algorithm, does not rely on logical mechanics, and can be seen as a stochastic parrot [2].

What are the key requirements for enterprise decision making

Enterprise decision automation involves the use of technology and software to automate decision-making processes within an organization. The key criteria for successful enterprise decision automation are as follows:

Accuracy and Reliability: The system should be accurate in processing data and making decisions to ensure the trust of users and stakeholders.
Scalability: The solution should be able to handle large volumes of data and decision-making requests without significant performance degradation.
Flexibility and Adaptability: Business environments are dynamic, and decision-making requirements may change. The system should be flexible enough to adapt to new business rules, policies, and regulations without requiring major devops cost.
Real-time Decision Making: In certain scenarios, real-time or near-real-time decision-making capabilities are crucial. The system should process data and provide decisions within acceptable time frames.
Transparency and Auditability: Enterprise decisions often impact critical processes, and stakeholders need to understand how decisions are made. The system should provide clear explanations for decisions, and it should be auditable for compliance and regulatory purposes.
Security and Data Privacy: Since decision automation deals with sensitive data, security and data privacy are paramount. The system should employ robust security measures to protect data from unauthorized access or manipulation.
Monitoring and Reporting: The system should have comprehensive monitoring and reporting capabilities to track the performance of decision-making processes and identify areas for improvement.
Cost-effectiveness: Consideration of the system’s cost in relation to its benefits is vital. The solution should offer a good return on investment and align with the organization’s budgetary constraints.

Overall, a successful enterprise decision automation solution should align with the organization’s objectives, streamline decision-making processes, and contribute to improved efficiency and productivity.

Do LLMs alone achieve all these requirements?

While some of LLMs show impressive results and some reasoning capabilities they fail as easily when repeating the experience, or changing it a little. You can experiment this double face bevahiour with the pizza ordering bot shown in the DeepLearning.ai OpenAI tutorial [3]. Depending on the runs, the bot provides the expected outcome or a surprising one, even with only a minor change in the input.

Another challenge with LLMs is their maximum token limit, which restricts the amount of context they can handle. When the required context exceeds this limit, it becomes problematic to achieve accurate results. One approach to address this is fine-tuning the LLM on a private corpus to inject more specific context. Additionally, reinforcement learning with human feedback has been applied to some LLMs to mitigate their statistical drawbacks, but perfection in automation remains elusive.

While LLMs offer practical NLP power, can we use them in combination with rule-based decision engines?

Rule-based decision engines offer a different approach to decision automation. They require explicit modeling to formalize a business ontology and precisely specify the logic of the business policy. This ensures consistent decision-making and transparency, making them a preferred choice in certain use cases.

In short, rules and LLMs have their sweet spots. The challenge is: how can we combine these technologies to capitalize on their strengths, just as we have invented composite materials to go beyond the properties of iron and carbon?

A quick view on LLM and Rules approaches

In this article, we have listed the following approaches to combine strengths of these technologies to automate enterprise graded decisions.

The LLMs are hosted in watsonx.ai[4] or alternate cloud AI services, and when available, they can also run locally in the IT environment.

1. Natural Language Understanding followed by Rule reasoning

Description: In this approach, we first utilize a Language Model-based Machine Learning (LLM) to comprehend plain text and extract structured data from natural language. Subsequently, we employ a causal rule engine to reason deterministically about this structured data, often combined with additional information from a system of records.

Pro: The integration of LLM for Natural Language Understanding (NLU) to extract structured entities and a rule-based engine is straightforward. Implementing the LLM-Rule pipeline involves sequentially calling the APIs of both engines and passing a parameter context between them.

Con: This approach is limited to identified text extraction capabilities. In cases where the extraction process does not find the expected data from the input text, it’s crucial to set appropriate guardrails to handle such scenarios during the reasoning phase.

2. Rule reasoning followed by Natural Language Generation with LLM

Description: In this approach, a rule-based decision engine first makes a decision based on structured data. Subsequently, a Language Model-based Machine Learning (LLM) generates a natural language outcome from the decision.

This pattern involves utilizing the rule inference engine for reasoning and employing the LLM to generate coherent natural language content. It is particularly useful for providing explanations and justifications, summarizing corporate decisions. The generated content can be tailored based on the depth of information desired and the recipient’s profile.

Figure 2. Rule reasoning followed by Natural Language Generation with LLM

Pro: This approach offers the advantage of leveraging the powerful Natural Language Generation (NLG) capabilities provided by the LLM to create well-phrased emails or letters for communicating automated decisions. It integrates smoothly with reasonable prompt engineering and decision input parameter passing.

Con: While rules concentrate on reasoning with structured data, the LLM focuses on Natural Language Processing (NLP) tasks. This separation might require careful consideration to ensure coherence and accuracy in the generated content. Automated testing of the Natural Language Generation (NLG) result needs care to cope with the range of structured data values and the variability of the generated text.

3. Rule reasoning driving Natural Language Processing with LLM

Description: In this approach, the rule inference engine acts as the master engine, invoking Language Model-based Machine Learning (LLM) on demand. The causal rule engine drives the rule evaluation and dynamically calls out the LLM for two delegated tasks:

Processing text received in its decision context for understanding (NLU) and extracting structured data.
Generating text (NLG) to produce, for example, a summary of the automated decision in plain text.

This integration goes in the continuity of the calling out from any Machine Learning model from IBM ODM[1] or IBM ADS [5], to consider probabilities of risk or opportunity during the decision making. Similarly, LLMs can be called from a rule, either remotely or locally, depending on its form factor.

Figure 3. Rule reasoning driving Natural Language Processing with LLM

Pro: A rule lead multi-usage pattern. LLMs are called on demand depending on the reasoning path for NLP tasks

Con: Reasoning and LLM engines are tightly coupled, necessitating a more intimate integration. The structured data used in decision-making must align with the expected NLP tasks. Proper guardrails are needed to mediate the structured-unstructured data frontier to ensure the reliable quality of the alloy.

4. Extract business rules from plain text with an LLM, and run these rules in a logical engine

Description: This approach utilizes a Language Model-based Machine Learning (LLM) to extract automation assets, including business rules, data models, and signatures, from plain text business policies. These extracted assets are then used to generate an automation project in IBM ADS or ODM. The success of this approach has already been prototyped with ADS.

Pro: Leveraging the LLM as a knowledge extraction tool enables the extraction of rules and an underlying data ontology. The extracted rules, once reviewed by a human, can automate decisions with traceability and determinism. A promising path to ease knowledge extraction and reach automation with a decreased TCO.

Con: This approach requires an efficient prompt chain or a fine-tuned model to achieve efficient and reliable extractions of the logic, regardless of the business domain and phrasing of the business policy. It also necessitates skills and tooling to efficiently extract automation knowledge. Additionally, companion tools need to be developed to validate and maintain automation assets in sync when source documents change or based on operational feedback, while ensuring a smooth end-to-end user experience.

5. Rules to bring reliable reasoning in a chatbot

Description: In this approach, a Large Language Model (LLM) is used to drive the conversational experience, handling Natural Language Processing (NLP) tasks. The LLM delegates to a rule-based decision engine to apply business decisions.

This pattern requires the chatbot to recognize, during the conversation, when to trigger a decision service. The chatbot guides the dialog to provide context and invokes the rule-based decision engine when all input parameters are set. The decision engine returns output parameters, which are then restituted in the conversation through Natural Language Generation (NLG).

IBM is actively working on incorporating this pattern to bring decision-making capabilities into Watson Orchestrate[6]. Additionally, clients can already develop tools in open-source frameworks like LangChain to invoke rule-based decisions from a bot as shown in this ODM with LangChain post.

Figure 5. Rules to bring reliable reasoning in a chatbot

Pro: This approach allows the chatbot to benefit from conversational user experience while delegating corporate reasoning to dedicated and deterministic engines.

Con: Implementing this approach requires seamless integration to detect when a decision needs to be triggered and delegated, manage the context, and handle error cases at the frontier between unstructured and structured data realms. Challenges may include dealing with different data formats and incomplete context information.

Summary

In this article, we explore the transformative power of Language Models (LLMs) in natural language processing and their potential for corporate decision automation. While LLMs offer impressive capabilities, they lack reliable and repeatable reasoning skills to meet strict decision-making requirements. To bridge this gap, the blog introduces five innovative approaches that combine LLMs with rule-based reasoning engines.

These patterns include invoking LLMs before or during rule execution, for processing plain text in complement of structured data, rules driving text processing, LLM-powered extraction of business rules from policies, and using rule-based decision services in conversational bots.

By blending neuronal and symbolic AI, these composite AI patterns aim to inspire new usages and help enterprises achieve the best of both worlds in business automation.