The future of finance: How LLMs are changing the game
Introduction
The integration of LLMs into financial services is not just an incremental step but a leap toward redefining the financial sector’s future. This advancement provides opportunities — from automating routine tasks and analyzing large datasets to detecting fraudulent activities and providing real-time customer support — making financial services more intuitive, personalized, and accessible than ever before.
Mastercard, the second-largest payment technology corporation worldwide, has recently taken advantage of this breakthrough by launching its own generative AI model to detect fraudulent activities saying it can boost fraud detection by up to 300%.
Given the competitive advantage and efficiency LLMs offer, in this blog, we’ll explore how LLMs work and navigate through possible use cases of leveraging LLMs to your advantage in the financial sector be it an individual or a large-scale corporation.
Understanding the landscape of LLMs in finance
At their core, LLMs are advanced artificial intelligence systems trained on vast amounts of text data. They can understand, generate, and interact with human language in a way that is remarkably coherent and contextually relevant. This capability allows LLMs to perform a variety of language-based tasks, from answering questions and generating reports to understanding complex financial regulations.
In the realm of finance, the implications of LLMs are profound. The industry heavily relies on data and textual information, from market reports and financial news to client interactions and regulatory documents. LLMs can navigate this sea of information efficiently, offering insights and automation opportunities that were previously unattainable.
Here are some potential applications of LLMs in the finance sector:
Automated customer service
LLMs can power chatbots and virtual assistants to handle customer inquiries, account management, and basic advisory services. This not only improves efficiency but also enhances the customer experience by providing round-the-clock service.
Fraud detection and compliance
By analyzing transaction patterns, communication, and regulatory texts, LLMs can identify suspicious activities and ensure compliance with financial regulations, helping financial institutions mitigate risks and avoid hefty penalties.
Financial analysis and reporting
Public companies issue board reports that help shareholders better understand their recent performance and their plans moving forward. LLMs can effectively analyze large swaths of these reports, teasing out industry-wide trends or broader movements individual analysts may miss.
Personalized financial advice
Leveraging LLMs, financial services can provide personalized advice based on an individual’s transaction history, financial goals, and market conditions, democratizing access to financial expertise.
Contract and document analysis
In finance, where contracts and documentation play a critical role, LLMs can expedite the review and extraction of pertinent information, streamlining processes like loan approvals and due diligence.
Real-world examples: LLMs in finance
The adoption of LLMs in finance is not just theoretical; numerous organizations are already reaping tangible benefits from their integration. Let’s look at a few cases below:
Automated Customer Support at JPMorgan Chase
JPMorgan Chase, one of the largest banks in the United States, has integrated LLM-driven chatbots into their customer support framework, the bank has significantly reduced response times and increased customer satisfaction rates.
The chatbots are capable of handling a wide range of inquiries, from transaction queries to account management issues, allowing human agents to focus on more complex customer needs.
They also launched their own LLM, docLLM which is designed for working with documents that have complex layouts by taking into account both, textual semantics and spatial layout hence, allowing efficient and automated document processing, and analysis which streamlines document handling and decision making.
Enhanced fraud detection at PayPal
PayPal has employed LLMs to bolster its fraud detection capabilities. By analyzing transaction patterns and user communications, the LLMs can identify anomalous behaviors that may indicate fraudulent activity. This proactive approach has enabled PayPal to mitigate risks and reduce financial losses due to fraud.
Personalized investment advice with wealthfront
Wealthfront, an automated investment service, leverages LLMs to provide personalized financial advice to its clients. By analyzing individual financial goals, risk tolerance, and market conditions, the LLMs tailor investment strategies to each client’s unique needs, making sophisticated investment advice accessible to the average person.
Best practices for LLMs in action
LLMs can process and analyze vast datasets more quickly and accurately than traditional methods, thus offering improved security measures and customer trust.
However, the speed and continuity at which new data is generated in the financial sector requires regularly updating LLMs with this data to keep the models accurate and relevant. Here are a few best particles derived from these real-world case studies helping us implement these LLMs in the financial industry:
Data quality and privacy
It must be ensured that the training data is high-quality, representative, and respects user privacy. Strict adherence to data protection regulations and ethical guidelines must be ensured when implementing LLMs in financial services.
Continuous learning
Regularly update the LLM with new data and feedback to maintain relevance and accuracy. The LLM solutions must grow with your business, accommodating increased demand and expanding data inputs.
User-Centric Design
Personalization is key to customer satisfaction in financial services. Focus on the user experience to build trust and ensure the solutions meet customer needs.
Regulatory Compliance
Adhere to financial regulations and standards, especially concerning data handling and decision-making processes.
By learning from these real-world applications, financial institutions can navigate the complexities of implementing LLMs more effectively, leading to enhanced operational efficiencies, improved customer experiences, and innovative service offerings.
W&B for your finance LLM projects
Weights & Biases (W&B) offers a comprehensive suite of machine-learning tools designed to enhance the development, monitoring, and optimization of LLMs in finance. The platform stands out for its user-friendly interface and robust features, making it an indispensable tool for financial analysts, data scientists, and researchers working on LLM projects. Here’s an overview of W&B’s features that are particularly beneficial for LLM applications in the finance sector:
W&B excels in experiment tracking, enabling users to log, compare, and visualize different models and their performances. In the context of fine-tuning LLMs requires careful adjustment of hyperparameters, which can significantly impact model performance.
W&B’s hyperparameter-tuning capabilities allow users to easily identify the most effective configurations, saving time and resources while enhancing model accuracy. Moreover, financial datasets are dynamic and can change rapidly. W&B provides dataset versioning tools that help users manage and track changes in their data over time. This feature ensures that LLMs in finance are trained on the most up-to-date and relevant data, leading to more accurate and timely insights.
W&B integrates seamlessly with popular machine learning frameworks and tools, making it easy to incorporate into existing workflows. This integration capability further fosters collaboration among team members by enabling them to share findings, models, and experiments easily.
Building a financial fraud identifier using an LLM
Now, let’s walk through a Python tutorial where we build a financial fraud identifier by classifying filing data as fraudulent or not using an LLM.
We’ll utilize the pre-tained GPT 3.5 on Financial Filings Statement Data to classify the filings and then leverage this pre-trained model for fine-tuning. This purpose is to demonstrate how financial organizations can automate fraud detection to minimize undetected fraudulent filings by leveraging LLMs.
1. Installing the necessary packages
!pip install openai
!pip install wandb
!wandb login2. Importing our libraries
import pandas as pd
import os
import json
import openai
import wandbReading the dataset and setting up W&B
We are using the dataset containing financial filings from various companies submitted to the U.S. Securities and Exchange Commission (SEC). The dataset consists of 85 companies involved in fraudulent cases and an equal number of companies not involved in fraudulent activities. We read the dataset using pandas and set up a new wandb run to log the model’s predictions.
df = pd.read_csv('/content/kaggle/Final_Dataset.csv')
wandb.init(project='openai_financial_fraud', name='LMM')4. Setting up the OpenAI API key
from IPython.display import Markdown, display
os.environ["OPENAI_API_KEY"] = 'add your openai API key'
openai.api_key = os.environ["OPENAI_API_KEY"]5. Evaluating performance using the pre-trained model
We first use the pre-trained model: ‘gpt 3.5 turbo’ to categorize whether the fillings are fraudulent or not. Since the fillings have a large context we condition the data to take only the first 300 tokens and retrieve the output for 10 samples. For this, we first define a function ‘score_review’ and set the content for the system to act as a financial assistant to categorize the financial fillings. The results of the model will be discussed at the end of the tutorial.
df['Fillings'] = df['Fillings'].apply(lambda x: ' '.join(x.split()[:300]))
def score_review(review):
response = openai.chat.completions.create(
messages=[
{"role": "system", "content": "You are a helpful financial assistant who classifies financial filings into two categories: fraudulent or not fraudulent."},
{"role": "user", "content": review},
],
model="gpt-3.5-turbo", temperature=0,
max_tokens=60
)
return response.choices[0].message.content
df['pretrained_results'] = df.iloc[160:].apply(lambda row: score_review(row['Fillings']), axis=1)6. Converting data to JSON for fine-tuning
To fine-tune the model on a given dataset, we first need to format the data in a way, such that each row is a JSON object.
output_file_path = 'output_data.jsonl'
with open(output_file_path, 'w') as file:
for index, row in df_downsampled.iterrows():
json_object = {
"messages": [
{"role": "system", "content": "You are a helpful financial assistant who classifies financial filings into two categories: fraudulent or not fraudulent."},
{"role": "user", "content": row['Fillings']},
{"role": "assistant", "content": str(row['Fraud'])}
]
}
file.write(json.dumps(json_object) + '\n')7. Fine-tuning the model
After formatting the data we upload it for fine-tuning using the Files API. Once the data is uploaded successfully, the generated training file ID is used to initiate the fine-tuning process.
from openai import OpenAI
client = OpenAI()
client.files.create(
file=open("output_data.jsonl", "rb"),
purpose="fine-tune"
)client.fine_tuning.jobs.create(
training_file="file-0uPDZaii1km0Uf6D9KwA5Klf",
model="gpt-3.5-turbo-1106")8. Evaluating the performance using our fine-tuned model
We use the fine-tuned model to evaluate the results for the same 10 samples used for the pre-trained model.
results = []
def score_review(review, original_score, old_score):
response = openai.chat.completions.create(
messages=[
{"role": "system", "content": "You are a helpful financial assistant who classifies financial filings into two categories: fraudulent or not fraudulent."},
{"role": "user", "content": review},
],
model="ft:gpt-3.5-turbo-1106:personal::93qxlK2g", temperature=0,
max_tokens=60
)
results.append({
"Fillings": review,
"Original_Label": original_score,
"pretrained_model_prediction": old_score,
"finetuned_model_prediction": response.choices[0].message.content,
})9. Logging the output to W&B
The resulting prediction and the corresponding fillings along with the original prediction are logged to wandb for efficient comparison of the results. This helps us to compare different predictions when using different models by storing the version of each.
df_results = pd.DataFrame(results)
wandb.log({"results_table": wandb.Table(dataframe=df_results)})10. Evaluating the results
The table below shows two examples taken from the logged wandb table along with the predictions for the pre-trained and fine-tuned model. The results show that the fine-tuned model is more accurate than the trained one, and the predictions are almost in the same format as the original prediction format.
This shows that fine-tuning not only learns the context to perform the base task but also learns the format in which to generate predictions. The results can be further improved by fine-tuning on more samples, which were only 10 in this scenario for demonstration purposes.
runs.summary["results_table"]