Financial Market Analysis using Large Language Models
Table of Contents
- Why Financial Market Analysis is needed?
- How ChatGPT can help, and why it cannot help 100 %
- Meet Retrieval Augmented Generation and AnswerMe API
- Can AnswerMe do better ?
- Conclusion
Why Financial Market Analysis is needed ?
Financial Market Analysis is by no means an easy task. It needs a combination of domain experience , trustworthy data and a set of intelligent tools. It is essential for financial analysts, investors of all types (short term, long term , hedge and growth investors, individual and institutional ones).
There are many approaches for financial market analysis : Quantitative and Fundamental. In summary, Quantitative approaches are based on applying models and algorithms over the “historical” data for stocks and securities. These models predict the price movements based on historical data characteristics.
On the other hand, Fundamental Analysis focuses on “extracting” relevant information from “trustworthy” documents (company reports, news from journals , etc..) and try to find, in depth, the core factors that affects a company’s performance. For example, it focus on understanding the current cash-flow status for the company, plans and investment to grow in different markets, different technological advances that the company had or plans to achieve etc…
How ChatGPT can help ?
As it is quite known, ChatGPT is one of the most advanced Large Language Models (LLMs) in the market. Basically, LLMs works by generating relevant text based on some “seed” inputs, which are usually “questions”. LLMs are Generative Models that are trained on large amount of text data. Hence, when receiving a seed text , like a question, it starts generating relevant text, which can be answers.
To connect the dots, we will start to ask ChatGPT a basic financial market question based some “Geographical Market Segmentation” , the question can be :
How was the financial market performance in US, Europe and Japan in 2023?
Let’s feed this question to ChatGPT 3.5 and see results:
Note : we will do this in two attempts to as results are not the same in each generation
Second Attempt
As we can see, ChatGPT generates some answers , but they don’t have real information. For example , in the first attempt , it provides like an “anatomy” to the US, Europe and Japan economies like what are the index funds the acts as indicator for each economy and what factors impact it. However, It provides no specific information about the economic performance.
Can RAG and AnswerMe API do any better ?
If we look at the second attempt results , we can find a “hint” that ChatGPT gave us to get better results : we need to access the latest data.
Based on this fact , let me introduce you to out AnswerMe API and its backbone technology : Retrieval Augmented Generation(RAG).
RAG is one of two typical approaches for “steering” a LLM model to generate answers given some context, which are Fine-Tuning and Retrieval Augmented Generation(RAG) . Fine-Tuning is about re-training the last few layers of the model given the documents from some context. However, RAG is a more lightweight approach that relies on blending Information Retrieval and Generative Models approaches to generate text given some context.
AnswerMe is an API , which is deployed on Rapid-API Marketplace, and developed based on RAG approach. It works in a two step process :
1. User uploads set of documents to be processed and blended with an LLM model
2. User sends requests with question, then the API back-end runs the question through the LLM + embedded text to generated answers given the learnt context from the uploaded documents. For more information see this tutorial.
Given this background, let’s see how we can answer the above question using AnswerMe
Can AnswerMe do Better ?
Before jumping into details, it’s advisable to get yourself familiarized with Rapid-API using this tutorial. Also, follow this article to create and account with valid API-Key.
First we try to find a good , trustworthy PDF document about the market in 2023. We have arbitrarily selected this one.
Based on the steps introduced in this AnswerMe tutorial, we make a two step process to make the system read for receiving questions :
Step 1 : Upload the file
import requests
url = "https://answer-me.p.rapidapi.com/psurl"
querystring = {"filename":"2023 global market outlook full report.pdf"}
headers = {
"X-RapidAPI-Key": "xxx",
"X-RapidAPI-Host": "answer-me.p.rapidapi.com"
}
response = requests.get(url, headers=headers, params=querystring)
print(response.json())(See the Endpoint documentation and examples here )
Then we wait for approx. 60 seconds till the process for text chunking, embedding and indexing is finished.
Step 2 : Send questions and generate answers
import requests
url = "https://answer-me.p.rapidapi.com/answerme"
querystring = {"question":"How was the financial market performance in US,
Europe and Japan in 2023?",
"filename":"2023 global market outlook full report.pdf"}
headers = {
"X-RapidAPI-Key": "xxx",
"X-RapidAPI-Host": "answer-me.p.rapidapi.com"
}
response = requests.get(url, headers=headers, params=querystring)
print(response.json())After the call to question-answering API we get this result :
{'question': 'How was the financial market performance in US, Europe and Japan
in 2023?', 'answer': "Based on the given context, we can gather some
information about the financial market performance in the US, Europe,
and Japan in mid-year 2023:\n\n
1. Equities:\n
- Equities have limited upside potential with recession
risk on the horizon.\n - Non-U.S. developed equities are cheaper than
U.S. equities, but there is a neutral preference until the Fed becomes
less hawkish and the U.S. dollar weakens significantly.\n
- Quality stocks, which have low debt and stable earnings growth,
are relatively cheap compared to the rest of the market.\n\n
2. Emerging Market Equities:\n
- Emerging market equities have underperformed developed markets,
despite the weaker U.S. dollar, which usually triggers outperformance.\n
- Concerns about China's economy have been a headwind, and these worries
are unlikely to lift over the near-term. A neutral stance is warranted for
now.\n\n
3. High Yield and Investment Grade Credit:\n
- High yield spreads are below their long-term average, while investment
grade credit spreads are close to their long-term averages.\n
- The poor cycle outlook is a challenge, with rising default rates as U.S.
recession probabilities increase.\n\n
4. Government Bonds:\n
- Government bond valuations are increasingly attractive, with U.S., UK, and
German bonds offering reasonable value.\n
- Japanese bonds are still expensive due to the Bank of Japan's yield
limit policy.\n - The U.S. yield curve may steepen in the coming months
after the Fed completes raising interest rates and markets start looking
toward monetary easing.\n\n
5. Real Assets:\n
- REITs (real-estate investment trusts) have more attractive valuations
compared to infrastructure and global equities.\n
- REITs should perform well when interest rates fall, as real estate
fundamentals appear healthy.\n
- Commodities, including oil, face headwinds from the lackluster Chinese
economy and the poor outlook for infrastructure and construction activity.
\n\n
6. U.S. Dollar:\n
- The U.S. dollar has trended lower over the past month as investors
speculate that the Fed is nearing the end of rate hikes.\n
- It could weaken further if markets become confident that a recession can
be avoided.\n
- The Japanese yen is considered attractive from a cycle, value, and
sentiment perspective, and many investors expect further yen weakness.\n\n
It's important to note that the given context does not specifically mention
the overall performance of the financial markets in the US, Europe, and Japan
in 2023. It provides insights into different asset classes and factors
influencing their performance."}Looking at AnswerMe result, we can find some more relevant pieces of information , more than ChatGPT. We can see a breakdown to a set of different financial asset-classes (Equities, Bonds, REITs, US. Dollar, Japanese yen etc…). Also it provides more insights about these assets, for ex. it mentions that government bonds are still attractive assets for buyers in US,UK , Germany.
Conclusion
Despite that the information provides by AnswerMe is not complete, however they are way better than ChatGPT 3.5, thanks to RAG approach that makes seamless integration between LLM and Information Retrieval.
There are several ways to improve results :
- Feed LLM model with more comprehensive documents : in our demo we used a small document (30 pages) , but our API supports till 500 pages documents in free plan.
- Prompt Engineering : The are several well-engineered prompts for several domains. For more information, look at this paper
- Tuning RAG parameters : like the size of document chunks ,length of overlap and tune policy of selecting top documents to blend into the model inference process. For more info, look at this great RAG tutorial
- Tuning LLM model : one simple parameter, yet effective , is the model “temperature” which controls the “randomness” of the model, and makes it explore different results each time.
If temperature is low, the model is quite rigid and generates almost same results each call, and doesn’t explore more possibilities. This is good when we want a very rigid and robust model. If we are in the exploration phase, we increase the temperature to make model more “exploratory”. However this can lead sometimes to inconsistent results and can lead to LLM-hallucination in severe cases.
For more info about LLM parameters, look at this article.
For more information about how to use our AnswerMe API , check out comprehensive tutorial.
If you need support regarding the API or for customized LLM and Generative Models solution , send us an email
Also Follow us on twitter for more similar LLM and Generative AI contents
