Effortless Company Research Using Open Source — Using Llama Index, Huggingface Embeddings & Llama 2 LLM On News

Photo by Sébastien Goldberg on Unsplash

Much has been written about large language models in the past few months. While large language models, or LLMs as popularly known, have taken the world by storm, people still need help finding good applications to justify their popularity.

One of the major areas that LLMs are being enhanced for is to use them on custom data. ChatGPT contains a cutoff of September 2021, so it has limited knowledge of current events. If we can supply our recent data to LLMs and ask them to make decisions based on that, it will solve many business problems.

Big businesses are also wary about privacy issues and data leakage. Many deal with overly sensitive data, so sending it to a service provider like OpenAI will come with its risks. To overcome that, they are looking for LLM solutions that are open-source and run locally. This gives them the power to run LLMs in their infrastructure, circumnavigating the privacy issues of cloud-hosted LLMs.

Enter LLAMA 2

Llama 2 is one of the foundational models that Meta has released, and it quickly became the world’s favorite open LLM as there is a big company like Meta behind it. It is also suitable for commercial adaptation as its license is pretty liberal. It also can run pretty well on commodity hardware, and its smaller versions with 7 billion and 13 billion parameters do not require that much beefy hardware while having decent performance.

One of the better ways you can use LLAMA is to use a quantized version of its models, which reduces the need for powerful hardware drastically. People have been running LLAMA 2 on their Macbooks without any hassles.

You can go to the LLAMA.cpp repository repository for more information. As the instructions will differ for each type of system and hardware, we will not be covering installing and running LLAMA 2 in this article.

Once Llama 2 has been installed, you can interface it with its Python libraries. For the purpose of this article, we will be using a library called LlamaIndex. This library allows us to build effective LLM applications using various LLMs. However, since we are dealing with sensitive data, we will use a local LLM model such as Llama.

Downloading The News Data

We will deal with news data and ask LLMs questions related to company research. For this, we will use a service called Intell. Intell has nice APIs that we can use to download news according to topics, companies, persons, and more.

You can obtain an API key from them and substitute that in the below code given below. This code will allow us to download a bunch of articles about Tesla from the past year.

import requests
import os
from dotenv import load_dotenv
from datetime import datetime

load_dotenv(override=True)
APIKEY = os.environ['APIKEY']

payload = {
    'token': APIKEY,
    'offset': 1000,
    'startDate': '2022-09-01',
    'endDate': '2023-08-31',
    'organizations': 'Tesla'
}

def fetch_data(keyword=None, next_url=None):
    if next_url == None:
        ## fetch using keywords
 
        payload = {
            'token': APIKEY,
            'offset': 1000,
            'startDate': '2022-09-01',
            'endDate': '2023-08-31',
            'organizations': keyword
        }
        try:
            resp = requests.get('https://app.intell.me/api/v3/feed/', params=payload)
            return resp.json()['result']
        except Exception as e:
            return None
    else:
        try:
            resp = requests.get(next_url)
            return resp.json()['result']
        except Exception as e:
            return None

file_list = []
has_results = True
keyword = 'tesla'
i = 0
next_url = None

while(has_results):
    i = i + 1
    
    if next_url == None:
        resp = fetch_data(keyword)
    else:
        resp = fetch_data(next_url=next_url)

    if resp == None:
        print('=== Exception From API ===')
        has_results = False
        break

    next_url = resp['nextUrl']
    data = resp['data']
    last_date = resp['data'][-1]['date']
    
    for datum in data:
        title = datum['title'].lower()
        content = datum['content']
        date = datum['date']
        
        if keyword in title:
            whole_article = title + '\n' + content
            file_list.append(whole_article)

            if (datetime.strptime(date, '%Y-%m-%d') < datetime.strptime('2023-01-01', '%Y-%m-%d')):
                has_results = False
                break

    print('=== i: {} ==='.format(str(i)))
    print('=== articles: {} ==='.format(len(file_list)))
    print('=== Done till: {}'.format(last_date))

For the final input to the LLM, we will have to write this data to a bunch of text files. These files will be consumed by LlamaIndex and be interfaced with the LLM.

for i in range(len(file_list)):
    file = open('../data/{}.txt'.format(i), 'a')
    file.write(file_list[i])
    file.close()

Using LlamaIndex to interface with Llama 2

To interface with Llama 2, we use the following code. First import the libraries and modules needed.

from llama_index import (
    SimpleDirectoryReader,
    VectorStoreIndex,
    ServiceContext,
)
from llama_index.llms import LlamaCPP
from llama_index.llms.llama_utils import (
    messages_to_prompt,
    completion_to_prompt,
)

We will have to supply the model path. This will typically be a gguf file.

model_path = ‘llama-2–7b-chat/ggml-model-q4_0.gguf’

Finally, we call the Llama.cpp using its python interface.

llm = LlamaCPP(
    model_path=model_path,
    temperature=0.1,
    max_new_tokens=256,
    context_window=3900,
    generate_kwargs={},
    model_kwargs={"n_gpu_layers": 1},
    messages_to_prompt=messages_to_prompt,
    completion_to_prompt=completion_to_prompt,
    verbose=True,
)

You will get a bunch of output which shows the model has been loaded

Let’s start a prompt

response = llm.complete(‘The president of India is ‘)

You can print the response also

print(response.text)

It’s giving me an output like this:

I apologize, but I cannot provide you with the current President of India as it is a constantly changing piece of information that I do not have access to in real-time. The President of India is elected through a democratic process and serves a five-year term. However, I can suggest some ways for you to find out who the current President of India is. You can check official government websites, news sources, or consult with a reliable reference book or encyclopedia.

Company Research From News Using Llama 2

Let’s load up the news we saved earlier and ask some questions based on them.

from llama_index import download_loader
SimpleDirectoryReader = download_loader('SimpleDirectoryReader')
loader = SimpleDirectoryReader('../data', recursive=True, exclude_hidden=True)
documents = loader.load_data()

from llama_index.node_parser import SimpleNodeParser
parser = SimpleNodeParser.from_defaults(chunk_size=1024, chunk_overlap=20)
nodes = parser.get_nodes_from_documents(documents)
from llama_index.embeddings import HuggingFaceEmbedding
embed_model = HuggingFaceEmbedding(model_name='BAAI/bge-small-en-v1.5')
service_context = ServiceContext.from_defaults(embed_model=embed_model, llm=llm)
index = VectorStoreIndex(nodes, service_context=service_context, show_progress=True)

That’s a bunch of code, but simply, we are just loading the news data and embedding it to run semantic queries. Finally, we can use a query engine to query the news and let LLM answer us.

Let’s start with something simple

response = index.as_query_engine().query(‘What’s the status of Tesla autopilot?’)

We get the following response

Based on the provided context information, it appears that Tesla’s Autopilot system is currently under investigation by the California attorney general’s office for safety issues and allegations of false advertising. The investigation is reportedly looking into complaints from customers and former Tesla employees who have been contacted by the office. It is important to note that Autopilot is not a fully autonomous driving system, but rather an advanced driver-assistance system designed to assist drivers with safety and convenience on the road. While it has been subject to numerous probes in the past, Tesla continues to improve and enhance its Autopilot features through software updates over time.

How about something else?

response = index.as_query_engine().query(‘Has the economic slowdown affected Tesla?’)

Yes, the economic slowdown has had an impact on Tesla’s sales growth and margins. According to the article, Tesla’s Q3 guidance shows a business slowdown due to maintenance, and the company produced 920k vehicles in the first two quarters but is guiding for 1.8 million vehicles for the year, implying a decline in production. Additionally, the company’s headline number showed a 47% YoY increase, but the article expects this to be the last quarter where the company pushes such growth numbers with next quarter being more towards neutral. The impact of margins is seen in the company’s financials, where despite revenue being higher than its peak, EBITDA margins dropped by 3.5%. Furthermore, the company’s FCF was $1 billion, giving the company a FCF yield of <0.5% and its growth in cash is even smaller. That’s despite the company’s capex decreasing quarter over quarter.

Finally, is it a good time to invest in Tesla stock?

response = index.as_query_engine().query(‘Is it a good time to invest in Tesla stock?’)

Based on the provided context, it seems that Tesla is poised for growth in 2023, with management expecting to exceed vehicle production targets and seeing a record number of orders so far in January. However, it’s important to note that the company’s delivery growth has slowed into the end of 2022, and there are political issues plaguing the brand. Despite these challenges, the author of the article suggests that it may be a good time to invest in Tesla stock due to its undervaluation and potential for significant acceleration in volume growth.

As the author notes, “as goes Tesla, so goes the entire EV industry,” and global search interest in ‘electric vehicles’ has spiked in January after being dormant for the second half of 2022. This suggests that there may be increased demand for EV stocks in general.

However, it’s important to conduct thorough research and due diligence before making any investment decisions. It’s also worth noting that the stock market can be unpredictable, and there are always risks involved with investing in any particular stock. Therefore, it’s crucial

What’s Next?

Here, we have seen how we can use LLMs locally in a privacy-preserving way to do company research. We can fine-tune these LLMs further using specific data to get better results. We will cover such cases in future blog posts.