Langchain agents and function calling using Llama 2 locally
Code can be found here — https://github.com/SandyShah/langchain_agents/tree/main
Well, this was tough and pretty late at night here. I’m not going to delve into many details of the implementation. You can find the reference code on GitHub and take it forward from there. I’ll share some of my experiences and snippets.
In this post, I’m going to recount my first experience in developing agents and friends. It wasn’t as easy as I thought. One can also call it function calling. OpenAI’s ChatGPT, as we all know, is limited in knowledge by the training dataset it had. So, we have RAG — Retrieval-Augmented Generation, in which we can use pre-trained models and provide them with our documents or context to generate relevant and latest results. Similarly, we have agents — in this, we use LLMs to translate natural language queries to some action using functions and APIs. For example, one can have a wiki searcher function, and based on the user’s query, the LLM will generate arguments and pass them to the wiki search, get the results, and then again, the LLM will use the wiki results and our query to provide relevant answers. You can have agents for flight booking/tracking, analyzing mails and news, and anything and everything. The new device, Rabbit R1, has some of these features. It uses various agents in the backend to take actions based on our natural language queries.
Related posts that I have written —
Enhancing Retrieval Augmented Generation with — ReRanker and UMAP visualization: Llama_Index and Llama 2
Exploring RAG Implementation with Metadata Filters — llama_Index
PandasDataFrame and LLMs: Navigating the Data Exploration Odyssey
Advance RAG — Query Augmentation using Llama 2 and LlamaIndex
In my last article, I demonstrated how we can use codeLlama to interact with our data in a pandas dataframe in natural language. Agents are a step further, and to start exploring, I took up the following tasks:
- I have a function that will go to my blog post at https://sandeeprshah.blogspot.com/, use the user-given keywords (use the blog post search option), and get the titles of relevant articles.
- I wanted to connect this with another agent, but things weren’t smooth sailing. In the second agent, I provide it with a link, and the agent just opens that post in a new window.
Now, the best part is that the LLM or our chatbot has access to both the functions all the time, and based on our query, it will decide which function to run. Wait, things get more interesting; it will extract relevant keywords or links based on the query and pass them as arguments to those functions. I have used llama 2–7B. It has been decent with the first call to the functions, but the way the tools and agents have been developed in Langchain, it can make multiple calls, and I did struggle with it.
For example, once I had the relevant blog post articles, I wanted the search to stop, but at times it will generate a random link from the article titles and try to open it. So there is tons and tons of scope for improvement.
Another thing that I will come back to is passing multiple arguments. The current implementation can pass only one argument, and also, the current implementation is outdated. It may take me a few weeks to really get around this function call and agents and come back with more use demonstrations, which are versatile and robust.
Below are some of the code snippets —
Defining functions and converting them to tools —
class BlogPosts(BaseModel):
"""Provides list of all the relevant blog posts and links based on the user query input."""
query: str = Field(description="list of key words to search the blogpost for.")
def get_post_names(query):
"""Provides list of all the relevant blog posts and links based on the user query input.
Output can have more than one post titles."""
# striping off unwanted text if any
query = query.replace('search terms', '')
query = query.replace('keywords', '')
query = query.replace(':', '')
query = query.replace("'", '')
posts=[]
driver = webdriver.Chrome() # Update this with your chromedriver path
blog_url = "https://sandeeprshah.blogspot.com/"
blog_url = blog_url+"search?q="+query
# Open the blog website in the browser
driver.get(blog_url)
time.sleep(5) # Wait for 5 seconds to let the page load
# Find all the search results (assuming search results are represented as links)
search_results = driver.find_elements(By.CSS_SELECTOR, 'h3')
# Extract and print the titles of the search results
print("Titles and links of blog posts:")
for result in search_results:
post_title = result.text
post_link = result.find_element(By.CSS_SELECTOR, 'a').get_attribute('href')
post_title_link = post_title+":"+post_link
posts.append(post_title)
# Close the browser
driver.quit()
return {"response": posts}
get_post_names_tool = Tool(
name="get_posts",
func=get_post_names,
description="Provides list of all the relevant blog posts and the links based on the user query input.",
args_schema = BlogPosts
)
----------------------------------------------
class SinglePost(BaseModel):
"""Given a blog title - find and open the blog in new browser window."""
link: str = Field(description="link of the blogpost to open in new window")
def open_blog_post_new_window(link):
"""given link for a blog article - open it in new window"""
# striping off unwanted text if any
link = link.replace('search terms', '')
link = link.replace("'", '')
posts=[]
driver = webdriver.Chrome() # Update this with your chromedriver path
driver.get(link)
time.sleep(5) # Wait for 5 seconds to let the page load
return {"response": "Post opened"}
open_blog_post_tool = Tool(
name="open_blog_post_new_window",
func=open_blog_post_new_window,
description="given link for a blog article - open it in new window",
args_schema = SinglePost
)
-----------------------------------------------------
tools = [
get_post_names_tool,
open_blog_post_tool
]First, you define the function and its parameters, and then you bundle it all up in ‘tools’. Description plays a crucial role in understanding which function will be called and also what arguments are taken by it. You need to play around with the description a lot. Then, you define the agent as shown below.
PREFIX = "<<SYS>> You are smart that selects a function from list of functions based on user queries.\
Run only one function tool at a time or in one query.<</SYS>>\n"
agent = initialize_agent(
tools,
llm,
agent="zero-shot-react-description",
verbose=True,
max_iterations = 4,
agent_kwargs={
'prefix': PREFIX,
# 'format_instructions': FORMAT_INSTRUCTIONS,
# 'suffix': SUFFIX
}
)
A = agent.run("Fetch me titles of blog posts relevant to keywords: 'marathon training'. \
I just want the list of titles.\
DO NOT OPEN ANY LINKS.")
In the above image — you can see I am getting outputs twice. Once it fetched a long list of titles and then it ran something on top of it and gave just two titles for it. This is something I still struggle to understand and fine tune. Working with open source — I always say I have to baby sit and hand hold using prompts but here I will have to play around a lot with function description, parameter passing and lot more. Below, you can see that once the post is opened, the LLM doesn’t perform any further actions.
BTW — there have been tons of new open source model releases and I am trying my best to stick with Llama 7B and that family as much as possible. New models don’t always work seamlessly with current setup and requires lots of time figuring out the dependencies and all — so I am sticking with llama family for now.
Excited to see what possibilities lie ahead with agents? Delve deeper into the realm of GenAI and LLMs, and let’s explore the endless potentials together! Share your journey and successful projects; your experiences could inspire and drive innovation in ways we never imagined. Connect with us, and let’s embark on this exhilarating journey together.