Build AI Agents with LangChain and OpenVINO™
Author: Ethan Yang
Normally, LLMs are limited to the knowledge on which they have been trained and the additional knowledge provided as context, as a result, if a useful piece of information is missing the provided knowledge, the model cannot “go around” and try to find it in other sources. This is the reason why we need to introduce the concept of Agents.
The core idea of agents is to use a language model to choose a sequence of actions to take. In agents, a language model is used as a reasoning engine to determine which actions to take and in which order. Agents can be seen as applications powered by LLMs and integrated with a set of tools like search engines, databases, websites, and so on. Within an agent, the LLM is the reasoning engine that, based on the user input, can plan and execute a set of actions that are needed to fulfill the request. Unlike traditional artificial intelligence, agents can actively think and call tools to accomplish a given goal step by step.
The above diagram shows the role of LLM in an AI agent workflow. LLM will split the user’s request into a sequence of tasks, and call (Action) different external tools (Tools) according to the order of these tasks. With the results obtained from tools (Observation), the agent will plan the next action, and then give final feedback when the user’s needs are met. Let’s take a look at how to implement this process with OpenVINO™ and LangChain through a simple example.
Create a tool
According to LangChain’s official tutorial, first, we need to define a set of tools for the agent to call. The official also gives a simple example of how to define a basic set of arithmetic functions by @tool decorators:
from langchain_core.tools import tool
@tool
def multiply(first_int: int, second_int: int) -> int:
"""Multiply two integers together."""
return first_int * second_int We can create multiple tools at the same time, and an agent will independently select the most suitable tool according to the user’s needs and obtain the results. In addition to creating a customized tool function, some ecosystem partners of LangChain have also pre-defined many useful tools for us, such as using Wikipedia for keyword searches or using ChatGPT service to complete more challenging tasks.
Create a Prompt Template
The second step is to create a Prompt Template, also known as Prompt Engineering in many LLM applications. The aim of it is to activate the LLM’s ability to plan and action. Currently, the most commonly used templates for local agents include Function Call, Self-ask, and ReAct. Since the Function Call capability needs to be enabled during model training, it will not be discussed here.
- Self-ask: Self-ask proposes a prompt paradigm that breaks down a question into sub-questions, where LLM asks itself whether the question can be rewritten/disassembled into a simpler sub-question or answers. After that search tool can be called to get the answer, and then LLM will continue to ask itself until the final answer is obtained.
- ReAct: As the name suggests, ReAct can be divided into two steps, Reason and Action. Reason generates the steps of analysis, and Action generates the request to call the tool. These two steps are alternated until the final result is obtained.
In Self-ask mode, self-asking is not only an inference step but also a query request for a search tool, while in ReAct, the call request for a tool follows the inference step, which can better support more tools other than search.
In this project, taking ReAct as an example, we can see that the following Prompt Template builds a chain with Thought, Action, Observation, and Final answer. This process of thought is required to be repeated several times until the LLM has enough information from the tool to support it in fulfilling user requests.
prompt = PromptTemplate.from_template(
"""Answer the following questions as best you can. You have access to the following tools:
{tools}
Use the following format:
Question: the input question you must answer
Thought: you should always think about what to do
Action: the action to take, should be one of [{tool_names}]
Action Input: the input to the action
Observation: the result of the action
... (this Thought/Action/Action Input/Observation can repeat N times)
Thought: I now know the final answer
Final Answer: the final answer to the original input question
Begin!
Question: {input}
Thought:{agent_scratchpad}"""
)Create an LLM instance
At present, OpenVINO™ has been integrated into the HuggingFace Pipeline which is one of the LLM components in LangChain. Developers can directly create an LLM instance based on OpenVINO by setting the backend of HuggingFace Pipeline, and call it like other LLM instances in LangChain, where model_ id can be a HuggingFace model ID, a local PyTorch or OpenVINO model path.
ov_llm = HuggingFacePipeline.from_model_id(
model_id=model_path,
task="text-generation",
backend="openvino",
model_kwargs={"device": device.value, "ov_config": ov_config},
pipeline_kwargs={"max_new_tokens": 1024},
)More information about the OpenVINO LLM component and how to use it can be found in:
https://python.langchain.com/v0.1/docs/integrations/llms/openvino/
Build and run the agent
In this example, we take the “Intel/neural-chat-7b-v3–1” large language model as the agent. After creating the LLM instance, tools, and prompt template, we can use the agent-related APIs in LangChain to build a local agent service very quickly.
output_parser = ReActSingleInputOutputParser()
agent = create_react_agent(ov_llm, tools, prompt, output_parser=output_parser)
agent_executor = AgentExecutor(agent=agent, tools=tools, verbose=True)Since LangChain’s default parser of ReAct output cannot well identify the generated results from HuggingFace Pipeline, we need to define a new custom_output_parser.py script to convert the original output of LLM into Action, such as how to match the input data of a tool function. Taking a simple mathematical operation as an example, we can see that after understanding the user’s question, the agent splits it into multiple actions, and combines the observations from multiple actions to get the final answer.
Question: Take 3 to the fifth power and multiply that by the sum of twelve and three
Thought: We need to exponentiate 3 to the power of 5, then multiply the result by the sum of 12 and 3
Action: exponentiate
Action Input: base: 3, exponent: 5
Observation: 243
Action: add
Action Input: first_int: 12, second_int: 3
Observation: 15
Action: multiply
Action Input: first_int: 243, second_int: 15
Observation: 3645
Thought: I now know the final answer
Final Answer: 3645In addition, with the help of a pre-defined Wikipedia tool in LangChain, we can also build an Agent tool for Wikipedia as the following example. This agent can get external knowledge through searching Wikipedia.
Conclusion
A LLM cannot do everything, but it can be smart enough to find a suitable helper for a specific task. As a gateway to artificial general intelligence (AGI), the agent is like an intelligent entity that perceives the world like a human, makes decisions, interacts with others, and performs actions. It is believed that with the support of OpenVINO™, a fully local AI agent service will become possible.
Resources
- Example of RAG based on LangChain and OpenVINO:
https://github.com/openvinotoolkit/openvino_notebooks/tree/latest/notebooks/llm-rag-langchain
- Example of an agent based on LangChain and OpenVINO:
https://github.com/openvinotoolkit/openvino_notebooks/tree/latest/notebooks/llm-agent-langchain
Notices & Disclaimers
Intel technologies may require enabled hardware, software, or service activation.
No product or component can be absolutely secure.
Your costs and results may vary.
© Intel Corporation. Intel, the Intel logo, and other Intel marks are trademarks of Intel Corporation or its subsidiaries. Other names and brands may be claimed as the property of others.
