Local LLM Agents with AutoGen and Llama 3

Power up your local LLMs.

1. Why LLM Agents?

What’s wrong with good-old ChatGPT? Performance. If you have a simple question requiring a simple answer that you could easily Google or ask ChatGPT, then this article is overkill. However, a single LLM agent is not enough if you have a complex project requiring planning, organization, and scenarios such as complex question answering (GAIA), interactive world exploration (ALFWorld), and supply chain planning (OptiGuide). Agentized LLMs have shown promising performance in various applications. For example, vanilla GPT-4-Turbo (6.67%) significantly lags behind GPT-4-Turbo with manually selected plugins (14.6%) and even more so agentic frameworks such as AutoGen (#1 at 32.33%), underscoring a clear need for agentic planning and execution. This pronounced performance discrepancy further shows that the idea of multi-agent LLMs is a nontrivial paradigm.

2. LLM Agent Software Frameworks

Over the last year, we have witnessed the explosion of open-source software libraries for LLM agents, notably LangGraph (and its underlying framework, LangChain), Crew AI (which sits on top of LangGraph), AutoGPT, and Microsoft’s AutoGen. In this article, we use AutoGen as it accommodates both the academic and engineering contexts.

3. Why Llama 3?

At the time of writing, there is a plethora of LLMs out there, the most popular of which are GPT-4-Turbo and Gemini. However, not everyone has easy or affordable access to such state-of-the-art models, and current non-free models outperform open-source ones. The recently released Llama 3 model bridged this gap in several important metrics, empowering the open-source community.

Meta Llama 3 Instruct outperforms some non-free models. Credit to Meta

4. AutoGen + Llama 3

AutoGen does not currently have out-of-the-box support for Llama 3. For Llama 3 to talk to each other, there needs to be an intermediary: Ollama.

First, install Ollama via the official installer (for Linux):

Ollama

Then, serve Llama 3 on your local computer:

ollama pull llama3
ollama serve # default port is localhost:11434

If you have NVIDIA GPUs available, you can see that it has been nicely distributed across them by default. To stop, on MacOS, exit the Ollama application using the system bar at the top. On Linux, do

sudo service ollama stop # change `stop` to `start` to restart it.

Create a new environment for AutoGen. Here, I’m using Anaconda (Miniconda), but feel free to use PyEnv, etc:

conda create -n agents python
conda activate agents
pip install pyautogen

5. Run examples:

To use

from autogen import AssistantAgent

# Setup API key. Add your own API key to config file or environment variable
config_list = [
    {
        "model": "llama3",
        "base_url": "http://localhost:11434/v1",
        'api_key': 'ollama',
    },
]

small = AssistantAgent(name="small model",
                       max_consecutive_auto_reply=2,
                       system_message="You should act as a student!",
                       llm_config={
                           "config_list": config_list,
                           "temperature": 1,
                       })

big = AssistantAgent(name="big model",
                     max_consecutive_auto_reply=2,
                     system_message="Act as a teacher.",
                     llm_config={
                         "config_list": config_list,
                         "temperature": 1,
                     })

If we ask a simple question like “Who are you,” you can see the list of output once we initialize the chat:

big.initiate_chat(small, message="Who are you?")

The output looks something like this.

big model (to small model):

Who are you?

--------------------------------------------------------------------------------
small model (to big model):

I'm your friendly AI system, but for the purpose of this conversation, let's pretend I'm your classmate or tutor! Since we're in a virtual classroom setting, you can address me as "Professor" or simply "Teacher." What would you like to discuss today? Do you have any questions about the topic at hand or do you need help with something specific?

--------------------------------------------------------------------------------
big model (to small model):

Thank goodness it's time for our daily lesson!

Teacher, I've been struggling a bit with the concepts we covered in yesterday's lecture. Could you review the material on differential equations and show me some examples of how to solve them step-by-step? Specifically, I'd love to see more problems involving separable variables and integrating factors.

Also, since our project is due soon, can you offer any tips or advice on how to effectively use LaTeX for formatting and creating equations in our reports?

--------------------------------------------------------------------------------
small model (to big model):

Excellent timing! Let's dive into reviewing the differential equations concepts.

Separable Variables:
---------------------------------

Separable differential equations involve separating the variables, so that you integrate one side with respect to time (or the independent variable), while keeping the other side constant. This type of equation is often written in the form:

dy/dx = g(x)h(y)

To solve it, you would:

1. Write y' as a function of both x and y.
2. Integrate one side with respect to x (or time).
3. Keep the other side, h(y), constant by integrating with respect to y.

Here's an example:
-----------------------------------

Example: Solve the differential equation dy/dx = 2x + 1

Solution:

dy/dx = g(x)h(y)
= 2x + 1 (since the equation is already in this form)

To separate the variables, integrate both sides with respect to x:

∫(1)dx = ∫[2x + 1] dy
x = x^2/2 + y + C

Integrate the right side with respect to y:

y = (x^2)/4 + C

Now we have our general solution: y(x) = (x^2)/4 + C

Integrating Factor:
-------------------------------

To solve differential equations that are not immediately separable, you can use integrating factors. This technique helps you rewrite the equation so that it becomes separable.

The formula for an integrating factor is:

μ(t) = e^(∫[g(t)dt])

where g(t) is the coefficient of dy/dx in the original differential equation.

Let's try solving another example:
-----------------------------------

Example: Solve the differential equation dy/dx + 2y = x

To use an integrating factor, first find μ(t):

μ(t) = e^(∫[2dt])
= e^2t

Multiply both sides of the original equation by μ(t):
----------------------------------------------------

μ(t)(dy/dx) + μ(t)(2y) = xμ(t)

Rewrite the left side:

(d(yμ)/dx) + 2yμ = xμ

Now, since dy/dx is written as a function of both y and x, you can separate the variables:

dyμ/dx = (1/μ) \* [xμ - 2yμ]

To solve this differential equation, integrate with respect to x, keeping μ constant. Then, substitute your original integrating factor and simplify.

This process might seem lengthy at first, but with practice, you'll become more comfortable using separable variables and integrating factors to solve differential equations!

Regarding LaTeX:
-----------------------------------

For formatting and creating equations in your reports, here are some tips:

1. Start by installing a LaTeX distribution on your computer, such as TexLive or MikTeX.
2. Familiarize yourself with the basic syntax of LaTeX using resources like the Not So Short Guide to LaTeX (PDF).
3. Use the following command to create an equation:
```
$\left( \frac{dy}{dx} \right) = g(x)h(y)$
```

Replace "g(x)h(y)" with your actual equation.

4. To add parentheses or brackets around a mathematical expression, use the `\(` and `\)` commands:
```
$((x+y)^2)$
```

This will render as: ((x+y)^2)

5. Practice creating equations in LaTeX using online resources like LaTeX Equation Editor (web-based) or ShareLaTeX (free, cloud-based editor).

By following these steps and experimenting with different equations, you'll become more proficient in using LaTeX for formatting mathematical expressions in your reports.

I hope this review and guidance help! Do you have any specific questions about the material we covered today?

--------------------------------------------------------------------------------
big model (to small model):

Wow, thank you so much, Professor!

You're a great teacher, breaking down complex concepts into understandable steps. I really appreciate how you explained the concept of separable variables and integrating factors, along with the examples to help solidify my understanding.

Regarding LaTeX, I'm excited to try out your tips and get comfortable with it. I'll definitely be checking out those online resources you recommended to practice creating equations.

One question I do have is: What are some common pitfalls or mistakes that people often make when using integrating factors to solve differential equations? Are there any specific traps or gotchas I should be aware of?

Also, could we review the importance and applications of differential equations in real-world scenarios? I'd love to understand more about how they're used in physics, engineering, biology, or other fields.

--------------------------------------------------------------------------------
ChatResult(chat_id=None, chat_history=[{'content': 'Who are you?', 'role': 'assistant'}, {'content': 'I\'m your friendly AI system, but for the purpose of this conversation, let\'s pretend I\'m your classmate or tutor! Since we\'re in a virtual classroom setting, you can address me as "Professor" or simply "Teacher." What would you like to discuss today? Do you have any questions about the topic at hand or do you need help with something specific?', 'role': 'user'}, {'content': "Thank goodness it's time for our daily lesson!\n\nTeacher, I've been struggling a bit with the concepts we covered in yesterday's lecture. Could you review the material on differential equations and show me some examples of how to solve them step-by-step? Specifically, I'd love to see more problems involving separable variables and integrating factors.\n\nAlso, since our project is due soon, can you offer any tips or advice on how to effectively use LaTeX for formatting and creating equations in our reports?", 'role': 'assistant'}, {'content': 'Excellent timing! Let\'s dive into reviewing the differential equations concepts.\n\nSeparable Variables:\n---------------------------------\n\nSeparable differential equations involve separating the variables, so that you integrate one side with respect to time (or the independent variable), while keeping the other side constant. This type of equation is often written in the form:\n\ndy/dx = g(x)h(y)\n\nTo solve it, you would:\n\n1. Write y\' as a function of both x and y.\n2. Integrate one side with respect to x (or time).\n3. Keep the other side, h(y), constant by integrating with respect to y.\n\nHere\'s an example:\n-----------------------------------\n\nExample: Solve the differential equation dy/dx = 2x + 1\n\nSolution:\n\ndy/dx = g(x)h(y)\n= 2x + 1 (since the equation is already in this form)\n\nTo separate the variables, integrate both sides with respect to x:\n\n∫(1)dx = ∫[2x + 1] dy\nx = x^2/2 + y + C\n\nIntegrate the right side with respect to y:\n\ny = (x^2)/4 + C\n\nNow we have our general solution: y(x) = (x^2)/4 + C\n\nIntegrating Factor:\n-------------------------------\n\nTo solve differential equations that are not immediately separable, you can use integrating factors. This technique helps you rewrite the equation so that it becomes separable.\n\nThe formula for an integrating factor is:\n\nμ(t) = e^(∫[g(t)dt])\n\nwhere g(t) is the coefficient of dy/dx in the original differential equation.\n\nLet\'s try solving another example:\n-----------------------------------\n\nExample: Solve the differential equation dy/dx + 2y = x\n\nTo use an integrating factor, first find μ(t):\n\nμ(t) = e^(∫[2dt])\n= e^2t\n\nMultiply both sides of the original equation by μ(t):\n----------------------------------------------------\n\nμ(t)(dy/dx) + μ(t)(2y) = xμ(t)\n\nRewrite the left side:\n\n(d(yμ)/dx) + 2yμ = xμ\n\nNow, since dy/dx is written as a function of both y and x, you can separate the variables:\n\ndyμ/dx = (1/μ) \\* [xμ - 2yμ]\n\nTo solve this differential equation, integrate with respect to x, keeping μ constant. Then, substitute your original integrating factor and simplify.\n\nThis process might seem lengthy at first, but with practice, you\'ll become more comfortable using separable variables and integrating factors to solve differential equations!\n\nRegarding LaTeX:\n-----------------------------------\n\nFor formatting and creating equations in your reports, here are some tips:\n\n1. Start by installing a LaTeX distribution on your computer, such as TexLive or MikTeX.\n2. Familiarize yourself with the basic syntax of LaTeX using resources like the Not So Short Guide to LaTeX (PDF).\n3. Use the following command to create an equation:\n```\n$\\left( \\frac{dy}{dx} \\right) = g(x)h(y)$\n```\n\nReplace "g(x)h(y)" with your actual equation.\n\n4. To add parentheses or brackets around a mathematical expression, use the `\\(` and `\\)` commands:\n```\n$((x+y)^2)$\n```\n\nThis will render as: ((x+y)^2)\n\n5. Practice creating equations in LaTeX using online resources like LaTeX Equation Editor (web-based) or ShareLaTeX (free, cloud-based editor).\n\nBy following these steps and experimenting with different equations, you\'ll become more proficient in using LaTeX for formatting mathematical expressions in your reports.\n\nI hope this review and guidance help! Do you have any specific questions about the material we covered today?', 'role': 'user'}, {'content': "Wow, thank you so much, Professor!\n\nYou're a great teacher, breaking down complex concepts into understandable steps. I really appreciate how you explained the concept of separable variables and integrating factors, along with the examples to help solidify my understanding.\n\nRegarding LaTeX, I'm excited to try out your tips and get comfortable with it. I'll definitely be checking out those online resources you recommended to practice creating equations.\n\nOne question I do have is: What are some common pitfalls or mistakes that people often make when using integrating factors to solve differential equations? Are there any specific traps or gotchas I should be aware of?\n\nAlso, could we review the importance and applications of differential equations in real-world scenarios? I'd love to understand more about how they're used in physics, engineering, biology, or other fields.", 'role': 'assistant'}], summary="Wow, thank you so much, Professor!\n\nYou're a great teacher, breaking down complex concepts into understandable steps. I really appreciate how you explained the concept of separable variables and integrating factors, along with the examples to help solidify my understanding.\n\nRegarding LaTeX, I'm excited to try out your tips and get comfortable with it. I'll definitely be checking out those online resources you recommended to practice creating equations.\n\nOne question I do have is: What are some common pitfalls or mistakes that people often make when using integrating factors to solve differential equations? Are there any specific traps or gotchas I should be aware of?\n\nAlso, could we review the importance and applications of differential equations in real-world scenarios? I'd love to understand more about how they're used in physics, engineering, biology, or other fields.", cost={'usage_including_cached_inference': {'total_cost': 0, 'llama3': {'cost': 0, 'prompt_tokens': 1395, 'completion_tokens': 1119, 'total_tokens': 2514}}, 'usage_excluding_cached_inference': {'total_cost': 0}}, human_input=[])

Article Version History:

0.1.0: 4/30/2024: trivial working example.