AutoGen AI Agent framework for beginners
Best Multi-AI Agent orchestration framework for complex Generative AI applications
Since ChatGPT’s inception, Generative AI has been on a roll. Every month, we are witnessing some ground breaking developments. One of the most useful and interesting being Multi-AI Agent Orchestration where multiple AI-Agents, with different assigned roles, engage with each other to complete a complex task. I’ve already discussed some interesting prototypes in my previous posts like Code Review, Interview system, Debate application, etc. that you can checkout at the bottom of this post
In this post, we will be deep-diving into the most interesting and popular Multi-Agent framework i.e. AutoGen by Microsoft
My debut book on LangChain is out now !!
What is AutoGen?
As mentioned, it is a Multi-AI Agent framework which enables creating multiple AI-Agents having different specializations and complete a task. One USP of AutoGen is it excels in coding related tasks and some special agents have code execution capabilities as well. Hence, if you’ve a use case say:
Analyze a given dataframe
AutoGen will not just write the code the analyze it, but is capable of executing it and show results as well.
Before we jump onto some tutorial videos, we must understand AutoGen’s two major agents:
- AssistantAgent: This is a general-purpose agent powered by an LLM. It can engage in natural language conversations, generate plans, and provide instructions. It is very similar to ChatGPT, but in python.
- UserProxyAgent: A special agent that acts as a proxy for the end-user (human prompt provider), enabling easy integration of human feedback and involvement. It can execute code, interact with tools, and relay information between the user and other agents.
So, usually, if an instruction is given, it is given using UserProxyAgent, the AssitantAgent will generate codes (if required) and UserProxyAgent would be able to execute those codes and get the final output.
There are other types of agents as well that I’m skipping for now
You can use AutoGen easily with OpenAI or Azure API. Things get a little trickier when trying to use Local LLMs but is feasible. Most of my below tutorials will be using Local LLMs using proxy server.
Let’s run through a few tutorials to understand AutoGen better
Building your 1st AutoGen app (using Local LLMs)
The tutorials explains how to
Setup local LLM or some other API (say HuggingFace) for AutoGen using Proxy Server
Build a basic application using AutoGen in python
Multi-Agent Conversation using AutoGen
Not just two agents, but you can create many customized agents using the AssistantAgent() and use the ChatManager feature to let these agents talk to each other in any fashion, be it round-robin or random. Checkout the below demo to understand how to set this up
As you saw how multiple AI-Agents with different roles were able to co-ordinate within themselves and are able to outline a product features and technical requirements.
AutoGen Studio: UI for AutoGen
The best part about AutoGen is it is not just limited to python coders or Data Scientists but anyone can easily access it using its Studio version providing a free UI. Checkout the below demo to understand how to set it up and get going.
Before ending, let’s discuss some of the issues I feel this package has and requires some modifications
The package is less intuitive. I’ve used Multi-Agent packages like CrewAI as well which are very easy to understand and use. AutoGen will require you to invest sometime before getting started
The support for different types of LLMs is limited. If you’ve got OpenAI/Azure, you’re good else you need to go through a way around to create proxy server and stuff which is cumbersome
Lack of extensive tutorials and resources.
Nonetheless, its a great package and worth trying anytime.
