A Beginner’s guide to Agents using AutoGen Studio and LM Studio

8 min readJan 19, 2024

image generated by author using DALL.E.3

Embarking on an AI Adventure: Understanding and Creating Agents

Imagine being able to create a piece of software that can make decisions, learn from experiences, and interact with its environment, almost like a human. Sounds like something out of a sci-fi movie, right? But this is exactly what agents in the world of artificial intelligence and machine learning can do! And the best part? You don’t need to be a seasoned AI expert to create them. With tools like AutoGen Studio and LMStudio, even beginners can dive into the fascinating world of agents. In this article, we’ll take a curious beginner’s journey into understanding what agents are, how to set up AutoGen Studio, and how to make it work locally using LMStudio. Ready to embark on this exciting adventure? Let’s get started!

Understanding Agents

So, what exactly are these agents we’re talking about? Picture this: You’re playing a video game, and there’s a character that seems to know exactly where to go, what to do, and how to win. That’s an agent! In the realm of AI, an agent is a piece of software that can perceive its environment (like the game world) and act upon it (like the character moving and making decisions) based on certain rules or algorithms.

But not all agents are created equal. Some agents, known as simple reflex agents, act based on what they’re currently perceiving, kind of like how we might instinctively swat away a fly. Other agents, called goal-based agents, are a bit more forward-thinking. They consider the future and act in a way that will help them achieve their goals. And then there are learning agents, the most advanced type, which can learn from their past experiences and adapt to changing circumstances, much like how we humans learn from our mistakes.

The magic of agents lies in their ability to automate complex tasks, make intelligent decisions, and interact with their environment in a way that mimics human intelligence. And the most exciting part? You can create them too! By exploring the world of agents, you’re opening the door to a universe of possibilities, where you can create systems that are not only efficient and effective but also capable of learning, adapting, and evolving.

Even though complex agents might require expert handling, for now, we can focus on creating and learning from simple agents.

AutoGenStudio

Introduction

Now that we’ve got a basic understanding of agents, it’s time to introduce the tool that will help us create and manage them — AutoGen Studio. Essentially, AutoGen Studio is a platform designed for the rapid prototyping of multi-agent solutions. It’s a practical tool that allows you to build and experiment with your own agents.

One of the key features of AutoGen Studio is its ability to let you define and modify agent workflows through a straightforward interface. You can create chat sessions with the specified agents and view results, such as chat history, generated files, and time taken. It’s a hands-on way to see your agents in action.

What’s more, AutoGen Studio is open source. This means you have the freedom to explore the code, make adjustments, and customize it to fit your specific needs. It’s a flexible tool that puts you in control.

But before we delve deeper, it’s important to get AutoGen Studio set up on your system. In the next section, we’ll walk you through the installation process and get you started on your journey with AI agents.

Setting Up AutoGen Studio: Installation and Execution Steps

Installing AutoGen Studio

Before you can start creating and managing agents, you need to install AutoGen Studio on your system.

Here’s a step-by-step guide to get you started:

Configuring a Language Model Provider: The first step is to set up access to a language model. You’ll need to configure your environment with either an `OPENAI_API_KEY` or an `AZURE_OPENAI_API_KEY`.

export OPENAI_API_KEY=your-key

The above step is optional if you would like to use a local model, but it is good to try out, especially the image generation example provided in skill, generate_images.py.

2. Installation: Once you’ve set up access to a language model, you can install AutoGen Studio. It’s recommended to use a virtual environment to avoid conflicts with existing Python packages. With Python 3.10 or newer active in your virtual environment, you can install Agent Studio using pip:

python -m install autogenstudio

If you prefer to install from source, ensure you have Python 3.10+ and Node.js (version above 14.15.0) installed. You can then clone the AutoGen Studio repository and install its Python dependencies.

Running AutoGen Studio

After you’ve installed AutoGen Studio, you can start running the application. Here’s how:

1. Run the web UI by entering the following in your python terminal:

autogenstudio ui --port 8081

This will start the application on the specified port (default:8081). You can then open your web browser and go to `http://127.0.0.1:8081/` to start using AutoGen Studio.

Now that you have AutoGen Studio installed and running, you’re ready to start exploring its capabilities. You can define and modify agent workflows, interact with agents and sessions, and expand agent skills.

AutoGenStudio terminal screenshot for image generation using gpt-preview-1106: Source, Author

Navigating the Features of AutoGen Studio

Exploring AutoGen Studio

Once you have AutoGen Studio up and running, you can start exploring its features. Here’s a brief overview of what you can do:

1. Creating Agents: You can create new agents by clicking on the ‘New Agent’ button. This will open a dialog where you can specify the agent’s name and description. You can also select the skills you want the agent to have.

2. Modifying Agent Workflows: AutoGen Studio allows you to define and modify the workflows of your agents. You can specify the tasks each agent should perform and the order in which they should be executed.

3. Interacting with Agents: You can interact with your agents through chat sessions. Simply select an agent and click on the ‘Chat’ button to start a session. You can then send messages to the agent and view its responses.

4. Viewing Session Results: After a chat session, you can view the results by clicking on the sidebar with ‘Sessions’ tab. This will show you the chat history, any files generated by the agent, and the time taken for each task.

5. Expanding Agent Skills: You can expand the skills of your agents by adding new ones. Simply click on the ‘Skills’ tab and select the skills you want to add.

Remember, AutoGen Studio is a tool for experimentation. Don’t be afraid to try different configurations and see what works best for your needs.

Making AutoGen Studio Work Locally with LM Studio

LM Studio is a simple tool that allows you to run a model locally. This can be particularly useful for tasks that require quick and secure access to AI-driven insights.

Deploying a Large Language Model Locally: A Comprehensive Guide

Introduction

medium.com

Steps:

Adding a New Skill: The first step is to add a new skill in AutoGen Studio. For example, you might want to create a function that generates a brief summary on a topic. This function would connect to a local LLM Model running in LM Studio to generate a concise two-line summary about a specified topic.


from typing import Optional
from openai import OpenAI

def generate_brief_topic_summary(topic: str) -> Optional[str]:
    """
    Connects to a local OpenAI server to generate a concise two-line summary about a specified topic. This function utilizes the OpenAI API in a local environment to provide quick and secure access to AI-driven insights. It's designed for obtaining brief overviews or introductions to a wide range of subjects, making it a valuable tool for content creation, research, or educational purposes.

    :param topic: The topic for which a brief summary is desired.
    :return: A string containing the two-line summary of the topic. Returns an error message if an exception occurs.
    """

    # Setup for local OpenAI client
    client = OpenAI(base_url="http://localhost:1234/v1", api_key="not-needed")

    try:
        # Creating a completion request to the local model
        completion = client.chat.completions.create(
            model="local-model",  # this field is currently unused
            messages=[
                {"role": "system", "content": "Write two lines about this topic."},
                {"role": "user", "content": topic}
            ],
            temperature=0.7,
        )
        return completion.choices[0].message
    except Exception as e:
        return f"An error occurred: {e}"

Build > New Skill

Source: Author’s local install of AutoGen Studio

2. Adding a new agent: You can use the existing agents or create one. Let’s create one, ‘local_assistant’.
Agents > New Agent
In the agents window, Add a model, give it a name ‘Local’. Enter parameters for your model. The Base URL will be the one provided by LM Studio (http://localhost:1234/v1) or any other port number if you have changed it.

Use “ ” in API key value, to avoid API related error messages.

Then, add the new skill (ex: generate_brief_topic_summary) to this agent.

3. Adding a new workflow: Add the local_assistant in Sender. You canuse the primary_assistant provided as a sample. Just make sure to replace the existing models with the local one as shown above.

4. Running the Agent: Once everything is set up, you can run the skill you created using an agent. You should see the output in AutoGent Studio and the processing in the terminal and LM Studio console.

Source: Author’s local install of LM Studio

Remember, this is just one example of what you can do with LM Studio and AutoGen Studio. The platforms are designed to be flexible and adaptable, so feel free to experiment with different functions/local AI models etcand see what works best for your needs.

Conclusion:

In this guide, we’ve learned how to use agents by setting up and running AutoGen Studio locally with LM Studio. We’ve gone through the steps of adding a new skill, an agent and a workflow, finally, running an agent with the skill in AutoGen Studio.

What’s great about these platforms is their flexibility. They can handle a wide variety of tasks and workflows, giving you the freedom to experiment and find what works best for you.

Developing Business Thinking AI Plugins using Semantic Kernel

The short courses available on Andrew Ng’s deeplearning.ai website provide comprehensive overviews of the latest AI…