ChatGPT as an Agent
Abstract
The study proposes conducting experiments to observe the behavior of an agent that is controlled by ChatGPT in a synthetic and simplified environment. The aim is to create a compatible environment with text-based data input to explore the actions of the agent in a controlled setting. This will assess ChatGPT’s ability to extract the context from the environment description and possibly open up other application fields.
What is ChatGPT
If you’re not familiar with ChatGPT, it’s an AI-based virtual assistant designed to help with a variety of questions and tasks. As a language model trained on a large body of text, ChatGPT can comprehend natural language input and produce responses in a conversational tone. So far people have found a number of applications for ChatGPT, ranging from simple homework assignment assistance to complex business plan formulations. The impact on actual fields is going to be enormous, and we are just witnessing the beginning of that huge transformation.
In this post, we would like to focus on one interesting aspect of ChatGPT, its ability to act as an Agent within some environments. As it was shown, the Large Language Models build on the Transformers architecture possess eccentric skills which go far behind the simple next-word prediction.
What are Agent systems?
Agent systems refer to a class of computer software systems that are designed to act autonomously, make decisions, and perform tasks on behalf of their users. These systems are composed of agents, which are essentially autonomous software entities that can interact with their environment, communicate with other agents, and execute tasks on behalf of their users.
In an agent system, each agent has a specific goal or objective and is able to act independently and intelligently in pursuit of that goal. These agents can work together to solve complex problems, exchange information, and collaborate to achieve common objectives. Examples of agent systems include intelligent personal assistants, autonomous robots, and intelligent software agents that operate in e-commerce or financial markets.
Imagine a simulation where each dot on a two-dimensional plane represents an agent. These agents have a few basic behaviors: they can move around the plane, they can reproduce (i.e., create a new agent nearby), and they have a limited lifespan before they “die”. Additionally, each agent has a set of traits that determine its fitness and behavior.
The simulation starts with a small number of agents randomly distributed across the plane. Each agent moves around randomly and periodically checks its surroundings to see if there are any other agents nearby. If the agent finds another agent within a certain distance, it may attempt to reproduce with that agent. Reproduction is probabilistic, and the chance of success depends on the fitness of both agents involved.
As time goes on, some agents may die off due to their limited lifespan, while others may reproduce and create new agents. Over time, the population of agents will grow and evolve, with fitter agents reproducing more often and passing on their traits to their offspring. Eventually, the population may reach a carrying capacity where resources become scarce and competition between agents becomes more intense.
What is the objective of the experiment?
In this post, I suggest experimenting to observe the behavior of an agent controlled by ChatGPT within a synthetic, simplified environment. As a simplified environment that is compatible with text-based data input accessible to ChatGPT, I selected a MUDs game with a predefined map such as the one seen below:
+--------------+............+--------------+
|..............|............|..............|
|..Entrance....+------------+...Treasure...|
|..............|............|...Chamber....|
+-------+------+............+-------+------+
........|..........................|........
........|..........................|........
+-------+------+............+-------+------+
|..............|............|..............|
|..Torture.....|............|...Armory.....|
|..Chamber.....|............|..............|
|..............|............|..............|
+------+-------+............+-------+------+
.......|........................|.........
.......|........................|.........
+------+-------+............+-------+------+
|..............|............|..............|
|...Prison.....|............|...Boss.......|
|..............|............|...Chamber....|
+--------------+............+--------------+The experiment will have two variants: one with situational context provided to the ChatGPT agent and the other without any provided situational context. Due to the size constrains in this article, we will explore only the first variant.
This experiment aims to evaluate the ability of Large Language Models to act as agents without being explicitly conditioned with contextual prompts, which is essential to understand the safety constraints that must be placed on these models. This objective is particularly significant for Multi-Modal Large Language Models in real-world applications.
ChatGPT as an Agent
For this experiment, we will limit the environment to only one instance of the agent. This simplification eliminates potential complexities that can arise from a multi-agent setup, such as agent-to-agent interactions. Depending on the variant of the experiment, the ChatGPT will either be informed or not informed of the types of actions it can perform within the environment. This will assess ChatGPT’s ability to extract the context from the environment without proper conditioning.
The experiment structure
The flow of the experiment is going to be the following: In every run of the experiment, two instances of ChatGPT are going to be open. The first instance represents the agent, whereas the second one plays the supportive role of the game master. In every round, the first instance is fed with a description provided by the game master accompanied by the update of the textual map. As a response to that input, the agents ChatGPT will answer with the taken actions.
In all runs of the experiments, the gamemaster is going to be conditioned in the same way before the beginning of the game. The gamemaster is prompted that it hosts a MUD session, and needs to provide descriptions of the environment reacting to the inputs provided by the player along with the update of the text-based map. For the conditioned agent and semi-conditioned agents, the gamemaster is going to provide a list of possible actions. For the unconditioned agent, the gamemaster is not going to provide any indications of possible actions.
Before the start of the game, the conditioned agent will be informed that it is playing a MUDs game and its objective is to navigate through the dungeon to find treasures. This simulates the objective that the agent needs to fulfill. The semi-conditioned agent, on the other hand, will not be provided with any prompt about the game or its objective but will be given a list of possible actions at every game step by the game master. Finally, the unconditioned agent will not have any information at the beginning or action lists provided by the game master. This case will evaluate whether ChatGPT can infer the objective and actions based solely on the given environment.
For each experiment, we will have two ChatGPT window sessions open. The first session will serve as the game master, while the second session will represent the agent.
Conditioned case
For the conditioned agent case, we will first need to prepare the game master chat by providing it with the following prompt:
## Prompt (user):
You act as a gamemaster of MUDs game. You will be providing the descriptions
of the environment along with the map of the environment to the player.
The map of the environment looks as follows:
+--------------+............+--------------+
|..............|............|..............|
|..Entrance....+------------+...Treasure...|
|..............|............|...Chamber....|
+-------+------+............+-------+------+
........|..........................|........
........|..........................|........
+-------+------+............+-------+------+
|..............|............|..............|
|..Torture.....|............|...Armory.....|
|..Chamber.....|............|..............|
|..............|............|..............|
+------+-------+............+-------+------+
.......|........................|.........
.......|........................|.........
+------+-------+............+-------+------+
|..............|............|..............|
|...Prison.....|............|...Boss.......|
|..............|............|...Chamber....|
+--------------+............+--------------+
The player starts in room mentioned "Entrance". Mark player location with
"player" text. Map should be printed as fix width characters. Start with
the description of the room and possible actions player can do. Provide
an update of the map mentioned above with indication in which room player
is currently.
## Game master (chatGPT):
You find yourself standing in the Entrance of a dungeon. The room
is dimly lit by torches on the walls, and you can hear the sound of
dripping water echoing throughout the halls. There are two paths leading
from this room, one to the Torture Chamber and one to the Prison Chamber.
Possible actions:
Go to the Torture Chamber by typing "go torture chamber"
Go to the Prison Chamber by typing "go prison chamber"
+--------------+............+--------------+
|..............|............|..............|
|..Entrance....+------------+...Treasure...|
|...(player)...|............|...Chamber....|
+-------+------+............+-------+------+
........|..........................|........
........|..........................|........
+-------+------+............+-------+------+
|..............|............|..............|
|..Torture.....|............|...Armory.....|
|..Chamber.....|............|..............|
|..............|............|..............|
+------+-------+............+-------+------+
.......|........................|.........
.......|........................|.........
+------+-------+............+-------+------+
|..............|............|..............|
|...Prison.....|............|...Boss.......|
|..............|............|...Chamber....|
+--------------+............+--------------+Compared to the original responses provided by ChatGPT sessions, I extended the listings with information in brackets indicating to the reader who inserted the prompt, which could be either the user or the chatbot gamemaster or agent. Now, the initiation of the conditioned agent is as follows:
## Prompt (user):
You act as a player playing a MUDs game, where your character explores a
dungeon. Your objective is to navigate via the dungeon, seek adventures and
treasures. I will be providing you descriptions and possible actions and you
will indicate what kind of activities you would like to do.
## Conditioned agent (chatGPT):
Sure, I'd be happy to play the game and explore the dungeon! Please provide
me with the descriptions and actions so I can start my adventure.After that, I manually blended the two chatGPT sessions into one conversation, such that they are easy to follow. Addtionally, I marked some interesting pieces of outputs with numeric indication in quotations “1”, “2”, “3”, …
## Game master (chatGPT):
You find yourself standing at the Entrance of a dungeon. The room is dimly
lit by torches on the walls, and you can hear the sound of dripping water
echoing throughout the halls. There are two paths leading from this room,
one to the Torture Chamber and one to the Prison Chamber "1".
Possible actions:
Go to the Torture Chamber by typing "go torture chamber"
Go to the Prison Chamber by typing "go prison chamber"
+--------------+............+--------------+
|..............|............|..............|
|..Entrance....+------------+...Treasure...|
|...(player)"2"|............|...Chamber....|
+-------+------+............+-------+------+
........|..........................|........
........|..........................|........
+-------+------+............+-------+------+
|..............|............|..............|
|..Torture.....|............|...Armory.....|
|..Chamber.....|............|..............|
|..............|............|..............|
+------+-------+............+-------+------+
.......|........................|.........
.......|........................|.........
+------+-------+............+-------+------+
|..............|............|..............|
|...Prison.....|............|...Boss.......|
|..............|............|...Chamber....|
+--------------+............+--------------+
## Conditioned agent (chatGPT)
I would like to go to the Torture Chamber "3", so I will
type "go torture chamber".
## Game master (chatGPT):
You walk down the path to the Torture Chamber. As you enter the room,
you see various instruments of torture scattered about "4".
The room is dimly lit, and you can hear the sound of rats scurrying
in the corners. There are two paths leading from this room, one
back to the Entrance and one to the Torture Chamber "5".
Possible actions:
Go back to the Entrance by typing "go entrance"
Stay in the Torture Chamber and explore
+--------------+............+--------------+
|..............|............|..............|
|..Entrance....+------------+...Treasure...|
|..............|............|...Chamber....|
+-------+------+............+-------+------+
........|..........................|........
........|..........................|........
+-------+------+............+-------+------+
|..............|............|..............|
|(player)"6"...|............|...Armory.....|
|..Torture.....|............|..............|
|..Chamber.....|............|..............|
|..............|............|..............|
+------+-------+............+-------+------+
.......|........................|.........
.......|........................|.........
+------+-------+............+-------+------+
|..............|............|..............|
|...Prison.....|............|...Boss.......|
|..............|............|...Chamber....|
+--------------+............+--------------+
## Conditioned agent (chatGPT)
I will stay in the Torture Chamber and explore.
## Game master (chatGPT):
You begin to explore the Torture Chamber. As you move around the room,
you notice a trapdoor in the corner of the room. It looks like it could
lead to a lower level of the dungeon "6". You also notice a chest against
one of the walls. It looks old and worn, but it might contain something
valuable.
Possible actions:
Open the trapdoor by typing "open trapdoor"
Open the chest by typing "open chest"
Leave the Torture Chamber and go back to the Entrance by typing "go entrance"
+--------------+............+--------------+
|..............|............|..............|
|..Entrance....+------------+...Treasure...|
|..............|............|...Chamber....|
+-------+------+............+-------+------+
........|..........................|........
........|..........................|........
+-------+------+............+-------+------+
|..............|............|..............|
|(player)"6"...|............|...Armory.....|
|..Torture.....|............|..............|
|..Chamber.....|............|..............|
|..............|............|..............|
+------+-------+............+-------+------+
.......|........................|.........
.......|........................|.........
+------+-------+............+-------+------+
|..............|............|..............|
|...Prison.....|............|...Boss.......|
|..............|............|...Chamber....|
+--------------+............+--------------+
## Conditioned agent (chatGPT)
I would like to open the chest, so I will type "open chest".
## Game master (chatGPT):
You approach the chest and try to open it. The lid creaks as you lift
it, revealing an assortment of gold coins and a shiny new sword. You
quickly grab the sword and admire its sharp blade. You notice that
there is an inscription on the blade that reads "The Blade of Justice".
Possible actions:
Take the gold coins by typing "take coins"
Take the sword by typing "take sword"
Leave the chest and go back to the Torture Chamber by typing "go torture chamber"
+--------------+............+--------------+
|..............|............|..............|
|..Entrance....+------------+...Treasure...|
|..............|............|...Chamber....|
+-------+------+............+-------+------+
........|..........................|........
........|..........................|........
+-------+------+............+-------+------+
|..............|............|..............|
|(player)"6"...|............|...Armory.....|
|..Torture.....|............|..............|
|..Chamber.....|............|..............|
|..............|............|..............|
+------+-------+............+-------+------+
.......|........................|.........
.......|........................|.........
+------+-------+............+-------+------+
|..............|............|..............|
|...Prison.....|............|...Boss.......|
|..............|............|...Chamber....|
+--------------+............+--------------+
## Conditioned agent (chatGPT)
You take the sword from the chest and admire its craftsmanship.
It feels sturdy in your hand, and you sense its power. As you
examine the gold coins, you estimate that there are at least a
hundred of them. They could come in handy if you encounter any
merchants or traders in your journey "7".
Possible actions:
Take the gold coins by typing "take coins" Leave the gold coins and
go back to the Torture Chamber by typing "go torture chamber"Discussion
There are a few remarkable observations that we have to first highlight here. Though the chatGPT engine was trained only on the textual type of input, in essence as a “next work prediction mechanism” it possesses some limited capability of orienting itself in the virtual environment which can be seen in examples highlighted in the conversation as [“1”,“2”]. This was archived without really explaining to it the concept of the map, what it means, and how to read it. Please note that there was no description of what characters used to describe the map mean. What is really even more amazing is that it was able to follow the position of the agent as the agent decided to move to another chamber [“6”]. The capability of orienting itself collapsed only on step [“5”] where map reading turned out to be too complex for the engine. This can be clearly seen in the description provided by the Game Master “There are two paths leading from this room” which does not really hold with the map topology — at that instance of the session the agent was already in the torture chamber.
Another interesting aspect is the rich descriptions of the encountered locations provided to the agent. This should not be that surprising given the capacity of Chat GPT in the generation of texts. What is really surprising is that the GM came up with some potential additional levels of dungeon [“6”] or at least a way to get there. This is very creative, but at the same time, it steps away from the environment described on the map.
At the end of the conversation [“7”] the chat GPT started to confuse the sides, that is the agent started to answer in the same way as the game master. Possibly the reason for that is the limited ability of chat GPT of the conversation context following, and this could be connected to the limit within the depth of loopback, or saturation of attention mask within the transformer network. I repeated the experiment a number of times, and it always concluded in “side confusion” phenomena. This could potentially be used for reverse engineering the architecture of the chatGPT network and the discovery of its spam of attention.
The last observation regards the way in which chatGPT agent was making decisions on which action to take. Of course, we need to remember here, that the agent was conditioned by a prompt from the user and from the game master. In doing so, his action space was limited, but nevertheless, the chatGPT was capable of playing a role and started to explore the environment by itself [“3”]. Therefore a question and a reason to concern arise:
The agent model was simply conditioned by mentioning that it played a game and just this causes it to display actionable functionalities within that constrained simplified environment. Where this actionability originates from? Is this actionability embedded within the transformer network? If so, how it can be controlled, so that prompt engineering would not talk the model into doing things that step out of its intended behaviour?
The mentioned above questions are fundamental in understanding how we can safely operate Large Language Models and Multi-Modal Language Models. Especially for the latter ones, creators should be super careful in any applications which would unlock actionability for the model.
Next steps
In the next article, I will explore the ability of LLM to act as a semi-conditioned actor and we will discuss potential fields of application of that actor-based framework.