Describing AI Environments by Type
Everything has one…
To define algorithms that solve problems, the problem is often structured in a way such that there is an agent (“being” that is solving the problem), an environment, a way of interacting with the environment, and an overall goal or “success.”
If you’re familiar with the Wumpus World, the agent is the, well, agent, the environment is the squares and obstacles the agent navigates, and the goal is to get the gold and get out.
A way to describe the overall problem and setting (also called the task environment) is by using PEAS. PEAS show the elements of the environment, but for this article, we’re going to focus on how to classify your problem’s environment. Knowing the environment description will help narrow down the best possible approach for interaction.
Environment Types
There are six different classifications we can ascribe to our task’s environment. They are as follows:
- Fully Observable or Partially Observable
- Deterministic or Stochastic
- Static or Dynamic
- Discrete or Continuous
- Single Agent or Multi-Agent
- Known or Unknown
Now, let’s dig a bit into each of these labels to understand them more fully.
Fully or Partially Observable
If an environment is fully observable, an agent is able to sense the complete state of the environment at all times. An example of this would be chess. At all times, a chess player is able to see all of their pieces, all of the opponent’s pieces, and the entire board’s squares.
The alternative is a partially observable environment. This would be a game like poker. An agent can see their cards and the face-up cards, but they are not able to view their opponents’ cards.
If there is no information about the environment available to the agent and the agent has no way to sense it, it is called unobservable. These scenarios are rare, and thus not normally considered a “type.”
Deterministic or Stochastic
A deterministic environment is one that has no element of “randomness.” That is, if an agent performs the same action in the same state, it will experience the same outcome every single time. When two hydrogen atoms bond to an oxygen atom, the end result is water every single time. You will never get anything except for water when that state and action occur.
Stochastic environments do have the potential for randomness. When you go to unlock your bike in the morning to ride to work, most of the time you will go downstairs, unlock your bike, and go. However, there is the potential that you will go downstairs to find your bike stolen. Or, maybe you have a flat tire. Or, there could be a crime scene near your bike and the area has been taped off.
Because of the complexity of billions of environmental elements and additional agents, most real-life environments are stochastic. Deterministic environments usually only occur if the environmental elements are tightly constrained, such as in human-defined games, or are building blocks of physics.
Static or Dynamic
Observing, thinking, and then acting. These are the steps each action goes through when performing an action. When an environment is static, nothing about the task changes while the agent is deliberating or thinking. So, if efficiency isn’t a part of the performance, an agent could think about the problem as long as they would like.
For our static example, we’re back to chess. Once it’s an opponent’s turn, if there is no time limit set, a player can observe where the pieces are and then think about their next turn without worrying about other pieces moving in the mean time.
A dynamic environment will continue to change as the agent thinks. When a soccer player has the ball and is examining other player’s positions in the hopes of scoring a goal, the task around them does not halt as they think. Think too long, an opposing player will zip by and steal the ball.
An in-between called semi-dynamic is when the environment does not change as the agent thinks, but the agent’s performance score does. For example, in a house with no living creatures, we could say that a robot vacuum is, for the most part, in a static environment. However, if the robot’s performance is measured by how quickly it finishes cleaning, then the longer the robot thinks about its next move, the lower its performance score goes.
Discrete or Continuous
A discrete environment is one where there is a limited (countable) number of perceptions and actions. A game like chess is discrete. On a turn, there are a countable number of possible movements.
Compare this to a continuous environment where the number of interactions that could be had is believed to be endless. Most real-life environments are continuous. A human agent has a goal of getting to work. They can move in any continuous direction they choose, smell millions of different scents, brush their hair in billions of different ways, and all of this could potentially have an effect on the actions taken later.
Single-Agent or Multi-Agent
Single-agent environments are simple; there is only one being that is pursuing a goal. Having a performance metric of winning a game of solitaire is an example. There are no opponents, no teammates, no other agents.
Many environments are multi-agent. As an agent in a classroom, you may have the goal of learning a subject. However, the teacher is also an agent with a very similar goal of teaching the subject to all of their students. Other classmates are agents who may have the same goal as you, or may even have a goal to interfere with or lower your performance metric.
Known or Unknown
Here, “known” or “unknown” don’t describe the environment, but instead describe the agent’s perception of the environment.
In a known environment, an agent is aware of the available actions and their consequences in terms of pursuing the objective(s). This can also be considered understanding the rules of the objective it is pursuing. For many adults, tic-tac-toe is a known environment. They know what they need to do to win and how their actions will affect those winnings on a probability basis.
In an unknown environment, an agent does not know what reactions the actions it performs will cause. Think of a human entering the world. They must be taught, or figure out through experience, every facet of life. This is also an example of how the environment may be unknown to one agent but known to another (a professional piano player playing a song).
Do not get known or unknown confused with fully observable and partially observable. A known environment can be partially observable (Wumpus World) and an unknown environment can be fully observable (a human playing an escape room).
And those are the environments! For more AI posts, feel free to follow my page or message me with desired topics; and if this post helped you, seeing applause brightens my day :)