In the Introduction of this series, we talked about Intelligence, Rationality and the concept of Rational Agents. Rational Agents are one that acts so as to achieve the best possible outcome when in uncertainty. The concept of rationality can be applied to a wide variety of agents operating in any imaginable environments.
Now we are going to see agents that can be called intelligent. How well an agent can behave depends on the environment the agent is in.
Agents and Environment
Basically, agents have sensors that perceive the environment and actuators that can interact with the environment.
For example, a human agent has eyes and ears which perceive the environment he/she is in and uses mouth, hands and legs for interacting with the environment. Likewise, a robotic agent has sensors like cameras and infrared finders for perceiving and motors for actuators that helps it interact with the environment.
Perceptual sequence
A percept is an agent’s perceptual input at any given instant. A perceptual sequence is the complete history of all perceptual inputs the agent has ever perceived. So if we specify the choice of actions for every possible percept sequence for an agent, we can completely define the behavior agent.
Generally, the agent’s behavior is described by the agent function that maps any given percept sequence to an action, this is because an agent’s choice of an action at any given moment may depend on the entire percept sequence, but not anything that it hasn’t perceived yet. This agent function is implemented internally for an artificial agent through an agent program.
Conceptually speaking, the rational agent is one that does the right thing. But, what does it mean to do the right thing?
Good Behavior: Concept of rationality
To actually know what is good behavior, we need to consider the consequences of the agent’s behavior.
The agent produces a sequence of percept according to the perceptual inputs it gets from the environment and acts accordingly to change the state of the environment. If the sequence is desirable, then the agent has performed well. This notion of desirability is captured by a performance measure that evaluates any given sequence of environment states.
It is important to note that it evaluates environment states and not the agent states, because if we measure the performance in terms of the agent’s opinion of its own performance, it could achieve perfect rationality by simply deluding itself that the performance is perfect.
Let us take an example,
Let us define a vacuum cleaner mini world, where there are two locations A and B. Vacuum cleaner is the agent and the locations A and B are the environments. The agent perceives which square it is in and whether there is dirt in the square. It can either move left or right and suck up the dirt. If the current square is dirty; then clean, otherwise move on to the next square.
If we talk about the performance measure of the above agent, we can define the measure in various ways. We can propose to measure the performance by the amount of dirt cleaned up in a fixed amount of time. But the problem with this is that, a rational agent could clean up the dirt and then dump it again on the ground then clean it again maximizing its performance. A more suitable measure would be to reward the agent for having a clean floor.
As a general rule it is better to design performance measures according to what one actually wants in the environment, rather than according to how one thinks the agent should behave.
Rationality
Rationality at any given point depends on four things:
1) The performance measure that defines the criterion for success.
2) The agent’s prior knowledge of the environment
3) The actions that the agent can perform
4) The agent’s percept sequence till date
From the above four points, we can formulate a formal definition of rational agents,
“ For each possible percept sequence, a rational agent should select an action that is expected to maximize its performance measure, given the evidence provided by the percept sequence and whatever built-in knowledge the agent has.”
The nature of environments
To design an agent, the first step should always be defining the task environment as completely as possible. Task environment is the description of Performance, Environment, Actuators and Sensors of an agent. Let us take an example task environment description of an automated taxi.
Properties of task environment
Task environments can be categorized according to their properties. These are:
Fully observable vs Partially observable
If an agent’s sensors give it access to the complete state of the environment at each point in time, then the task environment is said to be fully observable. In a fully observable task environment, there is no need to maintain internal state. A partially observable task environment is when there is noise or inaccuracies in the environment that the sensors get the data from. If an agent has no sensors at all, the task environment is said to be unobservable.
Single agent vs multi agent
Before describing single agent and multi agents, we need to know which entity can be viewed as an agent? Does agent A need to treat an object B as an agent?
To answer that we need to know whether B’s behavior is best described as maximizing performance measure whose value depends on agent A’s behavior. In a single agent environment there is only one agent, and the performance measure doesn’t depend on any other agents. But in multi agents there are more than one agents. For example, in chess there are two players, hence two agents and maximizing the performance measure of agent A depends on the minimizing performance measure of agent B, since chess is a competitive multi agent environment. On the other hand, automated taxis are a partially cooperative environment.
Deterministic vs Stochastic
If the next state of the environment is completely determined by the current state of the environment, then we say that the environment is deterministic otherwise it is stochastic. An environment is uncertain if it is not fully observable or not deterministic.
Episodic vs Sequential
In an episodic task environment, the experiences of an agent is divided into episodes and in each episode the agent receives a percept and performs a single action. In an episodic task environment the subsequent episodes do not depend on the previous episode. In sequential environments, current decisions affect all the future decisions.
Static vs Dynamic
If the environment can change while the agent is deliberating then the environment is said to be dynamic. If it doesn’t change then it is static. If the environment itself doesn’t change but the agent’s performance score does, then the environment is semi dynamic.
Discrete vs Continuous
This distinction applies to the state of the environment , to the way the time is handled, and to the percepts and actions of the agent. For example playing chess, the environment has a finite set of distinct states while taxi driving is a continuous state and continuous time problem.
Known vs Unknown
This distinction refers not to the environment itself, but the agent’s state of knowledge about the ‘laws of physics’ of the environment. In a known environment, the outcomes for all actions are given while in an unknown environment it is not.
From these properties of the task environment we can derive that the hardest case is partially observable, multiagent, stochastic, sequential, dynamic, continuous and unknown task environment.
An agent is the combination of the agent program and the architecture that supports the execution of the agent program. We will be discussing more about agent programs and its structure in the continuation of this series.
Reference,
Artificial Intelligence: A Modern Approach, by Peter Norvig and Stuart J. Russell
