HiLCoE School of Computer Science and Technology

Chapter Two: Intelligent Agents

Eyob N. (PhD)

Chapter Two: Intelligent Agents

Contents
What is an agent?
◦ Rational agents,
◦ Autonomous agents
Agents & Environments:
◦ Classes of environments
Agent structure;
Agent types

Chapter Two: Intelligent Agents

What’s an agent?
 An agent is anything that perceiving its environment
through sensors and acting upon that environment through
actuators
 Example:
 Human is an agent
 A robot is also an agent with cameras and motors
 A thermostat detecting room temperature.
Percept
 Agent’s perceptual inputs at any given instant
Percept sequence
What AI should fill
 Complete history of everything that the agent has ever
perceived.

Chapter Two: Intelligent Agents

Agent function & program
Agent’s behavior is mathematically
described by
 Agent function
 A function mapping any given percept sequence
to an action
Practically it is described by
 An agent program
 The real implementation
Vacuum-cleaner world
Perception: Clean or Dirty? where it is in?
Actions: Move left, Move right, suck, do
nothing
Vacuum-cleaner world
Program implements the agent function
tabulated in Fig. 2.3
Function Reflex-Vacuum-Agent([location,status]) return an
action
If status = Dirty then return Suck
else if location = A then return Right
else if location = B then return left
Concept of Rationality
Rational agent
 One that does the right thing
 = every entry in the table for the agent function is
correct (rational).
What is correct?
 The actions that cause the agent to be most successful
 What does it mean to do the right thing?-by
considering the consequences of the agent’s behavior.
 So we need ways to measure success.
Performance measure
Performance measure
 An objective function that determines
 How the agent does successfully
 E.g., 90% or 30% ?

An agent, based on its percepts

  action sequence :
if desirable, it is said to be performing well.
 No universal performance measure for all
agents
Performance measure
A general rule:
 Design performance measures according to
 What one actually wants in the environment
 Rather than how one thinks the agent should behave

E.g., in vacuum-cleaner world

 We want the floor clean, no matter how the agent
behave
 We don’t restrict how the agent behaves
Rationality
What is rational at any given time depends on
four things:
 The performance measure defining the criterion of
success
 The agent’s prior knowledge of the environment
 The actions that the agent can perform
 The agents’s percept sequence up to now
Rational agent
For each possible percept sequence,
 an rational agent should select
 an action expected to maximize its performance measure,
given the evidence provided by the percept sequence and
whatever built-in knowledge the agent has
E.g., an exam
 Maximize marks, based on
the questions on the paper & your knowledge
Example of a rational agent
Performance measure
 Awards one point for each clean square
 at each time step, over 1000 time steps
Prior knowledge about the environment
 The geography of the environment
 Only two squares
 The effect of the actions
Example of a rational agent
Actions that can perform
 Left, Right, Suck and NoOp
Percept sequences
 Where is the agent?
 Whether the location contains dirt?

Under this circumstance, the agent is

rational.
Omniscience
An omniscient agent
 Knows the actual outcome of its actions in
advance
 No other possible outcomes

 However, impossible in real world

An example
 crossing a street but died of the fallen cargo
door from 33,000ft  irrational?
Omniscience

Based on the circumstance, it is rational.

rationality is not the same as perfection
As rationality maximizes
 Expected performance
Perfection maximizes
 Actual performance
Hence rational agents are not omniscient.
Doing actions in order to modify future percepts—sometimes
called information gathering
Learning
Does a rational agent depend on only
current percept?
 No, the past percept sequence should also be
used
 This is called learning
 After experiencing an episode, the agent
 should adjust its behaviors to perform better for the
same job next time.
 Autonomy
If an agent just relies on the prior knowledge of its
designer rather than its own percepts then the agent
lacks autonomy
A rational agent should be autonomous- it should
learn what it can to compensate for partial or
incorrect prior knowledge.
E.g., a clock
 No input (percepts)
 Run only but its own algorithm (prior knowledge)
 No learning, no experience, etc.
Software Agents
Sometimes, the environment may not be the
real world
 E.g., flight simulator, video games, Internet
 They are all artificial but very complex
environments
 Those agents working in these environments are
called
 Software agent (softbots)
 Because all parts of the agent are software
Task environments
Task environments are the problems
 While the rational agents are the solutions
Specifying the task environment
 PEAS description as fully as possible
 Performance
 Environment

 Actuators

 Sensors

In designing an agent, the first step must always be to specify the

task environment as fully as possible.
Use automated taxi driver as an example
Task environments
Performance measure
 How can we judge the automated driver?
 Which factors are considered?
 getting to the correct destination
 minimizing fuel consumption

 minimizing the trip time and/or cost

 minimizing the violations of traffic laws

 maximizing the safety and comfort, etc.

Task environments
Environment
 A taxi must deal with a variety of roads
 Traffic lights, other vehicles, pedestrians, stray
animals, road works, police cars, etc.
 Interact with the customer
Task environments
Actuators (for outputs)
 Control over the accelerator, steering, gear
shifting and braking
 A display to communicate with the customers

Sensors (for inputs)

 Detect other vehicles, road situations
 GPS (Global Positioning System) to know
where the taxi is
 Many more devices are necessary
Task environments
A sketch of automated taxi driver
Agents and Environments
An AI system is composed of an agent and its environment.
An agent is anything that can be viewed as perceiving its
environment through sensors and acting upon that
environment through actuators.
 For instance:
◦ a human agent has eyes, ears, and other organs for sensors and
hands, legs, vocal tract, and so on for actuators.
◦ a robotic agent might have cameras for sensors and various
motors for actuators.

Chapter Two: Intelligent Agents

Agents and Environments(cont’d…)
An environment is everything in the world
which surrounds the agent, but it is not a part
of an agent itself.
It is a place where agent lives and operates.
It provides the agent with something to sense
and act upon it.

Chapter Two: Intelligent Agents

Properties of an environment
An environment can have various features
from the point of view of an agent:
◦ Fully observable versus Partially Observable
◦ Deterministic versus Stochastic
◦ Episodic versus sequential
◦ Static versus dynamic
◦ Discrete versus continuous
◦ Single-agent versus Multi-agent
◦ Known versus Unknown
◦ Accessible versus Inaccessible

Chapter Two: Intelligent Agents

Properties of task environments
Fully observable vs. Partially observable
 If an agent’s sensors give it access to the complete
state of the environment at each point in time then
the environment is effectively and fully
observable
 if the sensors detect all aspects
 That are relevant to the choice of action
 Partially observable
 An environment might be Partially observable
because of noisy and inaccurate sensors or
because parts of the state are simply missing from
the sensor data.
 Example:
 A local dirt sensor of the cleaner cannot tell
 Whether other squares are clean or not
Properties of task environments
Deterministic vs. stochastic
 next state of the environment Completely determined
by the current state and the actions executed by the
agent, then the environment is deterministic, otherwise,
it is Stochastic.
 Strategic environment: deterministic except for actions
of other agents
◦ -Cleaner and taxi driver are:
 Stochastic because of some unobservable aspects  noise or
unknown
Properties of task environments
Episodic vs. sequential
 An episode = agent’s single pair of perception & action
 The quality of the agent’s action does not depend on other
episodes
 Every episode is independent of each other
 Episodic environment is simpler
 The agent does not need to think ahead
Sequential
 Current action may affect all future decisions
◦ -Ex. Taxi driving and chess.
Properties of task environments
Static vs. dynamic
 A dynamic environment is always changing
over time
 E.g., the number of people in the street
 While static environment
 E.g., the destination
Semidynamic
 environment is not changed over time
 but the agent’s performance score does
Properties of task environments

Discrete vs. continuous

 If there are a limited number of distinct states,
clearly defined percepts and actions, the
environment is discrete
 E.g., Chess game

 Continuous: Taxi driving

Properties of task environments

Single agent VS. multiagent

 Playing a crossword puzzle – single agent
 Chess playing – two agents

 Competitive multiagent environment

 Chess playing
 Cooperative multiagent environment
 Automated taxi driver
 Avoiding collision
Properties of task environments

Known vs. unknown

This distinction refers not to the environment itslef but to the
agent’s (or designer’s) state of knowledge about the
environment.
-In known environment, the outcomes for all actions are
given. ( example: solitaire card games).
- If the environment is unknown, the agent will have to learn
how it works in order to make good decisions.( example:
new video game).
Examples of task environments
Structure of Intelligent Agents

Intelligent agent = architecture + program.

 Architecture: computing device with physical sensors and
actuators.
◦ It might be just an ordinary PC, or it might be a robotic car
with several onboard computers, cameras, and other sensors.
◦ It makes the percepts from the sensors available to the
program, runs the program and feeds the program's action
choices to the actuators as they are generated.
 Agent program: is a real implementation of an agent
function.
◦ it implements an agent function which maps percepts to
actions.

Chapter Two: Intelligent Agents

Agent Types
Agents can be grouped into five classes based on
their degree of perceived intelligence and capability.
◦ Simple reflex agent
◦ Model-based reflex agent
◦ Goal-based agent
◦ Utility-based agent
◦ Learning agent
Note: All of these agents can improve their
performance and generate better action over the
time.

Chapter Two: Intelligent Agents

Simple reflex agent
Itis the simplest agent.
It takes decisions on the basis of the current
percepts and ignore the rest of the percept history.
◦ It does not consider any part of percepts history during
their decision and action process.
It only succeed in the fully observable environment.
It works on Condition-action rule, which means:
◦ it maps the current state to action.
◦ Example: Room Cleaner agent works only if there is dirt
in the room.

Chapter Two: Intelligent Agents

Simple reflex agent(cont’d…)
 Problems for the simple reflex agent design approach:
◦ They have very limited intelligence
◦ They do not have knowledge of non-perceptual parts of the
current state
◦ Mostly too big to generate and to store.
◦ Not adaptive to changes in the environment. / cannot operates
well over dynamic Environment.

Chapter Two: Intelligent Agents

Model-based reflex agent
It can work in a partially observable
environment, and track the situation.
It has two important factors:
◦ Model: It is knowledge about "how things happen
in the world", so it is called a Model-based agent.
◦ Internal State: It is a representation of the current
state based on percept history.
These agents have the model, "which is
knowledge of the world" and based on the
model they perform actions.
Chapter Two: Intelligent Agents
Model-based reflex agent(cont’d…)
 Updatingthe agent state requires information about:
◦ How the world evolves.
◦ How the agent's action affects the world.

◦ n

Chapter Two: Intelligent Agents

Goal-based agent
 The knowledge of the current state environment is not
always sufficient to decide for an agent to what to do.
 The agent needs to know its goal which describes desirable
situations.
 Goal-based agents expand the capabilities of the model-
based agent by having the "goal" information.
 They choose an action, so that they can achieve the goal.
 These agents may have to consider a long sequence of
possible actions before deciding whether the goal is
achieved or not.
 Such considerations of different scenario are called
searching and planning, which makes an agent proactive.

Chapter Two: Intelligent Agents

Goal-based agent(cont’d…)

Chapter Two: Intelligent Agents

Utility-based agent
 These agents are similar to the goal-based agent but
provide an extra component of utility measurement
which makes them different by providing a measure of
success at a given state.
 Utility-based agent act based not only goals but also
the best way to achieve the goal.
 It is useful when there are multiple possible
alternatives, and an agent has to choose in order to
perform the best action.
 The utility function maps each state to a real number to
check how efficiently each action achieves the goals.

Chapter Two: Intelligent Agents

Utility-based agent(cont’d…)

Chapter Two: Intelligent Agents

Learning agent
It is the type of agent which can learn from its past experiences.
It starts to act with basic knowledge and then able to act and
adapt automatically through learning.
A learning agent has mainly four conceptual components, which
are:
◦ Learning element: is responsible for making improvements by
learning from environment by taking feedback from critic.
◦ Critic: describes how well the agent is doing with respect to a
fixed performance standard.
◦ Performance element: is responsible for selecting external
action.
◦ Problem generator: is responsible for suggesting actions that
will lead to new and informative experiences.

Chapter Two: Intelligent Agents

Learning agent(cont’d…)
It is able to learn, analyze performance, and
look for new ways to improve the
performance.

Chapter Two: Intelligent Agents