CS451/CS551/EE565

ARTIFICIAL
INTELLIGENCE
Agent Types
8-29-2008
Prof. Janice T. Searleman
[email protected], jetsza
Recap: Rational Agents
What is rational at a given time depends on four
things:
 Performance Measure
 Prior environment knowledge
 Actions
 Percept sequence to date (sensors).

Defn: A rational agent chooses whichever action

maximizes the expected value of the performance
measure given the percept sequence to date and
prior environment knowledge.
Recap: Agent functions
 An agent is completely specified by the
agent function mapping percept
sequences to actions
 One agent function (or a small
equivalence class) is rational
 Aim: find a way to implement the rational
agent function concisely
Recap: Structure of Intelligent Agents
 Agent = architecture + program
 Agent program: the implementation of f : P*  A, the
agent’s perception-action mapping
function Skeleton-Agent(Percept) returns Action
memory  UpdateMemory(memory, Percept)
Action  ChooseBestAction(memory)
memory  UpdateMemory(memory, Action)
return Action
 Architecture: a device that can execute the agent
program (e.g., general-purpose computer, specialized
device, bot, etc.)
Recap: Environments
To design a rational agent, we must specify
its task environment.

PEAS – way to describe the environment

 Performance
 Environment
 Actuators
 Sensors
 Recap: Environment types
Environment Accessible Deterministic Episodic Static Discrete

Operating Yes Yes No No Yes

System
Virtual Yes Yes Y/N No Y/N
Reality
Office No No No No No
Environment
Mars No Semi No Semi No

The environment types largely determine the agent design.

Agent types
 Four basic kind of agent programs will be
discussed:
 Simple reflex agents
 Model-based reflex agents
 Goal-based agents
 Utility-based agents
 All these can be turned into learning agents.
Agent types
Function TABLE-DRIVEN_AGENT(percept) returns an action

static: percepts, a sequence initially empty

table, a table of actions, indexed by percept sequence

append percept to the end of percepts

action  LOOKUP(percepts, table)
return action

This approach is doomed to failure

Agent types; simple reflex
• Select action on the basis
of only the current percept.
– e.g. the vacuum-agent
• Large reduction in possible
percept/action
situations(next page).
• Implemented through
condition-action rules
– If dirty then suck
Vacuum-cleaner world

 Percepts: location and contents, e.g.,

[A,Dirty]
 Actions: Left, Right, Suck, NoOp
The vacuum-cleaner world

function REFLEX-VACUUM-AGENT ([location, status])

return an action
if status == Dirty then return Suck
else if location == A then return Right
else if location == B then return Left
Reduction from 4T to 4 entries
Agent types; simple reflex
function SIMPLE-REFLEX-AGENT(percept) returns an
action

static: rules, a set of condition-action rules

state  INTERPRET-INPUT(percept)
rule  RULE-MATCH(state, rule)
action  RULE-ACTION[rule]
return action

Will only work if the environment is fully observable

otherwise infinite loops may occur.
Agent types; reflex and state
• To tackle partially
observable environments.
– Maintain internal state
• Over time update state
using world knowledge
– How does the world
change.
– How do actions affect
world.
 Model of World
Agent types; reflex and state
function REFLEX-AGENT-WITH-STATE(percept)
returns an action

static: rules, a set of condition-action rules

state, a description of the current world state
action, the most recent action.

state  UPDATE-STATE(state, action, percept)

rule  RULE-MATCH(state, rule)
action  RULE-ACTION[rule]
return action
Agent types; goal-based
• The agent needs a goal to know
which situations are desirable.
– Things become difficult
when long sequences of
actions are required to find
the goal.
• Typically investigated in search
and planning research.
• Major difference: future is taken
into account
• Is more flexible since
knowledge is represented
explicitly and can be
manipulated.
Agent types; utility-based
• Certain goals can be reached in
different ways.
– Some are better, have a
higher utility.
• Utility function maps a
(sequence of) state(s) onto a
real number.
• Improves on goals:
– Selecting between
conflicting goals
– Select appropriately between
several goals based on
likelihood of success.
Performance Measure vs.
Utility Function
 performance measure
- used by outside observer to evaluate
success
- function from histories to a real number
 utility function

- used by the agent itself to evaluate how

desirable states or histories are
- function from state(s) to a real number
Agent types; learning
• All previous agent-
programs describe methods
for selecting actions.
– Yet it does not explain the
origin of these programs.
– Learning mechanisms can be
used to perform this task.
– Teach them instead of
instructing them.
– Advantage is the robustness
of the program toward
initially unknown
environments.
Agent types; learning
• Learning element: introduce
improvements in performance
element.
– Critic provides feedback on
agents performance based on
fixed performance standard.
• Performance element: selecting
actions based on percepts.
– Corresponds to the previous
agent programs
• Problem generator: suggests
actions that will lead to new and
informative experiences.
– Exploration vs. exploitation
Behavior and performance of IAs
 Perception (sequence) to Action
Mapping:
f : P*  A
Ideal mapping: specifies which actions
an agent ought to take at any point in
time
Description: Look-Up-Table, Closed
Form, …
 A driving example: Beobots
 Goal: build robots that can operate in unconstrained
environments and that can solve a wide variety of tasks.

Beowulf + Robot =“Beobot” http://iLab.usc.edu/beobots/

Beowulf + robot =
“Beobot”
Look up table
Distance Action
10 No action
obstacle
5 Turn left sensor
30 degrees agent
2 Stop

Compare to a closed form:

 Output (degree of rotation) = F(distance)
 example: F(d) = 10/d
Using a look-up-table to encode f : P*  A
 Example: Collision Avoidance
 Sensors: 3 proximity sensors obstacle
 Effectors: Steering Wheel, Brakes sensors
 How to generate?
agent
 How large?
 How to select action?
Using a look-up-table to encode f : P*A
obstacle
sensors
 Example: Collision Avoidance
 Sensors: 3 proximity sensors agent
 Effectors: Steering Wheel, Brakes
 How to generate: for each p  Pl  Pm  Pr
generate an appropriate action, a  S  B
 How large: size of table = #possible percepts
times # possible actions = |Pl | |Pm| |Pr| |S| |B|
E.g., P = {close, medium, far}3
A = {left, straight, right}  {on, off}
then size of table = 27*3*2 = 162
 How to select action? Search.
Behavior and performance of IAs
 Perception (sequence) to Action
Mapping:
f : P*  A
Ideal mapping: specifies which actions
an agent ought to take at any point in
time
Description: Look-Up-Table, Closed
Form, …
Interacting Agents
Collision Avoidance Agent (CAA)
 Goals: Avoid running into obstacles
 Percepts ?
 Sensors?
 Effectors ?
 Actions ?
 Environment: Freeway

Lane Keeping Agent (LKA)

• Goals: Stay in current lane
• Percepts ?
• Sensors?
• Effectors ?
• Actions ?
• Environment: Freeway
Interacting Agents
Collision Avoidance Agent (CAA)
 Goals: Avoid running into obstacles
 Percepts: Obstacle distance, velocity, trajectory
 Sensors: Vision, proximity sensing
 Effectors: Steering Wheel, Accelerator, Brakes, Horn, Headlights
 Actions: Steer, speed up, brake, blow horn, signal (headlights)
 Environment: Freeway

Lane Keeping Agent (LKA)

• Goals: Stay in current lane
• Percepts: Lane center, lane boundaries
• Sensors: Vision
• Effectors: Steering Wheel, Accelerator, Brakes
• Actions: Steer, speed up, brake
• Environment: Freeway
Conflict Resolution by Action
Selection Agents
• Override: CAA overrides LKA
• Arbitrate: if Obstacle is Close then CAA
else LKA
• Compromise: Choose action that satisfies both
agents
• Any combination of the above

• Challenges: Doing the right thing

 Summary
 Intelligent Agents:
 Anything that can be viewed as perceiving its
environment through sensors and acting upon that
environment through its effectors to maximize
progress towards its goals.
 PAGE (Percepts, Actions, Goals, Environment)
 Described as a Perception (sequence) to Action
Mapping: f : P*  A
 Using look-up-table, closed form, etc.
 Agent Types: reflex, state, goal-based, utility-based
 Rational Action: The action that maximizes the
expected value of the performance measure given the
percept sequence to date