www.

studymaterial
CS8691 – Artificial Intelligence

CS8691 ARTIFICIAL INTELLIGENCE

UNIT I INTRODUCTION
Introduction–Definition - Future of Artificial Intelligence – Characteristics of Intelligent
Agents–Typical Intelligent Agents – Problem Solving Approach to Typical AI problems.

Artificial Intelligence – An Introduction

What is AI?

Artificial intelligence is the study of how to make computers do things which, at

the moment people do better.

Some definitions of artificial intelligence, organized into four categories

I. Systems that think like humans

1. "The exciting new effort to make computers think machines with minds, in the
full and literal sense." (Haugeland, 1985)

2. "The automation of activities that we associate with human thinking, activities

such as decision-making, problem solving, learning" (Bellman, 1978)

II. Systems that act like humans

3. "The art of creating machines that performs functions that require intelligence
when performed by people." (Kurzweil, 1990)

4. "The study of how to make computers do things at which, at the moment,

people are better." (Rich and Knight, 1991)

III. Systems that think rationally

5. "The study of mental faculties through the use of computational models."

(Chamiak and McDermott, 1985)

6. "The study of the computations that make it possible to perceive, reason, and
act." (Winston, 1992)

IV. Systems that act rationally

7. "Computational Intelligence is the study of the design of intelligent agents."

(Poole et al., 1998)

CSE – Panimalar Engineering College 1

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial
CS8691 – Artificial Intelligence

8. "AI is concerned with intelligent behavior in artifacts." (Nilsson, 1998)

The definitions on the 1, 2, 3, 4 measure success in terms of human

performance, whereas the ones on the 5, 6, 7, 8 measure against an ideal
concept of intelligence.

A system is rational if it does the "right thing," given what it knows.

The term AI is defined by each author in its own perceive, leads to four
important categories

i. Acting humanly: The Turing Test approach

ii. Thinking humanly: The cognitive modeling approach
iii. Thinking rationally: The "laws of thought" approach
iv. Acting rationally: The rational agent approach

(i) Acting humanly: The Turing Test approach

To conduct this test, we need two people and the machine to be

evaluated. One person plays the role of the interrogator, who is in a separate
room from the computer and the other person. The interrogator can ask
questions of either the person or the computer but typing questions and
receiving typed responses. However, the interrogator knows them only as A and
B and aims to determine which the person is and which is the machine.

The goal of the machine is to fool the interrogator into believing that is the
person. If the machine succeeds at this, then we will conclude that the machine
is acting humanly. But programming a computer to pass the test provides plenty
to work on, to possess the following capabilities.

 Natural language processing to enable it to communicate successfully

in English.
 Knowledge representation to store what it knows or hears;

CSE – Panimalar Engineering College 2

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial
CS8691 – Artificial Intelligence

 Automated reasoning to use the stored information to answer

questions and to draw new conclusions
 Machine learning to adapt to new circumstances and to detect and
extrapolate patterns.

Total Turing Test: the test which includes a video so that the interrogator can
test the perceptual abilities of the machine. To undergo the total Turing test, the
computer will need

 computer vision to perceive objects, and

 robotics to manipulate objects and move about

(ii) Thinking humanly: The cognitive modeling approach

To construct a machines program to think like a human, first it requires

the knowledge about the actual workings of human mind. After completing the
study about human mind it is possible to express the theory as a computer
program.

If the program’s inputs/output and timing behavior matched with the

human behavior then we can say that the program’s mechanism is working like a
human mind.

Example: General Problem Solver (GPS) – A problem solvers always keeps

track of human mind regardless of right answers. The problem solver is contrast
to other researchers, because they are concentrating on getting the right
answers regardless of the human mind.

An Interdisciplinary field of cognitive science uses computer models from

AI and experimental techniques from psychology to construct the theory of the
working of the human mind.

(iii) Thinking rationally: The "laws of thought" approach

Laws of thought were supposed to govern the operation of the mind and their
study initiated the field called logic

Example 1:"Socrates is a man; All men are mortal; therefore, Socrates is

mortal."

Example 2:“Ram is a student of III year CSE; All students are good in III year
CSE; therefore, Ram is a good student”

CSE – Panimalar Engineering College 3

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial
CS8691 – Artificial Intelligence

Syllogisms : A form of deductive reasoning consisting of a major premise, a

minor premise, and a conclusion

Syllogisms provided patterns for argument structures that always yielded correct
conclusions when given correct premises

There are two main obstacles to this approach.

1. It is not easy to take informal knowledge and state it in the formal terms
required by logical notation, particularly when the knowledge is less.

2. There is a big difference between being able to solve a problem "in

principle" and doing so in practice

(iv) Acting rationally: The rational agent approach

An agent is just something that acts. A rational agent is one that acts so as to
achieve the best outcome or, when there is uncertainty, the best expected
outcome. The study of rational agent has two advantages.

1. Correct inference is selected and applied

2. It concentrates on scientific development rather than other methods.

Foundation of Artificial Intelligence

AI derives the features from Philosophy, Mathematics, Psychology, Computer

Engineering, Linguistics topics.

Philosophy(428 B.C. – present)

Aristotle (384-322 B.C.) was the first to formulate a precise set of laws governing
the rational part of the mind. He developed an informal system of syllogisms for
proper reasoning, which allowed one to generate conclusions mechanically, given
initial premises.

Mathematics (c. 800-present)

 What are the formal rules to draw valid conclusions?

 What can be computed?
 How do we reason with uncertain information?

Philosophers staked out most of the important ideas of k1, but the leap to a
formal science required a level of mathematical formalization in three
fundamental areas: logic, computation, and probability

CSE – Panimalar Engineering College 4

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial
CS8691 – Artificial Intelligence

Economics (1776-present)

 How should we make decisions so as to maximize payoff?

 How should we do this when others may not go along?

The science of economics got its start in 1776, when Scottish philosopher Adam
Smith (1723-1790) published An Inquiry into the Nature and Causes of the
Wealth of Nations. While the ancient Greeks and others had made contributions
to economic thought, Smith was the first to treat it as a science, using the idea
that economies can be thought of as consisting of individual agents maximizing
their own economic well-being

Neuroscience (1861-present)

 How do brains process information?

Neuroscience is the study of the nervous system, particularly the brain. The
exact way in which the brain enables thought is one of the great mysteries of
science. It has been appreciated for thousands of years that the brain is
somehow involved in thought, because of the evidence that strong blows to the
head can lead to mental incapacitation

Computer Human Brain

Computational units 1 CPU,108 gates 1011 neurons
Storage units 1010 bits RAM 1011 neurons
1011 bits disk 1014 synapses
Cycle time 10-9 sec 10-3 sec
Bandwidth 1010 bits/sec 1014 bits/sec
Memory updates/sec 109 1014
Comparison of the raw computational resources and brain.

Psychology (1879 – present)

The origin of scientific psychology are traced back to the wok if German
physiologist Hermann von Helmholtz(1821-1894) and his student Wilhelm
Wundt(1832 – 1920). In 1879, Wundt opened the first laboratory of
experimental psychology at the University of Leipzig. In US,the development of
computer modeling led to the creation of the field of cognitive science. The
field can be said to have started at the workshop in September 1956 at MIT.

CSE – Panimalar Engineering College 5

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial
CS8691 – Artificial Intelligence

Computer engineering (1940-present)

For artificial intelligence to succeed, we need two things: intelligence and an

artifact. The computer has been the artifact of choice.A1 also owes a debt to the
software side of computer science, which has supplied the operating systems,
programming languages, and tools needed to write modern programs

Control theory and Cybernetics (1948-present)

Ktesibios of Alexandria (c. 250 B.c.) built the first self-controlling machine: a
water clock with a regulator that kept the flow of water running through it at a
constant, predictable pace. Modern control theory, especially the branch known
as stochastic optimal control, has as its goal the design of systems that maximize
an objective function over time.

Linguistics (1957-present)

Modem linguistics and AI, then, were "born" at about the same time, and grew
up together, intersecting in a hybrid field called computational linguistics or
natural language processing.

History of Artificial Intelligence

The gestation of artificial intelligence (1943-1955)

There were a number of early examples of work that can be characterized as AI,
but it was Alan Turing who first articulated a complete vision of A1 in his 1950
article "Computing Machinery and Intelligence." Therein, he introduced the
Turing test, machine learning, genetic algorithms, and reinforcement learning.

The birth of artificial intelligence (1956)

McCarthy convinced Minsky, Claude Shannon, and Nathaniel Rochester to help

him bring together U.S. researchers interested in automata theory, neural nets,
and the study of intelligence. They organized a two-month workshop at
Dartmouth in the summer of 1956. Perhaps the longest-lasting thing to come out
of the workshop was an agreement to adopt McCarthy's new name for the field:
artificial intelligence.

Early enthusiasm, great expectations (1952-1969)

The early years of A1 were full of successes-in a limited way. General Problem
Solver (GPS) was a computer program created in 1957 by Herbert Simon and
Allen Newell to build a universal problem solver machine. The order in which the

CSE – Panimalar Engineering College 6

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial
CS8691 – Artificial Intelligence

program considered subgoals and possible actions was similar to that in which
humans approached the same problems. Thus, GPS was probably the first
program to embody the "thinking humanly" approach. At IBM, Nathaniel
Rochester and his colleagues produced some of the first A1 programs. Herbert
Gelernter (1959) constructed the Geometry Theorem Prover, which was able to
prove theorems that many students of mathematics would find quite tricky.

Lisp was invented by John McCarthy in 1958 while he was at the Massachusetts
Institute of Technology (MIT). In 1963, McCarthy started the AI lab at Stanford.
Tom Evans's ANALOGY program (1968) solved geometric analogy problems that
appear in IQ tests, such as the one in Figure

Fig : The Tom Evan’s ANALOGY program could solve geometric analogy
problems as shown.
A dose of reality (1966-1973)

From the beginning, AI researchers were not shy about making predictions of
their coming successes. The following statement by Herbert Simon in 1957 is
often quoted:

“It is not my aim to surprise or shock you-but the simplest way I can summarize
is to say that there are now in the world machines that think, that learn and that
create. Moreover, their ability to do these things is going to increase rapidly
until-in a visible future-the range of problems they can handle will be coextensive
with the range to which the human mind has been applied.

Knowledge-based systems: The key to power? (1969-1979)

Dendral was an influential pioneer project in artificial intelligence (AI) of the

1960s, and the computer software expert system that it produced. Its primary
aim was to help organic chemists in identifying unknown organic molecules, by
analyzing their mass spectra and using knowledge of chemistry. It was done at

CSE – Panimalar Engineering College 7

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial
CS8691 – Artificial Intelligence

Stanford University by Edward Feigenbaum, Bruce Buchanan, Joshua Lederberg,

and Carl Djerassi.

AI becomes an industry (1980-present)

In 1981, the Japanese announced the "Fifth Generation" project, a 10-year plan
to build intelligent computers running Prolog. Overall, the A1 industry boomed
from a few million dollars in 1980 to billions of dollars in 1988.

The return of neural networks (1986-present)

Psychologists including David Rumelhart and Geoff Hinton continued the study of
neural-net models of memory.

AI becomes a science (1987-present)

In recent years, approaches based on hidden Markov models (HMMs) have

come to dominate the area. Speech technology and the related field of
handwritten character recognition are already making the transition to
widespread industrial and consumer applications.

The Bayesian network formalism was invented to allow efficient representation

of, and rigorous reasoning with, uncertain knowledge.

The emergence of intelligent agents (1995-present)

One of the most important environments for intelligent agents is the Internet .

Sample Applications

Autonomous planning and scheduling: A hundred million miles from Earth,

NASA's Remote Agent program became the first on-board autonomous planning
program to control the scheduling of operations for a spacecraft. Remote Agent
generated plans from high-level goals specified from the ground, and it
monitored the operation of the spacecraft as the plans were executed-detecting,
diagnosing, and recovering from problems as they occurred.

Game playing: IBM's Deep Blue became the first computer program to defeat
the world champion (Garry Kasparov) in a chess match. The value of IBM's stock
increased by $18 billion.

Autonomous control: The ALVINN computer vision system was trained to

steer a car to keep it following a lane. The computer-controlled minivan used to
navigate across the United States-for 2850 miles and it was in control of steering

CSE – Panimalar Engineering College 8

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial
CS8691 – Artificial Intelligence

the vehicle 98% of the time. A human took over the other 2%, mostly at exit
ramps.

Diagnosis: Medical diagnosis programs based on probabilistic analysis have

been able to perform at the level of an expert physician in several areas of
medicine

Logistics Planning: During the Gulf crisis of 1991, U.S. forces deployed a
Dynamic Analysis and Replanning Tool, DART to do automated logistics planning
and scheduling for transportation. This involved up to 50,000 vehicles, cargo,
and people at a time, and had to account for starting points, destinations,
routes, and conflict resolution

Robotics: Many surgeons now use robot assistants in microsurgery

Language understanding and problem solving: PROVERB is a computer

program that solves crossword puzzles better than most humans, using
constraints on possible word fillers, a large database of past puzzles, and a
variety of information sources including dictionaries and online databases such as
a list of movies and the actors that appear in them.

Typical problems to which AI methods are applied

Pattern recognition, Optical character recognition , Handwriting recognition ,

Speech recognition , Face recognition, Computer vision, Virtual reality and
Image processing , Diagnosis , Game theory and Strategic planning , Natural
language processing, Translation and Chatterboxes , Nonlinear control and
Robotics, Artificial life, Automated reasoning , Automation , Biologically
inspired computing ,Concept mining , Data mining , Knowledge
representation , Semantic Web , E-mail spam filtering, Robotics, ,Cognitive ,
Cybernetics , Hybrid intelligent system, Intelligent agent ,Intelligent control

INTELLIGENT AGENTS

Introduction - Agents and Environments

An agent is anything that can be viewed as perceiving its environment through

sensors andacting upon that environment through actuators.

Different types of agents

1. A human agent has eyes, ears, and other organs for sensors and hands,
legs, mouth, and other body parts for actuators.

CSE – Panimalar Engineering College 9

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial
CS8691 – Artificial Intelligence

2. A robotic agent might have cameras and infrared range finders for sensors
and various motors for actuators.

3. A software agent receives keystrokes, file contents, and network packets as

sensory inputs and acts on the environment by displaying on the screen, writing
files, and sending network packets.

4. Generic agent – A general structure of an agent who interacts with the

environment.

Fig : Agents interact with environments through sensors and effectors

(accuators)

The term percept is to refer to the agent's perceptual inputs at any given
instant.

PERCEPT SEQUENCE: Agent's percept sequence is the complete history of

everything the agent has ever perceived.

An agent's behavior is described by the agent function that maps any given
percept sequence to an action.

AGENT PROGRAM : The agent function for an artificial agent will be

implemented by an agent program.

Example : The vacuum-cleaner world has just two locations: squares A and B.
The vacuum agent perceives which square it is in and whether there is dirt in the
square. It can choose to move left, move right, suck up the dirt, or do nothing.
One very simple agent function is the following: if the current square is dirty,
then suck, otherwise move to the other square.

CSE – Panimalar Engineering College 10

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial
CS8691 – Artificial Intelligence

Fig : A vacuum-cleaner world with just two locations

Partial tabulation of a simple agent function for the vacuum-cleaner

world
• Percepts: location and status, e.g., [A,Dirty]
• Actions: Left, Right, Suck, NoOp

Percept Action
sequence

[A, Clean] Right

[A, Dirty] Suck

[B, Clean] Left

[B, Dirty] Suck

The agent program for a simple agent in the two-state vacuum

environment for above tabulation

function VACUUM-AGENT([location,status])
if status = Dirty then return Suck
else if location = A then return Right
else if location = B then return Left

CSE – Panimalar Engineering College 11

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial
CS8691 – Artificial Intelligence

Concept of Rationality

A rational agent is one that does the right thing. The right action is the one that
will cause the agent to be most successful.

Performance measures

A performance measure embodies the criterion for success of an agent's

behavior. When an agent is plunked down in an environment, it generates a
sequence of actions according to the percepts it receives. This sequence of
actions causes the environment to go through a sequence of states. If the
sequence is desirable, then the agent has performed well.

Rationality

Rational at any given time depends on four things:

1. The performance measure that defines the criterion of success.

2. The agent's prior knowledge of the environment.
3. The actions that the agent can perform.
4. The agent's percept sequence to date.

Definition of a rational agent:

For each possible percept sequence, a rational agent should select an action that
is expected to maximize its performance measure, given the evidence provided
by the percept sequence and whatever built-in knowledge the agent has. A
rational agent should be autonomous

Definition of an omniscient agent:

An omniscient agent knows the actual outcome of its actions and can act
accordingly; but omniscience is impossible in reality.

Autonomy

A rational agent should be autonomous-it should learn what it can to

compensate for partial or incorrect prior knowledge.

Information Gathering

Doing actions in order to modify future percepts is called as information

gathering.

CSE – Panimalar Engineering College 12

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial
CS8691 – Artificial Intelligence

Specifying the task environment

In the discussion of the rationality of any agent, we had to specify the

performance measure, the environment, and the agent's actuators and sensors.
We group all these together under the heading of the task environment and we
call this as PEAS (Performance, Environment, Actuators, Sensors) or
PAGE (Percept, Action, Goal, Environment) description. In designing an
agent, the first step must always be to specify the task environment.

Example : PEAS description of the task environment for agents

Agent Performance Environment Actuators Sensors

Type Measure

Automated Safe, fast, legal, Roads, Steering Cameras, sonar,

Taxi Driver comfortable trip, traffic, accelerator speedometer,
maximize profits pedestrian , GPS, odometer,
customers brake, accelerometer
signal, engine sensors,
horn, keyboard
display
Medical Healthy patient, Patient, Screen Keyboard (entry
diagnosis minimize costs, hospital, staff display of symptoms,
system lawsuits (question findings, patient's
tests, answers)
diagnoses
treatment
referrals)
Part-Picking Percentage of Conveyor belt Jointed arm Camera, joint
Robot parts in correct with parts, and hand angle sensors
bin bins
Interactive Maximize Set of Screen Keyboard
English tutor student’s score students display
on test (exercises)
robot soccer amount of goals soccer match legs cameras, sonar or
player scored field infrared
Satellite Correct Image Downlink from Display Color pixel arrays
Image Categorization satellite categorizati
Analysis on of scene
Refinery Maximum Refinery Valves, Temperature,
controller purity, safety operators pumps, pressure, chemical
heaters, sensors

CSE – Panimalar Engineering College 13

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial
CS8691 – Artificial Intelligence

displays
Vacuum minimize energy two squares Left, Right, Sensors to identify
Agent consumption, Suck, NoOp the dirt
maximize dirt
pick up
Properties of task environments (Environment Types)

Fully observable vs. partially observable

If an agent's sensors give it access to the complete state of the environment at

each point in time, then we say that the task environment is fully observable. A
chess playing system is an example of a system that operates in a fully
observable environment.

An environment might be partially observable because of noisy and inaccurate

sensors or because parts of the state are simply missing from the sensor data. A
bridge playing program is an example of a system operating in a partially
observable environment.

Deterministic vs. stochastic

If the next state of the environment is completely determined by the current

state and the action executed by the agent, then we say the environment is
deterministic; otherwise, it is stochastic

Image analysis systems are examples of deterministic. The processed image is

determined completely by the current image and the processing operations.

Taxi driving is clearly stochastic in this sense, because one can never predict the
behavior of traffic exactly;

Episodic vs. sequential

An episodic environment means that subsequent episodes do not depend on

what actions occurred in previous episodes.

In a sequential environment, the agent engages in a series of connected

episodes. In sequential environments, on the other hand, the current decision
could affect all future decisions. Chess and taxi driving are sequential.

CSE – Panimalar Engineering College 14

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial
CS8691 – Artificial Intelligence

Static vs. dynamic

If the environment can change while an agent is deliberating, then we say the
environment is dynamic for that agent; otherwise, it is static. Taxi driving is
clearly dynamic. Crossword puzzles are static.

Discrete vs. continuous

If the number of distinct percepts and actions is limited, the environment is

discrete, otherwise it is continuous. Taxi driving is a continuous state and
continuous-time problem. Chess game has a finite number of distinct states.

Single agent vs. Multi agent

The distinction between single-agent and multi agent environments may seem
simple enough. For example, an agent solving a crossword puzzle by itself is
clearly in a single-agent environment, whereas an agent playing chess is in a
two-agent environment.

Chess is a competitive multi agent environment. Taxi-driving environment is a

partially cooperative multi agent environment.

Environment Characteristics

Examples of task environments and their characteristics

Task Observable Deterministic Episodic Static Discrete Agent

Environment
Crossword Fully Deterministic Sequential Static Discrete Single
puzzle
Chess with a Fully Stochastic Sequential Semi Discrete Multi
clock

Poker Partially Stochastic Sequential Static Discrete Multi

Backgammon Fully Stochastic Sequential Static Discrete Multi

Taxi dnving Partially Stochastic Sequential Dynamic Continuous Multi

Medical Partially Stochastic Sequential Dynamic Continuous Single

diagnosis

Image- Fully Deterministic Episodic Semi Continuous Single

analysis

Part-picking Partially Stochastic Episodic Dynamic Continuous Single

robot

CSE – Panimalar Engineering College 15

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial
CS8691 – Artificial Intelligence

Refinery Partially Stochastic Sequential Dynamic Continuous Single

controller

Interactive Partially Stochastic Sequential Dynamic Discrete Multi

English tutor

• The simplest environment is

– Fully observable, deterministic, episodic, static, discrete and single-
agent.
• Most real situations are:
– Partially observable, stochastic, sequential, dynamic, continuous
and multi-agent.

Structure of the Agents

The job of AI is to design the agent program that implements the agent function
mapping percepts to actions.

Intelligent agent = Architecture + Agent program

Agent programs

Agent programs take the current percept as input from the sensors and return
an action to the actuators

The agent program takes the current percept as input, and the agent function
takes the entire percept history

Architecture is a computing device used to run the agent program.

The agent programs will use some internal data structures that will be updated
as new percepts arrive. The data structures are operated by the agents decision
making procedures to generated an action choice, which is then passed to the
architecture to be executed. Two types of agent programs are

1. A Skeleton Agent
2. A Table Lookup Agent

Skeleton Agent

The agent program receives only a single percept as its input.

If the percept is a new input then the agent updates the memory with the new
percept

CSE – Panimalar Engineering College 16

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial
CS8691 – Artificial Intelligence

function SKELETON-AGENT( percept) returns action

static: memory, the agent’s memory of the world
memory <- UPDATE-MEMORY(memory, percept)
action <- CHOOSE-BEST-ACTION(memory)
memory <- UPDATE-MEMORY(memory, action)
return action

Table-lookup agent

A table which consists of indexed percept sequences with its corresponding

action

The input percept checks the table for the same

function TABLE-DRIVEN-AGENT(percept) returns an action

static: percepts, a sequence initially empty

table, a table of actions, indexed by percept
sequence
append percept to the end of percepts
action  LOOKUP(percepts, table)
return action

Drawbacks of table lookup agent

• Huge table
• Take a long time to build the table
• No autonomy
• Even with learning, need a long time to learn the table entries

Four basic types in order of increasing generality

• Simple reflex agents

• Model-based reflex agents
• Goal-based agents
• Utility-based agents

Simple reflex agents

The simplest kind of agent is the simple reflex agent. These agents select actions
on the basis of the current percept, ignoring the rest of the percept history.

This agent describes about how the condition – action rules allow the agent to
make the connection from percept to action

CSE – Panimalar Engineering College 17

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial
CS8691 – Artificial Intelligence

It acts according to a rule whose condition matches the current state, as defined
by the percept.

Condition – action rule : if condition then action

Example : condition-action rule: if car-in-front-is-braking then initiate-

braking

Fig : Schematic diagram of a simple reflex agent.

Rectangles - to denote the current internal state of the agent's decision

process
Ovals - to represent the background information used in the process.

function SIMPLE-REFLEX-AGENT(percept) returns action

static : rules, a set of condition-action rules
state < - I N T E R P R E T - I N P U T (percept )
rule <- R U L E - M A T C H (state, rules),
action <- R U L E - A C T I O N [rule ]
return action

CSE – Panimalar Engineering College 18

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial
CS8691 – Artificial Intelligence

 INTERPRET-INPUT function generates an abstracted description of the

current state from the percept
 RULE-MATCH function returns the first rule in the set of rules that
matches the given state description
 RULE - ACTION – the selected rule is executed as action of the given
percept

Example : Medical Diagnosis System

If the patient has reddish brown spots then start the treatment for measles.
Model based Reflex Agents
An agent which combines the current percept with the old internal state to
generate updated description of the current state.

function REFLEX-AGENT-WITH-STATE(percept)returns action

static: state, a description of the current world state
rules, a set of condition-action rules
action, the most recent action, initially none
state <- UPDATE-STATE( state, action, percept)
rule <- RULE - MATCH ( state, rules )
action <- RULE-ACTION [rule] UPD
return action ATE-
STATE - is responsible for creating the new internal state description

Example: Medical Diagnosis system

CSE – Panimalar Engineering College 19

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial
CS8691 – Artificial Intelligence

If the Patient has spots then check the internal state (i. e) any change in the
environment may lead to cause spots on the patient. From this internal state the
current state is updated and the corresponding action is executed.
Goal based Agents

An Agent knows the description of current state as well as goal state. The action
matches with the current state is selected depends on the goal state.

Example : Medical diagnosis system

If the name of disease is identified for the patient then the treatment is given to
the patient to recover from him from the disease and make the patient healthy is
the goal to be achieved

Utility base agents

An agent which generates a goal state with high – quality behavior (i.e) if more
than one sequence exists to reach the goal state then the sequence with more
reliable, safer, quicker and cheaper than others to be selected.

Utility is a function that maps a state onto a real number, which describes the
associated degree of happiness

The utility function can be used for two different cases :

CSE – Panimalar Engineering College 20

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial
CS8691 – Artificial Intelligence

1. When there are conflicting goals, only some of which can be achieved (for
example, speed and safety)

2. When there are several goals that the agent can aim for, none of which
can be achieved with certainty, utility provides a way in which the
likelihood of success can be weighed up against the importance of the
goal

Example : Medical diagnosis System

If the patient disease is identified then the sequence of treatment which leads to
recover the patient with all utility measure is selected and applied

Learning agent

All agents can improve their performance through Learning

The learning task allows the agent to operate in unknown environments initially
and then become more competent than its initial knowledge.

A learning agent can be divided into four conceptual components:

1. Learning element
2. performance element
3. Critic
4. Problem generator

CSE – Panimalar Engineering College 21

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial
CS8691 – Artificial Intelligence

The learning element uses feedback from the critic on how the agent is doing
and determines how the performance element should be modified to do better in
the future. Learning element is also responsible for making improvements

Performance element is to select external action and it is equivalent to agent

The critic tells the learning element how well the agent is doing with respect to
a fixed performance standard

The last component of the learning agent is the problem generator. It is

responsible for suggesting actions that will lead to new and informative
experiences.

Problem solving – Introduction

Search is one of the operational tasks that characterize AI programs best. Almost
every AI program depends on a search procedure to perform its prescribed
functions. Problems are typically defined in terms of state, and solution
corresponds to goal states.

Problem solving using search technique performs two sequence of steps:

CSE – Panimalar Engineering College 22

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial
CS8691 – Artificial Intelligence

(i) Define the problem - Given problem is identified with its required
initial and goal state.
(ii) Analyze the problem - The best search technique for the given:
problem is chosen from different an AI search technique which derives
one or more goal state in minimum number of states.

Types of problem

In general the problem can be classified under anyone of the following four types
which depends on two important properties. They are

(i) Amount of knowledge, of the agent on the state and action description.
(ii) How the agent is connected to its environment through its percepts and
actions?

The four different types of problems are:

(i) Single state problem

(ii) Multiple state problem
(iii) Contingency problem
(iv) Exploration problem

Problem solving Agents

Problem solving agent is one kind of goal based agent, where the agent decides
what to do by finding sequence of actions that lead to desirable states. The
complexity arises here is the knowledge about the formulation process, (from
current state to outcome action) of the agent.

If the agent understood the definition of problem, it is relatively straight forward

to construct a search process for finding solutions, which implies that problem
solving agent should be an intelligent agent to maximize the performance
measure.

The sequence of steps done by the intelligent agent to maximize the

performance measure:
i) Goal formulation - based on current situation is the first step in problem
solving. Actions that result to a failure case can be rejected without further
consideration.
(ii)Problem formulation - is the process of deciding what actions and states
to consider and follows goal formulation.
(iii) Search - is the process of finding different possible sequence of actions
that lead to state of known value, and choosing the best one from the states.

CSE – Panimalar Engineering College 23

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial
CS8691 – Artificial Intelligence

(iv) Solution - a search algorithm takes a problem as input and returns a

solution in the form of action sequence.
(v) Execution phase - if the solution exists, the action it recommends can be
carried out.

A simple problem solving agent

function SIMPLE-PROBLEM-SOLVING-AGENT(p) returns an

action
input : p, a percept
static: s, an action sequence, initially empty
state, some description of the current world state
g, a goal initially null
problem, a problem formulation
state <- UPDATE-STATE(state, p)
if s is empty then
g <- FORMULATE-GOAL(state)
problem <-FORMULATE-PROBLEM(state,g)
s <- SEARCH(problem)
action <- RECOMMENDATION(s, state)
s <- REMAINDER(s, state)
return action

Note :

RECOMMENDATION - first action in the sequence

REMAINDER - returns the rest
SEARCH - choosing the best one from the sequence of actions
FORMULATE-PROBLEM - sequence of actions and states that lead to goal
state.
UPDATE-STATE - initial state is forced to next state to reach the goal state

Well-defined problems and solutions

A problem can be defined formally by four components:

1. initial state
2. successor function
3. goal test
4. path cost
The initial state that the agent starts in.

Successor function (S) - Given a particular state x, S(x) returns a set of states
reachable from x by any single action.

CSE – Panimalar Engineering College 24

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial
CS8691 – Artificial Intelligence

The goal test, which determines whether a given state is a goal state.
Sometimes there is an explicit set of possible goal states, and the test simply
checks whether the given state is one of them.

A path cost function that assigns a numeric cost to each path. The problem-
solving agent chooses a cost function that reflects its own performance measure.

A solution to a problem is a path from the initial state to a goal state

Operator - The set of possible actions available to the agent.

State space (or) state set space - The set of all possible states reachable
from the initial state by any sequence of actions.

Path (state space) - The sequence of action leading from one state to another

The effectiveness of a search can be measured using three factors. They are:

1 Solution is identified or not?

2. Is it a good solution? If yes, then path cost to be minimum.
3. Search cost of the problem that is associated with time and memory required
to find a solution.

For Example

Imagine an agent in the city of Arad, Romania, enjoying a touring holiday. Now,
suppose the agent has a nonrefundable ticket to fly out of Bucharest the
following day. In that case, it makes sense for the agent to adopt the goal of
getting to Bucharest. The agent's task is to find out which sequence of actions
will get it to a goal state.

This process of looking for such a sequence is called search.

A search algorithm takes a problem as input and returns a solution in the form of
an action sequence. Once a solution is found, the actions it recommends can be
carried out. This is called the execution phase.

Formulating problems

Initial state : the initial state for our agent in Romania might be described as
In(Arad)

Successor function : Given a particular state x, SUCCESSOR-FN(x) returns a

set of (action, successor) ordered pairs, where each action is one of the legal

CSE – Panimalar Engineering College 25

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial
CS8691 – Artificial Intelligence

actions in state x and each successor is a state that can be reached from x by
applying the action. For example, from the state In(Arad), the successor function
for the Romania problem would return
{( Go(Sibzu),In(Sibiu)), (Go(Timisoara), In(Tzmisoara)), (Go(Zerznd),In(Zerind)))

Goal test : The agent's goal in Romania is the singleton set {In(Bucharest)).

Path cost : The step cost of taking action a to go from state x to state y is
denoted by c(x, a, y).

Example Problems

The problem-solving approach has been applied to a vast array of task

environments.

A toy problem is intended to illustrate or exercise various problem-solving

methods. It can be given a concise, exact description. It can be used easily by
different researchers to compare the performance of algorithms

A real-world problem is one whose solutions people actually care about.

Some list of best known toy and real-world problems

Toy Problems

CSE – Panimalar Engineering College 26

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial
CS8691 – Artificial Intelligence

i) Vacuum world Problem

States: The agent is in one of two locations, each of which might or might not
contain dirt. Thus there are 2 * 22 = 8 possible world states.
Initial state: Any state can be designated as the initial state.
Successor function: three actions (Left, Right, and Suck).
Goal test: This checks whether all the squares are clean.
Path cost: Each step costs 1, so the path cost is the number of steps in the
path.

Fig : The complete state space for Vacuum World

ii) 8-puzzle Problem

The 8-puzzle problem consists of a 3 x 3 board with eight numbered tiles and a
blank space. A tile adjacent to the blank space can slide into the space. The
object is to reach a specified goal state

States: A state description specifies the location of each of the eight tiles and
the blank in one of the nine squares.
Initial state: Any state can be designated as the initial state.
Successor function: This generates the legal states that result from trying the
four actions (blank moves Left, Right, Up, or Down).
Goal test: This checks whether the state matches the goal configuration (Other
goal configurations are possible.)
Path cost: Each step costs 1, so the path cost is the number of steps in the
path.

CSE – Panimalar Engineering College 27

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial
CS8691 – Artificial Intelligence

Initial State Goal State

iii) 8-queens problem

The goal of the 8-queens problem is to place eight queens on a chessboard

such that no queen attacks any other. (A queen attacks any piece in the same
row, column or diagonal.

States: Any arrangement of 0 to 8 queens on the board is a state.

Initial state: No queens on the board.
Successor function: Add a queen to any empty square.
Goal test: 8 queens are on the board, none attacked.
Path cost : Zero (search cost only exists)

solution to the 8-queens problem.

iv) Crypt arithmetic Problem

In crypt arithmetic problems letters stand for digits and the aim is to find a
substitution of digits for letters such that the resulting sum is arithmetically
correct, each letter stand for a different digit

Rules

There should be no more than 10 distinct characters

The summation should be the longest word

CSE – Panimalar Engineering College 28

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial
CS8691 – Artificial Intelligence

The summation can not be too long

There must be a one-to-one mapping between letters and digits
The leftmost letter can't be zero in any word.

States: A crypt arithmetic puzzle with some letters replaced by digits

Initial state: No digits is assigned to the letters
Successor function: Replace all occurrences of a letter with a digit not already
appearing in the puzzle
Goal test: Puzzle contains only digits and represents a correct sum
Path cost : Zero

Example 1:

SEND
+MORE
----------
MONEY

Solution : S=9 , E = 5, N = 6, D=7, M= 1, O= 0, R = 8, Y=2

Example 2:

FORTY
+TEN
+TEN
-------
SIXTY
-------

Solution : F=2, O=9, R=7, T=8 , Y=6, E=5, N=0

v) Missionaries and cannibals problem

Three missionaries and three cannibals are on one side of a river, along with a
oat that can hold one or two people. Find a way to get everyone to the other
side, without ever leaving a group of missionaries in one place out numbers by
the cannibals in that place

Assumptions :

1. Number of trips is not restricted

2. Both the missionary and cannibal can row the boat

States: A state consists of an ordered sequence of two numbers representing

the number of missionaries and cannibals

CSE – Panimalar Engineering College 29

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial
CS8691 – Artificial Intelligence

Example : (i,j) = (3,3) three missionaries and three cannibals

Initial state: (i,j) = (3,3) in one side of the river

Successor function: The possible move across the river are:

1. One Missionary and One Cannibal

2. Two Missionaries
3. Two Cannibals
4. One Missionary
5. One Cannibal

Rule No. Explanation

(i) (i, j) : One missionary and one cannibal
can cross the river only when ((i-1) >=
(j-1)) in one side of the river and ((i+1)
>= (j+ 1)) in the other side of the
river.

(ii) (i,j) : Two missionaries can cross the

river only when ((i-2)>=j) in one side
of the
river and ((i+2)>=j) in the other side
of the river.
(iii) (i,j) : Two cannibals can cross the river
only when ((j-2)<= i) in one side of
the river and ((j+2)<= i) in the other
side of the river.
(iv) (i,j) : One missionary can cross the
river only when ((i-1)>=j)) in one side
of the river and ((i-1)>=j)) in the other
side of the river.
(v) (i,j) : One cannibal can cross the river
only when (((j-l)<=i) in one side of the
river
and (((j+l)<=i)in the other side of the
river.

Initial state : (i.j) = (3,3) in one side of the river.

Goal test: (i,j) = (3,3) in the other side of the river.

Path cost : Number of crossings between the two sides of the river.

CSE – Panimalar Engineering College 30

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial
CS8691 – Artificial Intelligence

Solution:

Bank1 Boat Bank2 Rule

Applied
(i,j)=(3,3) (i,j)=(0,0)
(3,1) -> (0,2) -> (0,2) (iii)
(3,2) <- (0,1) <- (0,1) (v)
(3,0) -> (0,2) -> (0,3) (iii)
(3,1) <- (0,1) <- (0,2) (v)
(1,1) -> (2,0) -> (2,2) (ii)
(2,2) <- (1,1) <- (1,1) (i)
(0,2) -> (2,0) -> (3,1) (ii)
(0,3) <- (0,1) <- (3,0) (v)
(0,1) -> (0,2) -> (3,2) (iii)
(0,2) <- (0,1) <- (3,1) (v)
(0,0) -> (0,2) -> (3,3) (iii)

Real-world problems

Airline travel problem

States: Each is represented by a location (e.g., an airport) and the current time.
Initial state: This is specified by the problem.
Successor function: This returns the states resulting from taking any
scheduled flight (perhaps further specified by seat class and location), leaving
later than the current time plus the within-airport transit time, from the current
airport to another.
Goal test: Are we at the destination by some pre specified time?
Path cost: This depends on monetary cost, waiting time, flight time, customs
and immigration procedures, seat quality, time of day, type of airplane, frequent-
flyer mileage awards, and so on.

Route-finding problem is defined in terms of specified locations and transitions

along links between them. Route-finding algorithms are used in a variety of
applications, such as routing in computer networks, military operations planning,
and airline travel planning systems

The traveling salesperson problem (TSP) is a touring problem in which each

city must be visited exactly once. The aim is to find the shortest tour.

CSE – Panimalar Engineering College 31

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial
CS8691 – Artificial Intelligence

A VLSI layout problem requires positioning millions of components and

connections on a chip to minimize area, minimize circuit delays, minimize stray
capacitances, and maximize manufacturing yield. The layout problem comes
after the logical design phase, and is usually split into two parts: cell layout and
channel routing. In cell layout, the primitive components of the circuit are
grouped into cells, each of which performs some recognized function. Each cell
has a fixed footprint (size and shape) and requires a certain number of
connections to each of the other cells. The aim is to place the cells on the chip so
that they do not overlap and so that there is room for the connecting wires to be
placed between the cells. Channel routing finds a specific route for each wire
through the gaps between the cells.

Robot navigation is a generalization of the route-finding problem described

earlier. Rather than a discrete set of routes, a robot can move in a continuous
space with (in principle) an infinite set of possible actions and states. For a
circular robot moving on a flat surface, the space is essentially two-dimensional.
When the robot has arms and legs or wheels that must also be controlled, the
search space becomes many-dimensional. Advanced techniques are required just
to make the search space finite. In addition to the complexity of the problem,
real robots must also deal with errors in their sensor readings and motor
controls.

Automatic assembly sequencing of complex objects by a robot was first

demonstrated by FREDDY (Michie, 1972). In assembly problems, the aim is to
find an order in which to assemble the parts of some object. If the wrong order
is chosen, there will be no way to add some part later in the sequence without
undoing some of the work already done. Checking a step in the sequence for
feasibility is a difficult geometrical search problem closely related to robot
navigation

CSE – Panimalar Engineering College 32

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial
CS8691 – Artificial Intelligence

UNIT II PROBLEM SOLVING METHODS

Problem solving Methods - Search Strategies- Uninformed - Informed - Heuristics
- Local Search Algorithms and Optimization Problems - Searching with Partial
Observations - Constraint Satisfaction Problems – Constraint Propagation -
Backtracking Search - Game Playing - Optimal Decisions in Games – Alpha - Beta
Pruning - Stochastic Games.

Searching for Solutions

Search techniques use an explicit search tree that is generated by the initial
state and the successor function that together define the state space. In general,
we may have a search graph rather than a search tree, when the same state can
be reached from multiple paths

Example Route finding problem

The root of the search tree is a search node corresponding to the initial state,
In(Arad).

The first step is to test whether this is a goal state.

Apply the successor function to the current state, and generate a new set of
states

In this case, we get three new states: In(Sibiu),In(Timisoara), and In(Zerind).

Now we must choose which of these three possibilities to consider further.

CSE – Panimalar Engineering College 1

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial
CS8691 – Artificial Intelligence

Continue choosing, testing, and expanding until either a solution is found or

there are no more states to be expanded.

The choice of which state to expand is determined by the search strategy

Tree Search algorithm

Task : Find a path to reach F from A

1. Start the sequence with the initial state and check whether it is a goal state or
not.

a, If it is a goal state return success.

b. Otherwise perform the following sequence of steps

From the initial state (current state) generate and expand the new set of states.
The collection of nodes that have been generated but not expanded is called as
fringe. Each element of the fringe is a leaf node, a node with no successors in
the tree.

Expanding A

Expanding B

Expanding C

CSE – Panimalar Engineering College 2

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial
CS8691 – Artificial Intelligence

Sequence of steps to reach the goal state F from (A = A - C - F)

2. Search strategy: In the above example we did the sequence of choosing,
testing and expanding until a solution is found or until there are no more states
to be expanded. The choice of which state to expand first is determined by
search strategy.
3. Search tree: The tree which is constructed for the search process over the
state space.
4. Search node: The root of the search tree that is the initial state of the
problem.

The general tree search algorithm

function TREE-SEARCH(problem. strategy) returns a

solution or failure
initialize the search tree using the initial state of
problem
loop do
if there are no candidates for expansion then return
failure
choose a leaf node for expansion according to strategy
if the node contains a goal state then return the
corresponding solution
else expand the node and add the resulting nodes to the
search tree

There are many ways to represent nodes, but we will assume that a node is a
data structure with five components:

STATE: the state in the state space to which the node corresponds
PARENT-NODE: the node in the search tree that generated this node;
ACTION (RULE): the action that was applied to the parent to generate the
node;
PATH-COST: the cost, traditionally denoted by g(n) , of the path from the initial
state to the node
DEPTH: the number of steps along the path from the initial state.

The collection of nodes represented in the search tree is defined using set or
queue representation.

CSE – Panimalar Engineering College 3

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial
CS8691 – Artificial Intelligence

Set : The search strategy would be a function that selects the next node to be
expanded from the set

Queue: Collection of nodes are represented, using queue. The queue operations
are defined as:

MAKE-QUEUE(elements) - creates a queue with the given elements

EMPTY(queue)-returns true only if there are no more elements in the queue.
REM0VE-FIRST(queue) - removes the element at the front of the queue and
returns it
INSERT ALL (elements, queue) - inserts set of elements into the queue and
returns the resulting queue.
FIRST (queue) - returns the first element of the queue.
INSERT (element, queue) - inserts an element into the queue and returns the
resulting queue

The general tree search algorithm with queue representation

function TREE-SEARCH(problem,fringe) returns a

solution, or failure
fringe <- INSERT(MAKE-NODE(INITIAL-STATE[problem]),
fringe)
loop do
if EMPTY?(fringe) then return failure
node <- REMOVE-FIRST(fringe)
ifGOAL-TEST[problenl]applied to STATE[node] succeeds
then return SOLUTION(node)
fringe <- INSERT-ALL(EXPAND(node, problem),fringe)

CSE – Panimalar Engineering College 4

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial
CS8691 – Artificial Intelligence

function EXPAND(node, problem) returns a set of nodes

successors <- the empty set
for each <action, result> in SUCCESSOR-FN
[problem](STATE[node])do
S <- a new NODE
STATE[s] <- result
PARENT-NODE[s] <- node
ACTION[s] <- action
PATH-COST[s] <- PATH-COST[node]+STEP-COST(node,action,s)
DEPTH[s] <- DEPTH[node] + 1
add s to successors
return successors

Example: Route finding problem

Task. : Find a path to reach E using Queuing function in general tree search
algorithm

Measuring problem solving performance

The search strategy algorithms are evaluated depends on four important

criteria’s. They are:

CSE – Panimalar Engineering College 5

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial
CS8691 – Artificial Intelligence

(i) Completeness : The strategy guaranteed to find a solution when there is

one.
(ii) Time complexity : Time taken to run a solution
(iii) Space complexity : Memory needed to perform the search.
(iv) Optimality : If more than one way exists to derive the solution then the
best one is Selected

Definition of branching factor (b): The number of nodes which is connected

to each of the node in the search tree. Branching factor is used to find space and
time complexity of the search strategy

Solving Problems by Searching

The searching algorithms are divided into two categories

1. Uninformed Search Algorithms (Blind Search)

2. Informed Search Algorithms (Heuristic Search)

There are six Uninformed Search Algorithms

1. Breadth First Search

2. Uniform-cost search
3. Depth-first search
4. Depth-limited search
5. Iterative deepening depth-first search
6. Bidirectional Search

There are three Informed Search Algorithms

1. Best First Search

2. Greedy Search
3. A* Search
Blind search Vs Heuristic search

Blind search Heuristic search

CSE – Panimalar Engineering College 6

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial
CS8691 – Artificial Intelligence

No information about the number of The path cost from the current state to
steps (or) path cost from current state the goal state is calculated, to select
to goal state the minimum path cost as the next
state.
Less effective in search method More effective in search method
Problem to be solved with the given Additional information can be added as
information assumption to solve the problem

Breadth-first search

Breadth-first search is a simple strategy in which the root node is expanded

first, then all the successors of the root node are expanded next, then their
successors, and so on. In general, all the nodes are expanded at a given depth
in the search tree before any nodes at the next level are expanded.

Breadth-first search can be implemented by calling TREE-SEARCH with an empty

fringe that is a first-in-first-out (FIFO) queue, assuring that the nodes that are
visited first will be expanded first.

In other words, calling TREE-SEARCH(Problem, FIFO-QUEUE())results in a

breadth-first search. The FIFO queue puts all newly generated successors at the
end of the queue, which means that shallow nodes are expanded before deeper
nodes

Breadth first search trees after node expansions

Example: Route finding problem

CSE – Panimalar Engineering College 7

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial
CS8691 – Artificial Intelligence

Task: Find a ,path from. S to G using BFS

The path in the 2nd depth level is selected, (i.e) SBG{or) SCG.

CSE – Panimalar Engineering College 8

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial
CS8691 – Artificial Intelligence

Algorithm :

function BREADTH-FIRST-SEARCH(problem) returns a solution, or failure

node ←a node with STATE = problem.INITIAL-STATE, PATH-COST = 0
if problem.GOAL-TEST(node.STATE) then return SOLUTION(node)
frontier ←a FIFO queue with node as the only element
explored ←an empty set
loop do
if EMPTY?( frontier ) then return failure
node←POP( frontier ) /* chooses the shallowest node in frontier */
add node.STATE to explored
for each action in problem.ACTIONS(node.STATE) do
child ←CHILD-NODE(problem, node, action)
if child.STATE is not in explored or frontier then
if problem.GOAL-TEST(child.STATE) then return SOLUTION(child)
frontier ←INSERT(child, frontier )

Time and space complexity:

Example:

Time complexity

= 1 +b + b 2 + . . . . . . . + b d

= O(b d)

The space complexity is same as time complexity because all the leaf nodes of
the tree must be maintained in memory at the same time = O(b d)

Completeness: Yes

Optimality: Yes, provided the path cost is a non decreasing function of the
depth of the node

CSE – Panimalar Engineering College 9

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial
CS8691 – Artificial Intelligence

Advantage: Guaranteed to find the single solution at the shallowest depth level

Disadvantage: Suitable for only smallest instances problem (i.e.) (number of

levels to be minimum (or) branching factor to be minimum)
')

Uniform-cost search

function UNIFORM-COST-SEARCH(problem) returns a solution, or failure

node ←a node with STATE = problem.INITIAL-STATE, PATH-COST = 0
frontier ←a priority queue ordered by PATH-COST, with node as the only
element
explored ←an empty set
loop do
if EMPTY?( frontier ) then return failure
node←POP( frontier ) /* chooses the lowest-cost node in frontier */
if problem.GOAL-TEST(node.STATE) then return SOLUTION(node)
add node.STATE to explored
for each action in problem.ACTIONS(node.STATE) do
child ←CHILD-NODE(problem, node, action)
if child.STATE is not in explored or frontier then
frontier ←INSERT(child, frontier )
else if child.STATE is in frontier with higher PATH-COST then
replace that frontier node with child

Breadth-first search is optimal when all step costs are equal, because it always
expands the shallowest unexpanded node. By a simple extension, we can find an
algorithm that is optimal with any step cost function. Instead of expanding the
shallowest node, uniform-cost search expands the node n with the lowest
path cost. Note that if all step costs are equal, this is identical to breadth-first
search.

Uniform-cost search does not care about the number of steps a path has, but
only about their total cost.

Example: Route finding problem

CSE – Panimalar Engineering College 10

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial
CS8691 – Artificial Intelligence

Task : Find a minimum path cost from S to G

Since the value of A is less it is expanded first, but it is not optimal.

B to be expanded next

SBG is the path with minimum path cost.

No need to expand the next path SC, because its path cost is high to reach C
from S, as well as goal state is reached in the previous path with minimum cost.

Time and space complexity:

Time complexity is same as breadth first search because instead of depth level
the minimum path cost is considered.

CSE – Panimalar Engineering College 11

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial
CS8691 – Artificial Intelligence

Time complexity: O(b d) Space complexity: O(b d)

Completeness: Yes Optimality: Yes

Advantage: Guaranteed to find the single solution at minimum path cost.

Disadvantage: Suitable for only smallest instances problem.

Depth-first search

Depth-first search always expands the deepest node in the current fringe of
the search tree

The search proceeds immediately to the deepest level of the search tree, where
the nodes have no successors. As those nodes are expanded, they are dropped
from the fringe, so then the search "backs up" to the next shallowest node that
still has unexplored successors. This strategy can be implemented by TREE-
SEARCH with a last-in-first-out (LIFO) queue, also known as a stack.

Depth first search tree with 3 level expansion

Example: Route finding problem

CSE – Panimalar Engineering College 12

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial
CS8691 – Artificial Intelligence

Task: Find a path from S to G using DFS

The path in the 3rd depth level is selected. (i.e. S-A-D-G

Algorithm:

function DFS(problem) return a solution or failure

TREE-SEARCH(problem, LIFO-QUEUE())

Time and space complexity:

In the worst case depth first search has to expand all the nodes

Time complexity : O(bm).

The nodes are expanded towards one particular direction requires memory for
only that nodes.

Space complexity : O(bm)

CSE – Panimalar Engineering College 13

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial
CS8691 – Artificial Intelligence

b=2
m = 2 :. bm=4

Completeness: No

Optimality: No

Advantage: If more than one solution exists (or) number of levels is high then
DFS is best because exploration is done only in a small portion of the whole
space.

Disadvantage: Not guaranteed to find a solution

Depth - limited search

1. Definition: A cut off (maximum level of the depth) is introduced in this search
technique to overcome the disadvantage of depth first search. The cutoff value
depends on the number of states.

Example: Route finding problem

The number of states in the given map is 5. So, it is possible to get the goal
state at a maximum depth of 4. Therefore the cutoff value is 4

CSE – Panimalar Engineering College 14

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial
CS8691 – Artificial Intelligence

Task : Find a path from A to E.

A recursive implementation of depth-limited search

function DEPTH-LIMITED-SEARCH(problem, limit) returns a

solution, or failure/cutoff
return RECURSIVE-DLS(MAKE-NODE(INITIAL-STATE [problem]),
problem, limit)

function RECURSIVE-DLS(node, problem, limit) returns a

solution, or failure/cutoff
cutoff-occurred? <- false
if GOAL-TEST[problem](STATE[node]) then return
SOLUTION(node)
else if DEPTH[node] =limit then return cutoff
else for each successor in EXPAND(node, problem) do
result <- RECURSIVE-DLS(successor, problem, limit)
if result = cutoff then cutoff-occurred?<- true
else if result  failure then return result
if cutoff-occurred? then return cutoff else return
failure

Time and space complexity:

CSE – Panimalar Engineering College 15

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial
CS8691 – Artificial Intelligence

The worst case time complexity is equivalent to BFS and worst case DFS.
Time complexity : O(bl)

The nodes which is expanded in one particular direction above to be stored.

Space complexity : O(bl)

Optimality: No, because not guaranteed to find the shortest solution first in the
search technique.

Completeness : Yes, guaranteed to find the solution if it exists.

Advantage: Cut off level is introduced in the DFS technique

Disadvantage : Not guaranteed to find the optimal solution.

Iterative deepening search

Iterative deepening search

Definition: Iterative deepening search is a strategy that sidesteps the issue of

choosing the best depth limit by trying all possible depth limits.

Example: Route finding problem

Task: Find a path from A to G

Limit = 0

Limit = 1

CSE – Panimalar Engineering College 16

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial
CS8691 – Artificial Intelligence

Limit = 2

Solution: The goal state G can be reached from A in four ways. They are:

1. A – B – D - E – G ------- Limit 4
2. A - B - D - E - G ------- Limit 4
3. A - C - E - G ------- Limit 3
4. A - F - G ------ Limit2

Since it is a iterative deepening search it selects lowest depth limit (i.e.) A-F-G is
selected as the solution path.

The iterative deepening search algorithm :

function ITERATIVE-DEEPENING-SEARCH (problem) returns a

solution, or failure
inputs : problem
for depth <- 0 to  do
result <-DEPTH-LIMITED-SEARCH(problem, depth)
if result  cutoff then return result

Time and space complexity :

Iterative deepening combines the advantage of breadth first search and depth
first search (i.e) expansion of states is done as BFS and memory requirement is
equivalent to DFS.

Time complexity : O(bd)

Space Complexity : O(bd)

Optimality: Yes, because the order of expansion of states is similar to breadth

first search.

CSE – Panimalar Engineering College 17

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial
CS8691 – Artificial Intelligence

Completeness: yes, guaranteed to find the solution if it exists.

Advantage: This method is preferred for large state space and the depth of the
search is not known.

Disadvantage : Many states are expanded multiple times

Example : The state D is expanded twice in limit 2

Bidirectional search

Definition : Bidirectional search is a strategy that simultaneously search both

the directions (i.e.) forward from the initial state and backward from the goal,
and stops when the two searches meet in the middle.

Example: Route finding problem

Task : Find a path from A to E.

Search from forward (A) :

Search from backward (E) :

Time and space complexity:

The forward and backward searches done at the same time will lead to the
solution in O(2bd/2) = O(bd/2)step, because search is done to go only halfway

CSE – Panimalar Engineering College 18

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial
CS8691 – Artificial Intelligence

If the two searches meet at all, the nodes of at least one of them must all be
retained in memory requires O(bd/2) space.
Optimality: Yes, because the order of expansion of states is done in both the
directions.

Completeness: Yes, guaranteed to find the solution if it exists.

Advantage : Time and space complexity is reduced.

Disadvantage: If two searches (forward, backward) does not meet at all,

complexity arises in the search technique. In backward search calculating
predecessor is difficult task. If more than one goal state 'exists then explicit,
multiple state search is required

Comparing uninformed search strategies

Criterion Breadth Uniform Depth Depth Iterative Bi

First Cost First Limited Deepening direction
Complete Yes Yes No No Yes Yes
Time O(bd) O(bd) O(bm) O(bl) O(bd) O(bd/2)
Space O(bd) O(bd) O(bm) O(bl) O(bd) O(bd/2)
Optimal Yes Yes No No Yes Yes

Avoiding Repeated States

The most important complication of search strategy is expanding states that

have already been encountered and expanded before on some other path

A state space and its exponentially larger search tree

CSE – Panimalar Engineering College 19

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial
CS8691 – Artificial Intelligence

The repeated states can be avoided using three different ways. They are:

1. Do not return to the state you just came from (i.e) avoid any successor that is
the same state as the node's parent.
2. Do not create path with cycles (i.e) avoid any successor of a node that is the
same as any of the node's ancestors.
3. Do not generate any state that was ever generated before.

The general TREE-SEARCH algorithm is modified with additional data structure,

such as :

Closed list - which stores every expanded node.

Open list - fringe of unexpanded nodes.

If the current node matches a node on the closed list, then it is discarded and it
is not considered for expansion. This is done with GRAPH-SEARCH algorithm.
This algorithm is efficient for problems with many repeated states

function GRAPH-SEARCH (problem, fringe) returns a

solution, or failure
closed <- an empty set
fringe <- INSERT (MAKE-NODE(INITIAL-STATE[problem]),
fringe)
loop do
if EMPTv?(fringe) then return failure
node <- REMOVE-FIKST (fringe)
if GOAL-TEST [problem ](STATE[node]) then return SOLUTION
(node)
if STATE [node] is not in closed then
add STATE [node] to closed
fringe <- INSERT-ALL(EXPAND(node, problem), fringe)

The worst-case time and space requirements are proportional to the size of the
state space, this may be much smaller than O(bd)

Informed search and exploration

Uninformed search strategies can find solutions to problems by systematically

generating new states and testing them against the goal. These strategies are
inefficient in most cases.

CSE – Panimalar Engineering College 20

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial
CS8691 – Artificial Intelligence

An informed search Strategy uses problem-specific knowledge and it can find

solutions more efficiently.

Informed Heuristic Search Strategies

An informed search strategy uses problem-specific knowledge beyond the

definition of the problem itself and it can find solutions more efficiently than an
uninformed strategy.

The general approach is best first search that uses an evaluation function in
TREE-SEARCH or GRAPH-SEARCH.

Best-first search is an instance of the general TREE-SEARCH or GRAPH-SEARCH

algorithm in which a node is selected for expansion based on an evaluation
function, f(n)

The node with the lowest evaluation is selected for expansion, because the
evaluation measures distance to the goal.

Best-first search can be implemented within our general search framework via a
priority queue, a data structure that will maintain the fringe in ascending order of
f –values

Implementation of Best-first search using general search algorithm

function BEST-FIRST-SEARCH(problem, EVAL-FN) returns a

solution sequence
inputs: problem, a problem
EVAL-FN, an evaluation function
QUEUEING -FN<- a function that orders nodes by EVAL-FN
return TREE-SEARCH(problem, QUEUEING-FN)

The key component of these algorithms is a heuristic functions denoted h(n)

h(n) = estimated cost of the cheapest path from node n to a goal node.

One constraint: if n is a goal node, then h(n) = 0

The two types of evaluation functions are:

(i) Expand the node closest to the goal state using estimated cost as the
evaluation is called greedy best first search.

CSE – Panimalar Engineering College 21

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial
CS8691 – Artificial Intelligence

(ii) Expand the node on the least cost solution path using estimated cost and
actual cost as the evaluation function is called A*search

Greedy best first search (Minimize estimated cost to reach a goal)

Definition : A best first search that uses h(n) to select next node to expand is
called greedy search
Evaluation function : The estimated cost to reach the goal state, denoted by
the letter h(n)

h(n)= estimated cost of the cheapest path from the state

at node n to a goal state

Algorithm :

Function GREEDY-BEST-FIRST SEARCH (problem) returns a

solution or failure
return BEST-FIRST-SEARCH (problem, h)

Example 1 : Route Finding Problem

Problem : Route finding Problem from Arad to Burcharest

Heuristic function : A good heuristic function for route-finding problems is

Straight-Line Distance to the goal and it is denoted as h SLD(n).

CSE – Panimalar Engineering College 22

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial
CS8691 – Artificial Intelligence

hSLD(n) = Straight-Line distance between n and the goal

locatation

Note : The values of hSLD(n) cannot be computed from the problem description
itself. Moreover, it takes a certain amount of experience

Values of hSLD-straight-line distances to Bucharest

Solution :

From the given graph and estimated cost, the goal state is identified as B u c h
a r e s t from Arad. Apply the evaluation function h (n) to find a path from Arad
to Burcharest from A to B

CSE – Panimalar Engineering College 23

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial
CS8691 – Artificial Intelligence

The first node to be expanded from Arad will be Sibiu, because it is closer to
Bucharest than either Zerind or Timisoara.

The next node to be expanded will be Fagaras, because it is closest.

Fagaras in turn generates Bucharest, which is the goal.

For this particular problem, greedy best-first search using h SLD finds a solution
without ever expanding a node that is not on the solution path; hence, its search
cost is minimal. It is not optimal, however: the path via Sibiu and Fagaras to
Bucharest is 32 kilometers longer than the path through Rimnicu Vilcea and
Pitesti. This shows why the algorithm is called "greedy'-at each step it tries to get
as close to the goal as it can.

Minimizing h(n) is susceptible to false starts. Consider the problem of getting

from Iasi to Fagaras. The heuristic suggests that Neamt be expanded first,
because it is closest to Fagaras, but it is a dead end. The solution is to go first to
Vaslui-a step that is actually farther from the goal according to the heuristic-and
then to continue to Urziceni, Bucharest, and Fagaras.

Time and space complexity : Greedy search resembles depth first search,
since it follows one path to the goal state, backtracking occurs when it finds a
dead end. The worst case time complexity is equivalent to depth first search,
that is O(b m), where m is the maximum depth of the search space. The greedy

CSE – Panimalar Engineering College 24

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial
CS8691 – Artificial Intelligence

search retains all nodes in memory, therefore the space complexity is also O(bm)
The time and space complexity can be reduced with good heuristic function.

Optimality : It is not optimal, because the next level node for expansion is
selected only depends on the estimated cost and not the actual cost.

Completeness : No, because it can start down with an infinite path and never
return to try other possibilities.

Example 2 : Finding the path from one node to another node

Solution :

From the given graph and estimated cost, the goal state IS identified as B from
A.

Apply the evaluation function h(n) to find a path from A to B

CSE – Panimalar Engineering College 25

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial
CS8691 – Artificial Intelligence

From F, goal state B is reached. Therefore the path from A to Busing greedy
search is A - S - F - B = 450 (i.e) (140 + 99 + 211)

A* search ( Minimizing the total estimated solution cost)

The most widely-known form of best-first search is called A* search (pronounced

"A-star search"). A* search is both complete and optimal.
It evaluates nodes by combining g(n), the cost to reach the node, and h(n.),the
cost to get from the node to the goal

f(n) =g(n) + h(n)

g(n) - path cost from the start node to node n

h(n) - estimated cost of the cheapest path from n to the goal
f (n) - estimated cost of the cheapest solution through n

A* Algorithm

function A* SEARCH(problem) returns a solution or failure

return BEST-FIRST-SEARCH (problem, g+h)

CSE – Panimalar Engineering College 26

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial
CS8691 – Artificial Intelligence

Example 1 : Route Finding Problem

Problem : Route finding Problem from Arad to Burcharest

Heuristic function : A good heuristic function for route-finding problems is

Straight-Line Distance to the goal and it is denoted as h SLD(n).

hSLD(n) = Straight-Line distance between n and the goal locatation

Values of hSLD-straight-line distances to Bucharest

Stages in an A* search for Bucharest. Nodes are labeled with f (n) = g

(n) + h(n)

CSE – Panimalar Engineering College 27

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial
CS8691 – Artificial Intelligence

Example 2 : Finding the path from one node to another node

CSE – Panimalar Engineering College 28

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial
CS8691 – Artificial Intelligence

Solution:

From the given graph and estimated cost, the goal state is identified as B from A
Apply the evaluation function f(n) = g(n) +h(n) to find a path from A to B

CSE – Panimalar Engineering College 29

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial
CS8691 – Artificial Intelligence

From P, goal state B is reached. Therefore the path from A to B using A* search
is A – S - R - P -B : 418 (ie) {140 + 80 + 97 + 101), that the path cost is less
than Greedy search path cost.

Time and space complexity: Time complexity depends on the heuristic

function and the admissible heuristic value. Space complexity remains in the
exponential order.

The behavior of A* search

Monotonicity (Consistency)

In search tree any path from the root, the f- cost never decreases. This condition
is true for almost all admissible heuristics. A heuristic which satisfies this property
is called monotonicity(consistency).

A heuristic h(n) is consistent if, for every node n and every successor n' of n
generated by any action a, the estimated cost of reaching the goal from n is no

CSE – Panimalar Engineering College 30

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial
CS8691 – Artificial Intelligence

greater than the step cost of getting to n' plus the estimated cost of reaching the
goal from n':

If the heuristic is non-monotonic, then we have to make a minor correction that

restores monotonicity.

Example for monotonic

Let us consider two nodes n and n’, where n is the parent of n’

For example

g(n) = 3 and h(n) = 4. then f(n) = g(n) + h(n) = 7.

g(n’) = 54 and h(n’) = 3. then f(n’) = g(n’) + h(n’) = 8

Example for Non-monotonic

Let us consider two nodes n and n’, where n is the parent of n’. For example

g(n) = 3 and h(n) = 4. then f(n) = g(n) + h(n) = 7.

g(n’) = 4 and h(n’) = 2. then f(n’) = g(n’) + h(n’) = 6.

To reach the node n the cost value is 7, from there to reach the node n' the
value of cost has to increase as per monotonic property. But the above example
does not satisfy this property. So, it is called as non-monotonic heuristic.

How to avoid non-monotonic heuristic?

We have to check each time when we generate anew node, to see if its f-cost is
less that its parent’s f-cost; if it is we have to use the parent’s f- cost instead.

Non-monotonic heuristic can be avoided using path-max equation.

CSE – Panimalar Engineering College 31

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial
CS8691 – Artificial Intelligence

f(n') = max (f{n), g(n') + h(n'))

Optimality

A* search is complete, optimal, and optimally efficient among all algorithms

A* using GRAPH-SEARCH is optimal if h(n) is consistent.

Completeness

A* is complete on locally finite graphs (graphs with a finite branching factor)

provided there is some constant d such that every operator costs at least d.

Drawback

A* usually runs out of space because it keeps all generated nodes in memory

Memory bounded heuristic search

The simplest way to reduce memory requirements for A* is to adapt the idea of
iterative deepening to the heuristic search context, resulting in the iterative-
deepening A" (IDA*) algorithm.

The memory requirements of A* is reduced by combining the heuristic function

with iterative deepening resulting an IDA* algorithm.

Iterative Deepening A* search (IDA*)

Depth first search is modified to use an f-cost limit rather than a depth limit for
IDA* algorithm.

Each iteration in the search expands all the nodes inside the contour for the
current f-cost and moves to the next contour with new f - cost.

Space complexity is proportional to the longest path of exploration that is bd is

a good estimate of storage requirements

Time complexity depends on the number of different values that the heuristic
function can take on

Optimality: yes, because it implies A* search.

Completeness: yes, because it implies A* search.

CSE – Panimalar Engineering College 32

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial
CS8691 – Artificial Intelligence

Disadvantage: It will require more storage space in complex domains (i.e) Each
contour will include only one state with the previous contour. To avoid this, we
increase the f-cost
limit by a fixed amount  on each iteration, so that the total number of iteration
is proportional to 1/  . Such an algorithm is called  admissible.

The two recent memory bounded algorithms are:

 Recursive Best First Search (RBfS)

 Memory bounded A* search (MA*)

Recursive Best First Search (RBFS)

A recursive algorithm with best first search technique uses only linear space.
It is similar to recursive depth first search with an inclusion (i.e.) keeps track of
the f-value of the best alternative path available from any ancestor of the current
node.

If the current node exceeds this limit, the recursion unwinds back to the
alternative path and replaces the f-value of each node along the path with the
best f-value of its children.

The main idea lies in to keep track of the second best alternate node (forgotten
node) and decides whether it's worth to reexpand the subtree at some later
time.

CSE – Panimalar Engineering College 33

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial
CS8691 – Artificial Intelligence

Algortihm For Recursive Best-First Search

function RECURSIVE-BEST-FIRST-SEARCH(problem) returns a

solution, or failure
RBFS(problem,MAKE-NODE(INITIAL-STATE[problem]),  )
function, RBFS(problem, node, f_limit) returns a
solution, or failure and a new f-cost limit
if GOAL-TEST[problem](state) then return node
successors <- EXPAND(node, problem)
if successors is empty then return failure, 
for each s in successors do
f[s]<-max(g(s) + h(s),f[node])
repeat
best <- the lowest f-value node in successors
if f[best] > f_limit then return failure,f[best]
alternative <- the second-lowest f-value among successors
result,f[best]<-
RBFS(problem,best,min(f_limit,alternative))
if result  failure then return result

Stages in an RBFS search for the shortest route to Bucharest .

CSE – Panimalar Engineering College 34

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial
CS8691 – Artificial Intelligence

Example:

a) After expanding A, S, and R, the current best leaf(P) has a value that is worse
than the best alternative path (F)

f-limit value of each recursive call is shown on top of each current node. After
expanding R, the condition f[best] >f-limit (417 > 415) is true and returns
f[best] to that node.

b) After unwinding back to and expanding F

Here the f[best] is 450 and which is greater than the f-limit of 417. Therefore if
returns and unwinds with f[best] value to that node.

CSE – Panimalar Engineering College 35

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial
CS8691 – Artificial Intelligence

c) After switching back to Rand expanding P.

The best alternative path through T costs at least 447, therefore the path
through R and P is considered as the best one.

Time and space complexity : RBFS is an optimal algorithm if the heuristic

function h(n) is admissible. Its time complexity depends both on the accuracy of
the heuristic function and on how often the best path changes as nodes are
expanded. Its space complexity is O(bd), even though more is available.

A search techniques (algorithms) which uses all available memory are:

a. MA* (Memory - bounded A*)

b. SMA* (Simplified MA*)

Simplified Memory - bounded A* search (SMA*)

SMA* algorithm can make use of all available memory to carry out the search.

Properties of SMA* algorithm:

(a) It will utilize whatever memory is made available to it.

(b) It avoids repeated states as far as its memory allows.

It is complete if the available memory is sufficient to store the deepest solution

path.
It is optimal if enough memory is available to store the deepest solution path.
Otherwise, it returns the best solution that can be reached with the available
memory.

Advantage: SMA* uses only the available memory.

CSE – Panimalar Engineering College 36

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial
CS8691 – Artificial Intelligence

Disadvantage: If enough memory is not available it leads to unoptimal

solution.

Space and Time complexity: depends on the available number of node.

The SMA* Algorithm

function SMA*(problem) returns a solution sequence

inputs: problem, a problem
local variables: Queue, a queue of nodes ordered by
f-cost

Queue<-MAKE-QUEUE({MAKE-NODE(INITIAL-STATE[problem])} )
loop do
if Queue is empty then return failure
n<-deepest least-f-cost node in Queue
if GOAL-TEST(n) then return success
s<-NEXT-SUCCESSOR(n)
if s is not a goal and is at maximum depth then
f{s)<- 
else
f{s)<- MAX(f(n), g(s)+h(s))
if all of n’s successors have been generated then
update n’s f-cost and those of its ancestors if necessary
if SUCCESSORS(n) all in the memory then remove n from
Queue
if memory is full then
delete shallowest, highest-f-cost node in Queue
remove it from its parent's successor list
insert its parent on Queue if necessary
insert s on Queue
end

CSE – Panimalar Engineering College 37

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial
CS8691 – Artificial Intelligence

Example:

The values at the nodes are given as per the A* function i.e. g+h=f

From the Figure we identified that the goal states are D,F,J,I because the h value
of these nodes are zero (marked as a square)

Available memory - 3 nodes of storage space.

Task: Find a optimal path from A to anyone of the goal state.

Solution:

CSE – Panimalar Engineering College 38

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial
CS8691 – Artificial Intelligence

HEURISTIC FUNCTIONS

The 8-puzzle was one of the earliest heuristic search problems.

Given :

CSE – Panimalar Engineering College 39

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial
CS8691 – Artificial Intelligence

Task : Find the shortest solution using heuristic function that never over
estimates the number of steps to the goal.

Solution : To perform the given task two candidates are required, which are
named as h1 and h2

h1 = the number of misplaced tiles.

All of the eight tiles are out of position in the above figure, so the start state
would have hl = 8. hl is an admissible heuristic, because it is clear that any tile
that is out of place must be moved at least once.

h2 = the sum of the distances of the tiles from their goal positions. Because tiles
cannot move along diagonals, the distance we will count is the sum of the
horizontal and vertical distances. This is called as the city block distance or
Manhattan distance. h2 is also admissible, because any move can do is move
one tile one step closer to the goal. Tiles 1 to 8 in the start state give a
Manhattan distance of

h2=3+1+2+2+2+3+3+2=18.

True solution cost is h1 + h2 = 26

Example :

h1=7

h2 = 2 + 3 + 3 + 2 + 4 + 2 + 0 + 2 = 18

True Solution Cost is h1 + h2 = 25

Effective branching factor(b*)

CSE – Panimalar Engineering College 40

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial
CS8691 – Artificial Intelligence

In the search tree, if the total number of nodes expanded by A* for a particular
problem is N, and the solution depth is d, then b* is the branching factor that a
uniform tree of depth d, would have N nodes. Thus:

N= l+ b* + (b*)2 + (b*)3 + +(b*)d

Example:

For example, if A* finds a solution at depth 5 using 52 nodes, then the effective
branching factor is 1.92.

Depth = 5
N = 52

Effective branching factor is 1.92.

Relaxed problem

A problem with less restriction on the operators is called a relaxed problem. If

the given problem is a relaxed problem then it is possible to produce good
heuristic function. Example: 8 puzzle problem, with minimum number of
operators.

Local Search Algorithms And Optimization Problems

In many optimization problems, the path to the goal is irrelevant; the goal state
itself is the solution.

The best state is identified from the objective function or heuristic cost function.
In such cases, we can use local search algorithms (ie) keep only a single current
state, try to improve it instead of the whole search space explored so far

For example, in the 8-queens problem, what matters is the final configuration of
queens, not the order in which they are added.

Local search algorithms operate a single current state (rather than multiple
paths) and generally move only to neighbors of that state. Typically, the paths
followed by the search are not retained.

They have two key advantages:

CSE – Panimalar Engineering College 41

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial
CS8691 – Artificial Intelligence

(1) They use very little memory-usually a constant amount; (2) They can often
find reasonable solutions in large or infinite (continuous) state spaces for which
systematic algorithms are unsuitable.

The local search problem is explained with the state space land scape. A
landscape has:

Location - defined by the state

Elevation - defined by the value of the heuristic cost function or objective

function, if elevation corresponds to cost then the lowest valley (global
minimum) is achieved. If elevation corresponds to an objective function, then the
highest peak (global maximum) is achieved.

A complete local search algorithm always finds a goal if one exists, an optimal
algorithm always finds a global minimum/maximum.

A one-dimensional state space landscape in which elevation

corresponds to the objective function.

Applications

Integrated - circuit design

Factory - floor layout
Job-shop scheduling
Automatic programming
Vehicle routing
Telecommunications network Optimization
Advantages

 Constant search space. It is suitable for online and offline search

CSE – Panimalar Engineering College 42

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial
CS8691 – Artificial Intelligence

 The search cost is less when compare to informed search

 Reasonable solutions are derived in large or continuous state space for
which systematic algorithms are unsuitable.

Some of the local search algorithms are:

1. Hill climbing search (Greedy Local Search)

2. Simulated annealing
3. Local beam search
4. Genetic Algorithm (GA)

Hill Climbing Search (Greedy Local Search)

The hill-climbing search algorithm is simply a loop that continually moves in

the direction of increasing value. It terminates when it reaches a "peak" where
no neighbor has a higher value. The algorithm does not maintain a search tree,
so the current node data structure need only record the state and its objective
function value. At each step the current node is replaced by the best neighbor;

Hill-climbing search algorithm

function HILL-CLIMBING(problem) returns a state that is a

local maximum
inputs: problem, a problem
local variables:current, a node and neighbor, a node
current <- MAKE-NODE(INITIAL-STATE[problem])
loop do
neighbor <- a highest-valued successor of current
if VALUE[neighbor] <= VALUE[current] then
return STATE[current]
current <- neighbor

To illustrate hill-climbing, we will use the 8-queens, where each state has 8
queens on the board, one per column. The successor function returns all possible
states generated by moving a single queen to another square in the same
column (so each state has 8 x 7 = 56 successors).

Hill-climbing algorithms typically choose randomly among the set of best

successors, if there is more than one.

The heuristic cost function h is the number of pairs of queens that are attacking
each other, either directly or indirectly.

CSE – Panimalar Engineering College 43

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial
CS8691 – Artificial Intelligence

The global minimum of this function is zero, which occurs only at perfect
solutions.

An 8-queens state with heuristic cost estimate h = 17, showing the value of h for
each possible successor obtained by moving a queen within its column. The best
moves are marked.

A local minimum in the 8-queens state space; the state has h = 1 but every
successor has a higher cost.

CSE – Panimalar Engineering College 44

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial
CS8691 – Artificial Intelligence

Hill climbing often gets stuck for the following reasons:

Local maxima or foot hills : a local maximum is a peak that is higher than
each of its neighboring states, but lower than the global maximum

Example :

The evaluation function value is maximum at C and from their there is no path
exist for expansion. Therefore C is called as local maxima. To avoid this state,
random node is selected using back tracking to the previous node.

Plateau or shoulder: a plateau is an area of the state space landscape where

the evaluation function is flat. It can be a flat local maximum.

Example :

The evaluation function value of B C D are same, this is a state space of plateau.
To avoid this state, random node is selected or skip the level (i.e) select the
node in the next level

CSE – Panimalar Engineering College 45

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial
CS8691 – Artificial Intelligence

Ridges: Ridges result in a sequence of local maxima that is very difficult for
greedy algorithms to navigate. But the disadvantage is more calculations to be
done function

Structure of hill climbing drawbacks

Variants of hill-climbing

Stochastic hill climbing - Stochastic hill climbing chooses at random from

among the uphill moves; the probability of selection can vary with the steepness
of the uphill move.

First-choice hill climbing - First-choice hill climbing implements stochastic hill

climbing by generating successors randomly until one is generated that is better
than the current state

Random-restart hill climbing - Random-restart hill climbing adopts the well

known adage, "If at first you don't succeed, try, try again." It conducts a series
of hill-climbing searches from randomly generated initial state, stopping when a
goal is found.

Simulated annealing search

An algorithm which combines hill climbing with random walk to yield both
efficiency and completeness

In metallurgy, annealing is the process used to temper or harden metals and

glass by heating them to a high temperature and then gradually cooling them

When the search stops at the local maxima, we will allow the search to take
some down Hill steps to escape the local maxima by allowing some "bad" moves
but gradually decrease their size and frequency. The node is selected randomly
and it checks whether it is a best move or not. If the move improves the
situation, it is executed.  E variable is introduced to calculate the probability of
worsened. A Second parameter T is introduced to determine the probability.

CSE – Panimalar Engineering College 46

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial
CS8691 – Artificial Intelligence

The simulated annealing search algorithm

function SIMULATED-ANNEALING(problem, schedule) returns a

solution state
inputs: problem, a problem
schedule, a mapping from time to "temperature"
local variables: current, a node
next, a node
T, a "variable" controlling the probability of downward
steps
current <- MAKE-NODE(INITIAL-STATE[problem])
for t<- l to  do
T <- schedule[t]
if T = 0 then return current
next <- a randomly selected successor of current
 E <- VALUE[next] - VALUE[current]
if  E > 0 then current <- next
else current <- next only with probability e  E/T

Property of simulated annealing search

T decreases slowly enough then simulated annealing search will find a global
optimum with probability approaching one

Applications

VLSI layout
Airline scheduling

Local beam search

Local beam search is a variation of beam search which is a path based algorithm.
It uses K states and generates successors for K states in parallel instead of one
state and its successors in sequence. The useful information is passed among the
K parallel threads.

The sequence of steps to perform local beam search is given below:

CSE – Panimalar Engineering College 47

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial
CS8691 – Artificial Intelligence

 Keep track of K states rather than just one.

 Start with K randomly generated states.
 At each iteration, all the successors of all K states are generated.
 If anyone is a goal state stop; else select the K best successors from the
complete list and repeat.

This search will suffer from lack of diversity among K states.

Therefore a variant named as stochastic beam search selects K successors at

random, with the probability of choosing a given successor being an increasing
function of its value.

Genetic Algorithms (GA)

A genetic algorithm (or GA) is a variant of stochastic beam search in which

successor states are generated by combining two parent states, rather than by
modifying a single state

GA begins with a set of k randomly generated states, called the population.

Each state, or individual, is represented as a string over a finite alphabet.

For Example an 8 queen’s state could be represented as 8 digits, each in the

range from 1 to 8.

Initial population: K randomly generated states of 8 queen problem

Individual (or) state: Each string in the initial population is individual (or)
state. In one state, the position of the queen of each column is represented.

Example: The state with the value 24748552 is derived as follows:

The Initial Population (Four randomly selected States) are :

CSE – Panimalar Engineering College 48

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial
CS8691 – Artificial Intelligence

Evaluation function (or) Fitness function: A function that returns higher

values for better State. For 8 queens problem the number of non attacking pairs
of queens is defined as fitness function

Minimum fitness value is 0

Maximum fitness value is : 8*7/2 = 28 for a solution

The values of the four states are 24, 23, 20, and 11.

The probability of being chosen for reproduction is directly proportional to the

fitness score, which is denoted as percentage.

24 / (24+23+20+11) = 31%
23 / (24+23+20+11) = 29%
20 / (24+23+20+11) = 26%
11 / (24+23+20+11) = 14%

Selection : A random choice of two pairs is selected for reproduction, by

considering the probability of fitness function of each state. In the example one
state is chosen twice probability of 29%) and the another one state is not
chosen (Probability of 14%)

Cross over: Each pair to be mated, a crossover point is randomly chosen. For
the first pair the crossover point is chosen after 3 digits and after 5 digits for the
second pair.

CSE – Panimalar Engineering College 49

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial
CS8691 – Artificial Intelligence

offspring : Offspring is created by crossing over the parent strings in the

crossover point. That is, the first child of the first pair gets the first 3 digits from
the first parent and the remaining digits from the second parent. Similarly the
second child of the first pair gets the first 3 digits from the second parent and
the remaining digits from the first parent.

The 8-queens states corresponding to the first two parents

Mutation : Each location is subject to random mutation with a small

independent probability. One digit was mutated in the first, third, and fourth
offspring

CSE – Panimalar Engineering College 50

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial
CS8691 – Artificial Intelligence

Production of Next Generation of States

The initial population in (a) is ranked by the fitness function in (b), resulting in
pairs for mating in (c). They produce offspring in (d), which are subject to
mutation in(e).

function GENETIC-ALGORITHM(population, FITNESS-FN)

returns an individual
inputs: population, a set of individuals
FITNESS-FN, a function that measures the fitness of an
individual
repeat
new-population <- empty set
loop for i from 1 to SIZE(population) do
x <- RANDOM-SELECTION(Population,FITNESS-FN)
y <- RANDOM-SELECTION(Population,FITNESS-FN)
child <- REPRODUCE(yX),
if (small random probability) then child MUTATE(chi1d)
add child to new-population
population <- new-population
until some individual is fit enough, or enough time has
elapsed
return the best individual in population, according to
FITNESS-FN
function REPRODUCE(x,y), returns an individual
inputs: x, y, parent individuals
n <- LENGTH(x)
c <- random number from 1 to n
return APPEND(SUBSTRING(x,1,c),SUBSTRING(y, c + 1, n ))

The sequence of steps to perform GA is summarized below:

 A successor state is generated by combining two parent states.

 Start with K randomly generated states population
 Each state or individual is represented as a string over a finite alphabet
(often a string of O's and 1's)

CSE – Panimalar Engineering College 51

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial
CS8691 – Artificial Intelligence

 Evaluation function (fitness function) is applied to find better states

with higher values.
 Produce the next generation of states by selection, crossover and
mutation

Local Search In Continuous Spaces

Local Search in continuous space is the one that deals with the real world
problems.

 One way to solve continuous problems is to discretize the

neighborhood of each state.
 Stochastic hill climbing and simulated annealing are applied directly in
the continuous space
 Steepest - ascent hill climbing is applied by updating the formula of
current state

x <- x +   f(x)

 - small constant
 f(x) - magnitude & direction of the steepest slope.
 Empirical gradient, line search, Newton-Raphson method can be
applied in this domain to find the successor state.
 Local search methods in continuous space may also lead to local
maxima, ridges and plateau. This situation is avoided by using random
restart method.

Online Search Agents and Unknown Environments

Online search agent operates by interleaving computation and action, that is first
it takes an action, then it observes the environment and computes the next
action, whereas ,the offline search computes complete solution (problem solving
agent) before executing the problem solution.

online search agents suits well for the following domains.

 Dynamic or Semi dynamic domain

 Stochastic domain

Online search is a necessary idea for an exploration problem, where the states
and actions are unknown to the agent. For example, consider a newborn baby
for exploration problem and the baby's gradual discovery of how the world
works is an online search process

CSE – Panimalar Engineering College 52

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial
CS8691 – Artificial Intelligence

Online search problems

An online search problem can be solved by an agent executing actions rather

than by a computational process. The agent knows the following terms to do the
search in the given environment

 ACTIONS(S) - which returns a list of actions allowed in state s ;

 c(s, a, s’) - The step-cost function known to the agent when it reaches
s’
 GOAL-TEST(S).

Searching With Partial Information

When the knowledge of the states or actions is incomplete about the

environment, then only partial information is known to the agent. This
incompleteness lead to three distinct problem types. They are:
(i) Sensorless problems (conformant problems) : If the agent has no
sensors at all, then it could be in one of several possible initial states, and each
action might
therefore lead to one of possible successor states.

(ii) Contigency problems: If the environment is partially observable or if

actions are uncertain, then the agent's percepts provide new information after
each action. A problem is called adversarial if the uncertainty is caused by the
actions of another agent. To handle the situation of unknown circumstances the
agent needs a contigency plan.

(iii) Exploration problem: It is an extreme case of contigency problems, where

the states and actions of the environment are unknown and the agent must act
to discover
them.

CONSTRAINT SATISFACTION PROBLEMS(CSP)

Constraint satisfaction problems (CSP) are mathematical problems where one

must find states or objects that satisfy a number of constraints or criteria. A
constraint is a restriction of the feasible solutions in an optimization problem.

Some examples for CSP's are:

The n-queens problem

A crossword puzzle
A map coloring problem
The Boolean satisfiability problem

CSE – Panimalar Engineering College 53

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial
CS8691 – Artificial Intelligence

A cryptarithmetic problem

All these examples and other real life problems like time table scheduling,
transport scheduling, floor planning etc. are instances of the same pattern,

A Constraint Satisfaction Problem(or CSP) is defined by a set of variables

{X1,X2,….Xn,} and a set of constraints {C1,C2,…,Cm}. Each variable Xi has a
nonempty domain D, of possible values. Each constraint Ci involves some
subset of variables and specifies the allowable combinations of values for that
subset.

A State of the problem is defined by an assignment of values to some or all of

the variables,{Xi = vi, Xj = vj,…}. An assignment that does not violate any
constraints is called a consistent or legal assignment.

A complete assignment is one in which every variable is mentioned, and a

solution to a CSP is a complete assignment that satisfies all the constraints.
Some CSPs also require a solution that maximizes an objective function.

Example for Constraint Satisfaction Problem :

The map coloring problem. The task of coloring each region red, green or blue in
such a way that no neighboring regions have the same color.

Map of Australia showing each of its states and territories

We are given the task of coloring each region either red, green, or blue in such a
way that the neighboring regions must not have the same color.

CSE – Panimalar Engineering College 54

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial
CS8691 – Artificial Intelligence

To formulate this as CSP, we define the variable to be the regions: WA, NT, Q,
NSW, V, SA, and T.

The domain of each variable is the set {red, green, blue}.

The constraints require neighboring regions to have distinct colors: for example,
the allowable combinations for WA and NT are the pairs

{(red,green),(red,blue),(green,red),(green,blue),(blue,red),(blue,green)}.

(The constraint can also be represented as the inequality WA  NT)

There are many possible solutions, such as

{ WA = red, NT = green, Q = red, NSW = green, V = red ,SA = blue,T = red}.

Constraint Graph : A CSP is usually represented as an undirected graph, called

constraint graph where the nodes are the variables and the edges are the binary
constraints.

The map-coloring problem represented as a constraint graph.

CSP can be viewed as a standard search problem as follows :

 Initial state : the empty assignment {},in which all variables are
unassigned.
 Successor function : a value can be assigned to any unassigned
variable, provided that it does not conflict with previously assigned
variables.
 Goal test : the current assignment is complete.
 Path cost : a constant cost(E.g.,1) for every step.

CSE – Panimalar Engineering College 55

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial
CS8691 – Artificial Intelligence

Every solution must be a complete assignment and therefore appears at depth n

if there are n variables. So Depth first search algorithms are popular for CSPs.

Varieties of CSPs

Discrete variables

Discrete variables can have

 Finite Domains
 Infinite domains

Finite domains

The simplest kind of CSP involves variables that are discrete and have finite
domains.

Map coloring problems are of this kind. The 8-queens problem can also be
viewed as finite-domain CSP, where the variables Q 1,Q2,…..Q8 are the positions
each queen in columns 1,….8 and each variable has the domain
{1,2,3,4,5,6,7,8}.

If the maximum domain size of any variable in a CSP is d, then the number of
possible complete assignments is O(d n) – that is, exponential in the number of
variables.

Finite domain CSPs include Boolean CSPs, whose variables can be either true
or false.

Infinite domains

Discrete variables can also have infinite domains – for example, the set of
integers or the set of strings. With infinite domains, it is no longer possible to
describe constraints by enumerating all allowed combination of values. For
example, if Jobl, which takes five days, must precede Jobs, then we would need
a constraint language of algebraic inequalities such as

Startjob1 + 5 <= Startjob 3.

Continuous domains

CSPs with continuous domains are very common in real world. For example, in
operation research field, the scheduling of experiments on the Hubble Telescope
requires very precise timing of observations; the start and finish of each

CSE – Panimalar Engineering College 56

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial
CS8691 – Artificial Intelligence

observation and maneuver are continuous-valued variables that must obey a

variety of astronomical, precedence and power constraints.

The best known category of continuous-domain CSPs is that of linear

programming problems, where the constraints must be linear inequalities
forming a convex region. Linear programming problems can be solved in time
polynomial in the number of variables.

Varieties of constraints :

Unary constraints – Which restricts a single variable.

Example : SA  green

Binary constraints - relates pairs of variables.

Example : SA  WA

Higher order constraints involve 3 or more variables.

Example : cryptarithmetic puzzles. Each letter stands for a distinct digit

The aim is to find a substitution of digits for letters such that the resulting sum is
arithmetically correct, with the added restriction that no leading zeros are
allowed.

Constraint graph for the cryptarithmetic Problem

Alldiff constraint can be broken down into binary constraints - F  T, F  U, and

so on.
The addition constraints on the four columns of the puzzle also involve several
variables and can be written as

CSE – Panimalar Engineering College 57

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial
CS8691 – Artificial Intelligence

O + O = R + 10 . X1
X1 + W + W = U + 10 . X2
X1 + T + T = O + 10 . X3
X3 = F

Where X1, X2, and X3 are auxiliary variables representing the digit (0 or 1)
carried over into the next column.

Real World CSP's : Real world problems involve read-valued variables,

 Assignment problems Example : who teaches what class.

 Timetabling Problems Example : Which class is offered when & where?
 Transportation Scheduling
 Factory Scheduling

Backtracking Search for CSPs

The term backtracking search is used for depth-first search that chooses
values for one variable at a time and backtracks when a variable has no legal
values left to assign.

Part of search tree generated by simple backtracking for the map

coloring problem

CSE – Panimalar Engineering College 58

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial
CS8691 – Artificial Intelligence

Improving backtracking efficiency is done with general purpose methods, which

can give huge gains in speed.

 Which variable should be assigned next and what order should be tried?
 What are the implications of the current variable assignments for the other
unassigned variables?
 Can we detect inevitable failure early?

Variable & value ordering: In the backtracking algorithm each unassigned

variable is chosen from minimum Remaining Values (MRV) heuristic, that is
choosing the variable with the fewest legal values. It also has been called the
"most constrained variable" or "fail-first" heuristic.

If the tie occurs among most constrained variables then most constraining
variable is chosen (i.e.) choose the variable with the most constraints on
remaining variable. Once a variable has been selected, choose the least
constraining value that is the one that rules out the fewest values in the
remaining variables.

Propagating information through constraints

So far our search algorithm considers the constraints on a variable only at the
time that the variable is chosen by SELECT-UNASSIGNED-VARIABLE. But by
looking at some of the constraints earlier in the search, or even before the
search has started, we can drastically reduce the search space.

Forward checking

One way to make better use of constraints during search is called forward
checking. Whenever a variable X is assigned, the forward checking process

CSE – Panimalar Engineering College 59

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial
CS8691 – Artificial Intelligence

looks at each unassigned variable Y that is connected to X by a constraint and

deletes from Y ’s domain any value that is inconsistent with the value chosen for
X.

The progress of a map-coloring search with forward checking.

In forward checking WA = red is assigned first; then forward checking deletes

red from the domains of the neighboring variables NT and SA. After Q = green,
green is deleted from the domains of NT, SA, and NSW. After V = blue, blue is
deleted from the domains of NSW and SA, leaving SA with no legal values. NT
and SA cannot be blue

Constraint propagation

Although forward checking detects many inconsistencies, it does not detect all of
them.

Constraint propagation is the general term for propagating the implications of

a constraint on one variable onto other variables.

Constraint propagation repeatedly enforces constraints locally to detect

inconsistencies. This propagation can be done with different types of consistency
techniques. They are:

Node consistency (one consistency)

Arc consistency (two consistency)
Path consistency (K-consistency)

Node consistency

 Simplest consistency technique

 The node representing a variable V in constraint graph is node consistent if
for every value X in the current domain of V, each unary constraint on V is
satisfied.
 The node inconsistency can be eliminated by simply removing those values
from the domain D of each variable V that do not satisfy unary constraint on
V.

CSE – Panimalar Engineering College 60

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial
CS8691 – Artificial Intelligence

Arc Consistency

The idea of arc consistency provides a fast method of constraint propagation

that is substantially stronger than forward checking. Here, 'arc’ refers to a
directed arc in the constraint graph, such as the arc from SA to NSW. Given the
current domains of SA and NSW, the arc is consistent if, for every value x of SA,
there is some value y of NSW that is consistent with x.

In the constraint graph, binary constraint corresponds to arc. Therefore this type
of consistency is called arc consistency.

Arc (vi, vj) is arc consistent if for every value X the current domain of vi there is
some value Y in the domain of vj such vi =X and vj=Y is permitted by the binary
constraint between vi and vj

Arc-consistency is directional ie if an arc (vi, vj) is consistent than it does not

automatically mean that (vj, vi) is also consistent.

An arc (vi, vj) can be made consistent by simply deleting those values from the
domain of Di for which there is no corresponding value in the domain of D j such
that the binary constraint between Vi and vj is satisfied - It is an earlier detection
of inconstency that is not detected by forward checking method.

The different versions of Arc consistency algorithms are exist such as AC-I,
AC2,AC-3, AC-4, AC-S; AC-6 & AC-7, but frequently used are AC-3 or AC-4.

AC - 3 Algorithm

In this algorithm, queue is used to cheek the inconsistent arcs.

When the queue is not empty do the following steps:

Remove the first arc from the queue and check for consistency.

CSE – Panimalar Engineering College 61

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial
CS8691 – Artificial Intelligence

If it is inconsistent remove the variable from the domain and add a new
arc to the queue
Repeat the same process until queue is empty

function AC-3( csp) returns the CSP, possibly with

reduced domains
inputs: csp, a binary CSP with variables {X1, X2, . . . ,
Xn}
local variables: queue, a queue of arcs, initially all
the arcs in csp
while queue is not empty do
(Xi, Xj) <- REMOVE-FIRST(queue)
if REMOVE-INCONSISTENT-VALUEXS(xi,xj) then
for each Xk in NEIGHBORS[Xj) do
add (Xk , Xi)to queue

function REMOVE-INCONSISTENT-VALUEXS(xi,xj) returns true

iff we remove a value
removed <-false
for each x in DOMAIN[xi] do
if no value y in DOMAIN[xj] allows (x, y) to satisfy the
constraint between Xi and Xj
then delete x from DOMAIN[Xi]removed <- true
return removed

k-Consistency (path Consistency)

A CSP is k-consistent if, for any set of k - 1 variables and for any consistent
assignment to those variables, a consistent value can always be assigned to any
kth variable

1-consistency means that each individual variable by itself is consistent;

this is also called node consistency.
2-consistency is the same as arc consistency.
3-consistency means that any pair of adjacent variables can always be
extended to a third neighboring variable; this is also called path
consistency.

CSE – Panimalar Engineering College 62

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial
CS8691 – Artificial Intelligence

Handling special constraints

Alldiff constraint - All the variables involved must have distinct values.

Example: Crypt arithmetic problem

The inconsistency arises in Alldiff constraint when m>n (i.e.) m variables are
involved in the constraint and n possible distinct values are there. It can be
avoided by selecting the variable in the constraint that has a singleton. domain
and delete the variable's value from the domain of remaining variables, until the
singleton variables are-exist. This simple algorithm will resolve the inconsistency
of the problem.

Resource constraint (Atmost Constraint) - Higher order constraint or

atmost constraint, in which consistency is achieved by deleting the maximum
value of any domain if it is not consistent with minimum values; of the other
domains.

Intelligent backtracking

Chronological backtracking : When a branch of the search fails, back up to

the preceding variable and try a different value for it (i.e.) the most recent
decision point is revisited. This will lead to inconsistency in real world problems
(map coloring problem) that can't be resolved. To overcome the disadvantage of
chronological backtracking, an intelligence backtracking method is proposed.

Conflict directed backtracking: When a branch of the search fails, backtrack

to one of the set of variables that caused the failure-conflict set. The conflict set
for variable X is the set of previously assigned variables that are connected to X
by constraints. A backtracking algorithm that was conflict sets defined in this way
is called conflict directed backtracking

Local Search for CSPs

Local search method is effective in solving CSP's, because complete state

formulation is defined.
Initial state - assigns a value to every variable.
Successor function - works by changing the value of each variable

Advantage : useful for online searching when the problem changes.

Ex : Scheduling problems

The MIN-CONFLICTS algorithm for solving CSPs by local search.

CSE – Panimalar Engineering College 63

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial
CS8691 – Artificial Intelligence

function MIN-CONFLICTS (CSP, max-steps) returns a

solution or failure
inputs: csp, a constraint satisfaction problem
max-steps, the number of steps allowed before giving up
current <- an initial complete assignment for csp
for i = 1 to max-steps do
if current is a solution for csp then return current
var <- a randomly chosen, conflicted variable from
VARIABLES[CSP]
value <- the value v for var that minimizes
CONFLICTS(var, v, current, csp)
set var = value in current
return failure

A two-step solution for an &-queens problem using min-conflicts. At each stage,

a queen is chosen for reassignment in its column. The number of conflicts (in
this case, the number of attacking queens) is shown in each square.

The structure of problems

The complexity of solving a CSP-is strongly related to the structure of its

constraint graph. If CSP can be divided into independent sub problems, then
each sub problem is solved independently then the solutions are combined.
When the n variables are divided as n/c subproblems, each will take dc work to
solve. Hence the total work is O(dc n/c). If n=lO, c=2 then 5 problems are
reduced and solved in less time.

Completely independent sub problems are rare, in most cases sub problems of a
CSP are connected

The way how to convert the constraint graph of a tree structure CSP into linear
ordering of the variables consistent with the tree is shown in Figure. Any two
variables are connected by atmost one path in the tree structure

CSE – Panimalar Engineering College 64

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial
CS8691 – Artificial Intelligence

If the constraint graph of a CSP forms a tree structure then it can be solved in
linear time number of variables). The algorithm has the following steps.
1. Choose any variable as the root of the tree and order the variables from the
root to the leaves in such a way that every node's parent in the tree preceeds it
in the ordering label the variables X l .... Xn in order, every variable except the
root has exactly one parent variable.

2. For j from n down 2, apply arc consistency to the arc (Xi , Xj), where Xi is the
parent of Xj removing values from Domain [Xi] as necessary.

3. For j from 1 to n, assign any value for Xj consistent with the value assigned
for Xi, where Xi is the parent of Xj Keypoints of this algorithm are as follows:

Step-(2), CSP is arc consistent so the assignment of values in step (3)

requires no backtracking.
Step-(2), the arc consistency is applied in reverse order to ensure the
consistency of arcs that are processed already.

General constraint graphs can be reduced to trees on two ways. They are:

(a) Removing nodes - Cutset conditioning

(b) Collapsing nodes together - Tree decomposition.

(a) Removing nodes - Cutset conditioning

Assign values to some variables so that the remaining variables form a

tree.

CSE – Panimalar Engineering College 65

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial
CS8691 – Artificial Intelligence

Delete the value assigned variable from the list and from the domains of
the other variables any values that are inconsistent with the value chosen
for the variable.

This works for binary CSP's and not suitable for higher order constraints.

The remaining problem (tree structure) is solved in linear order time

variables.

Example: In the constraint graph of map coloring problem, the region SA is

assigned with a value and it is removed to make the problem in the form of tree
structure, then it is solvable in linear time

The original constraint graph

The constraint graph after the removal of SA

If the value chosen for the variable to be deleted for tree structure is
wrong, then the following algorithm is executed.

(i) Choose a subset S from VARIABLES[CSP] such that the constraint

graph becomes a tree after removal of S-cycle cutset.

(ii) For each variable on S with assignment satisfies all constraints on S.

CSE – Panimalar Engineering College 66

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial
CS8691 – Artificial Intelligence

* Remove from the deomains of the remaining variables any values that
are inconsistent with the assignment for S.
* If the remaining CSP has a solution, return it with the assignment for S.

(b) Collapsing nodes together-Tree decomposition

Construction of tree decomposition the constraint graph is divided into a

set of subproblems, solved independently and the resulting solutions are
combined.

Works well, when the subproblem is small.

Requirements of this method are:

 Every variable in the base problem should appear in atleast one of

the subproblem.
 If the binary constraint is exist, then the same constraint must
appear in atleast one of the subproblem. .
 If the variable appears in two subproblems in the tree, it must
appear in every subproblem along the path connecting those
subproblems, that is the variable should be assigned with same
value and constraint in every subproblem.

A tree decompositon of the constraint graph

Solution

 If any subproblem has no solution then the entire problem has no

solution.
 If all the sub problems are solvable then a global solution is achieved.

Adversarial Search

CSE – Panimalar Engineering College 67

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial
CS8691 – Artificial Intelligence

Competitive environments, in which the agent’s goals are in conflict, give rise to
adversarial search problems-often known as games.

In our terminology, games means deterministic, fully observable environments in

which there are two agents whose actions must alternate and in which the utility
values at the end of the game are always equal and opposite. For example, if
one player wins a game of chess (+1), the other player necessarily loses (-1).

There are two types of games

1. Perfect Information ( Example : chess, checkers)

2. Imperfect Information ( Example : Bridge, Backgammon)

In game playing to select the next state, search technique is required. Game
playing itself is considered as a type of search problems. But, how to reduce the
search time to make on a move from one state to another state.

The pruning technique allows us to ignore positions of the search tree that
make no difference to the final choice.

Heuristic evaluation function allows us to find the utility (win, loss, and
draw) of a state without doing a complete search.

Optimal Decisions in Games

A Game with two players - Max and Min.

 Max, makes a move first by choosing a high value and take turns moving
until the game is over
 Min, makes a move as a opponent and tries to minimize the max player
score, until the game is over.

At the end of the game (goal state or time), points are awarded to the winner.

The components of game playing are :

Initial state - Which includes the board position and an indication of whose
move and identifies the player to the move.
Successor function - which returns a list of (move, state) pairs, each
indicating a legal move and the resulting state.

Terminal test - Which determines the end state of the game. States where the
game has ended are called terminal states.

CSE – Panimalar Engineering College 68

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial
CS8691 – Artificial Intelligence

Utility function (also called an objective function or payoff function), - which

gives a numeric value for the terminal states. In chess, the outcome is a win,
loss, or draw, with values +1, -1, or 0. Some games have a wider , variety of
possible outcomes; the payoffs in backgammon range from +192 to -192.

The initial state and the legal moves for each side define the game tree for the
game

Example : Tic – Tac – Toe (Noughts and Crosses)

From the initial state, MAX has nine possible moves. Play alternates between
MAX'S placing an X and MIN'S placing an O until we reach leaf nodes
corresponding to ter.mina1 states such that one player has three in a row or all
the squares are filled.

Initial State : Initial Board Position

Successor Function : Max placing X’s in the empty square

Min placing O’s in the empty square

Goal State : We have three different types of goal state, any one to be
reached.
i) If the O’s are placed in one column, one row (or) in the diagonal
continuously then, it is a goal state of min player. (Won by Min Player)
ii) If the X’s are placed in one column, one row (or) in the diagonal
continuously then, it is a goal state of min player. (Won by Max Player)
iii) If the all the nine squares are filled by either X or O and there is no
win condition by Max and Min player then it is a state of Draw between
two players.
Some terminal states

Won by Min Players

X O X
O
O

CSE – Panimalar Engineering College 69

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial
CS8691 – Artificial Intelligence

Won by Max Players

X O X
X
X 0 O

Draw between two Players

X O X
O O X
X O O

Utility function

Win = 1 Draw =0 Loss = -1

A (partial) search tree for the game of tic-tac-toe

Optimal strategies

In a normal search problem, the optimal solution would be a sequence of moves

leading to a goal state-a terminal state that is a win. In a game, an optimal
strategy leads to outcomes at least as good as any other strategy when one is
playing an infallible opponent

Given a game tree, the optimal strategy can be determined by examining the
minimax value of each node, which we write as MINIMAX- VALUE(n).

CSE – Panimalar Engineering College 70

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial
CS8691 – Artificial Intelligence

MINIMAX- VALUE(n) =

UTILITY(n) if n is a terminal state

Maxs  successors(n) MAX-VALUE(s) if n is a MAX node
Mins  Successors(n) MAX-VALUE(s) if n is a MIN node

Even a simple game like tic-tac-toe is too complex for us to draw the entire game
tree.

The possible moves for MAX at the root node are labeled al, a2, and a3. The
possible replies to a1 for MIN are b1, b2, b3, and so on.
A Two Ply Game Tree

 - Moves by Max Player

 - Moves by Min Player

 The terminal nodes show the utility values for MAX;

 The other nodes are labeled with their minimax values.
 MAX'S best move at the root is al, because it leads to the successor
with the highest minimax value
 MIN'S best reply is bl, because it leads to the successor with the lowest
minimax value.
 The root node is a MAX node; its successors have minimax values 3, 2,
and 2; so it has a minimax value of 3.
 The first MIN node, labeled B, has three successors with values 3, 12,
and 8, so its minimax value is 33

The minimax algorithm

The minimax algorithm computes the minimax decision from the current
state. It uses a simple recursive computation of the minimax values of each
successor state, directly implementing the defining equations. The recursion
proceeds all the way down to the leaves of the tree, and then the minimax
values are backed up through the tree as the recursion unwinds.

CSE – Panimalar Engineering College 71

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial
CS8691 – Artificial Intelligence

For Example

The algorithm first recurses down to the three bottom left nodes, and uses the
UTILITY function on them to discover that their values are 3, 12, and 8
respectively. Then it takes the minimum of these values, 3, and returns it as the
backed-up value of node B. A similar process gives the backed up values of 2 for
C and 2 for D. Finally, we take the maximum of 3,2, and 2 to get the backed-up
value of 3 for the root node.

An algorithm for minimax decision

function MINIMAX-DECISION (state) returns an action

inputs: state, current state in game
v <- MAX-VALUE(state)
return the action in SUCCESSORS(state) with value v

function MAX-VALUE(state) returns a utility value

if TERMINAL-TEST(state) then return UTILITY(state)
v<- - 
for a, s in SUCCESSORS(state) do
v <- MAX(v, MIN-VALUE(s))
return v

function MIN-VALUE(state) returns a utility value

if TERMINAL-TEST(state) then return UTILITY(state)
V <- 
for a, s in SUCCESSORS(state)do'
v <- MIN(v, MAX-VALUE(s))
return v

 Generate the whole game tree, all the way down to the terminal state.
 Apply the utility function to each terminal state to get its value.

CSE – Panimalar Engineering College 72

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial
CS8691 – Artificial Intelligence

 Use utility functions of the terminal state one level higher than the current
value to determine Max or Min value.
 Minimax decision maximizes the utility under the assumption that the
opponent will play perfectly to minimize the max player score.

Complexity : If the maximum depth of the tree is m, and there are b legal
moves at each point then the time complexity of the minimax algorithm is O(bm).
This algorithm is a depth first search, therefore the space requirements are linear
in m and b. For real games the calculation of time complexity is impossible,
however this algorithm will be a good basis for game playing.

Completeness : It the tree is finite, then it is complete.

Optimality : It is optimal when played against an optimal opponent

ALPHA - BETA PRUNING

Pruning - The process of eliminating a branch of the search tree from

consideration without examining is called pruning.

The two parameters of pruning technique are:

Alpha (  ) : Best choice for the value of MAX along the path (or) lower bound
on the value that on maximizing node may be ultimately assigned.

Beta (  ) : Best choice for the value of MIN along the path (or) upper bound
on the value that a minimizing node may be ultimately assigned.

Alpha - Beta pruning : The  and  values are applied to a minimax tree, it
returns the same move as minimax, but prunes away branches that cannot
possibly influence the final decision is called Alpha - Beta pruning (or) Cutoff
Consider again the two-ply game tree from

Let the two unevaluated successors of node C have values x and y and let z be
the minimum of x and y. The value of the root node is given by

CSE – Panimalar Engineering College 73

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial
CS8691 – Artificial Intelligence

MINIMAX-VALUE(ROOT)=max((min(3,12,8),min(2,x,y),min(l4,5,2 ))
= max(3, min(2, x, y), 2)
= max(3, z, 2) where z <=2
= 3.

In other words, the value of the root and hence the minimax decision are
independent of the values of the pruned leaves x and y.

Stages in the calculation of the optimal decision for the game tree

(a) The first leaf below B has the value 3. Hence, B, which is a MIN node, has a
value of at most 3

(b) The second leaf below B has a value of 12; MIN would avoid this move, so
the value of B is still at most 3

c) The third leaf below B has a value of 8; we have seen all B's successors, so
the value of B is exactly 3. Now, we can infer that the value of the root is at least
3, because MAX has a choice worth 3 at the root.

(d) The first leaf below C has the value 2. Hence, C, which is a MIN node, has a
value of at most 2. But we know that B is worth 3, so MAX would never choose
C. Therefore, there is no point in looking at the other successors of C. This is an
example of alpha-beta pruning.

CSE – Panimalar Engineering College 74

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial
CS8691 – Artificial Intelligence

(e) The first leaf below D has the value 14, so D is worth atmost 14. This is still
higher than MAX'S best alternative (i.e., 3), so we need to keep exploring D's
successors. Notice also that we now have bounds on all of the successors of the
root, so the root's value is also at most 14.

(f) The second successor of D is worth 5, so again we need to keep exploring.

The third successor is worth 2, so now D is worth exactly 2. MAX'S decision at
the root is to move to B, giving a value of 3.

The alpha-beta search algorithm

function ALPHA-BETA-SEARCH (returns an ) action

inputs: state, current state in game
v<- M A X -V A L U E (State,-  , +  )
return the action in SUCCESSORS(state) with value v

CSE – Panimalar Engineering College 75

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial
CS8691 – Artificial Intelligence

function MAX - VALUE ( state,  ,  ) returns  utility

value
inputs: state, current state in game
 , the value of the best alternative for MAX along the
path to state
 , the value of the best alternative for MIN along the
path to state
if TERMINAL-TEST(state) then return) UTILITY( s t a t e
)
v<- - 
for  , S in S U C C E S S O R S (state) do
v <- MAX(v,MIN-VALUE(S,  ,  ))
if v >=  then return v
 <- MAx(  , v)
return v

function MIN - VALUE ( state,  ,  ) returns  utility

value
inputs: state, current state in game
 , the value of the best alternative for MAX along the
path to state
 , the value of the best alternative for MIN along the
path to state
if TERMINAL-TEST(state) then return) UTILITY( s t a t e
)
v<- + 
for  , S in S U C C E S S O R S (state) do
v <- MIN(v,MAX-VALUE(S,  ,  ))
if v <=  then return v
 <- MIN(  , v)
return v

Effectiveness of Alpha – Beta Pruning

Alpha - Beta pruning algorithm needs to examine only O(bd/2) nodes to pick the
best move, instead of O(bd) with minimax algorithm, that is effective branching
factor is
 instead of b.

CSE – Panimalar Engineering College 76

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial
CS8691 – Artificial Intelligence

Imperfect Real Time Decisions

The minimax algorithm generates the entire game search space, whereas the
alpha-beta algorithm allows us to prune large parts of it. However, alpha-beta
still has to search all the way to terminal states for at least a portion of the
search space. This depth is usually not practical, because moves must be made
in a reasonable amount of time-typically a few minutes at most.

Shannon’s proposed instead that programs should cut off the search earlier and
apply a heuristic evaluation function to states in the search, effectively turning
non terminal nodes into terminal leaves.

 The utility function is replaced by an Evaluation function

 The terminal test is replaced by a Cut-off test

1. Evaluation function

Example: Chess Problem

In chess problem each material (Queen, Pawn, etc) has its own value that is
called as material value. From this depends on the move the evaluation function
is calculated and it is applied to the search tree.

This suggests that the evaluation function should be specified by the rules of
probability.

For example If player A has a 100% chance of winning then its evaluation
function is 1.0 and if player A has a 50% chance of winning, 25% of losing and
25% of being a draw then the probability is calculated as; 1x0.50 -lxO.25 +
OxO.25 = 0.25.

As per this example is concerned player A is rated higher than player B. The
material value, of each piece can be calculated independently with-out
considering other pieces in the board is also called as one kind of evaluation
function and it is named as weighted linear function. It can be expressed as

Eval(s) = w1f1(s) + w2f2(s) + w3f3(s)..... + wnfn (s)

w - Weights of the pieces (1 for Pawn, 3 for Bishop etc)
f - A numeric value which represents the numbers of each kind of piece on the
board.

CSE – Panimalar Engineering College 77

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial
CS8691 – Artificial Intelligence

2. Cut - off test

To perform a cut-off test, an evaluation function, should be applied to positions

that are quiescent, that is a position that will not swing in bad value for long time
in the search tree is known as waiting for quiescence.

Quiescence search - A search which is restricted to consider only certain types

of moves, such as capture moves, that will quickly resolve the uncertainties in
the position.

Horizon problem - When the program is facing a move by the opponent that
causes serious damage and is ultimately unavoidable

Example:
1. Beginning of the search - one ply

2. This diagram shows the situation of horizon problem that is when one level is
generated from B, it causes bad value for B

3. When one more successor level is generated from E and F and situation
comes down and the value of B is retained as a good move. The time B is waited
for this situation is called waiting for quiescence.

CSE – Panimalar Engineering College 78

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial
CS8691 – Artificial Intelligence

Games That Include An Element Of Chance

Backgammon Game

Backgammon is a typical game that combines luck and skill. Dice are rolled at
the beginning of a player's turn to determine the legal moves.

Goal State

The goal of the game is to move all one's pieces off the board. White moves
clockwise toward 25, and black moves counterclockwise toward 0

Successor Function or Operator

Move to any position except where two or more of the opponent pieces.
If it moves to a position with one opponent piece it is captured and again it has
to start from Beginning

Task : In the position shown, White has rolled 6-5. So Find out the legal moves
for the set of the dice thrown as 6 and 5.

Solution :

There are Four legal moves. They are

 (5-11,5-10)
 (5-11, 19-24)

CSE – Panimalar Engineering College 79

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial
CS8691 – Artificial Intelligence

 (10-16,5-10)
 (5-11,11-16)

A game tree in backgammon must include chance nodes in addition to MAX

and MIN nodes. Chance nodes are shown as circles. The branches leading from
each chance node denote the possible dice rolls, and each is labeled with the roll
and the chance that it will occur. There are 36 ways to roll two dice, each equally
likely; but because a 6-5 is the same as a 5-6, there are only 21 distinct rolls.
The six doubles (1-1 through 6-6) have a 1/36 chance of coming up, the other
15 distinct rolls a 1/18 chance each.

The resulting positions do not have definite minimax values. Instead, we have to
only calculate the expected value, where the expectation is taken over all the
possible dice rolls that could occur.

Terminal nodes and MAX and MIN nodes (for which the dice roll is known) work
exactly the same way as before; chance nodes are evaluated by taking the
weighted average of the values resulting from all possible dice rolls, that is,

EXPECTIMINIMAX(n)=

UTILITY( n ) if n is a terminal state

Max s  successors(n) EXPECTIMINIMAX(S) if n is a MAX node
Min s  successors(n) EXPECTIMINIMAX(S) if n is a MIN node
 s  successors(n) P(s).EXPECTIMINIMAX(S) if n is a chance node

CSE – Panimalar Engineering College 80

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial
CS8691 – Artificial Intelligence

where the successor function for a chance node n simply augments the state of
n with each possible dice roll to produce each successor s and P(s) is the
probability that that dice roll occurs.

Card games

Card games are interesting for many reasons besides their connection with
gambling.

Imagine two players, MAX and MIN, playing some practice hands of four-card
two handed bridge with all the cards showing.

The hands are as follows, with MAX to play first:

MAX : 6, 6, 9, 8

MIN : 2, 4, 10 , 5

Suppose that MAX leads wiht 9. MIN must now follow suit, playing either with
10 or 5 . MIN plays with 10 and wins the trick.

MIN goes next turn leads the with 2. MAX has no spades (and so cannot win
the trick) and therefore must throw away some card. The obvious choice is the
6 because the other two remaining cards are winners.

Now, whichever card MIN leads for the next trick, MAX will win both remaining
tricks and the game will be tied at two tricks each.

CSE – Panimalar Engineering College 81

Download Useful Materials @ www.rejinpaul.com

Artificial Intelligence www.studymaterial

1
Lecture #12 – Knowledge Representation
Outline
Artificial Intelligence
ICS461 ƒ Ontological engineering
Fall 2010 ƒ Categories and objects
ƒ Actions, situations and events
ƒ Mental events and mental objects
Nancy E. Reed ƒ The internet shopping world
ƒ Reasoning systems for categories
[email protected]
ƒ Reasoning with default information
ƒ Truth maintenance systems 2

Ontological Engineering The Upper Ontology of the World

ƒ How to create more general and flexible
representations.
• Concepts like actions, time, physical objects and
beliefs
• Operates on a bigger scale than knowledge
engineering
ƒ Define general framework of concepts
• Upper ontology
ƒ Limitations of logic representation
• Red, green and yellow tomatoes: exceptions and
uncertainty
3 4

Differences with Special-Purpose

Categories and Objects
Ontologies
ƒ A general-purpose ontology should be ƒ KR requires the organization of objects into
applicable in more or less any special-purpose categories
domain. • Interaction at the level of the object
• Add domain-specific axioms • Reasoning at the level of categories
ƒ In any sufficiently demanding domain ƒ Categories play a role in predictions about
diff
differentt areas off knowledge
k l d needd to
t be
b objects
unified. • Based on perceived properties
• Reasoning and problem solving could involve several ƒ Categories can be represented in two ways by
areas simultaneously first-order logid (FOL)
ƒ What do we need to express? • Predicates: apple(x)
• Categories, Measures, Composite objects, Time, Space, • Reification of categories into objects: apples
Change, Events, Processes, Physical Objects, ƒ Category = set of its members
Substances, Mental Objects, Beliefs 5 6

Download Useful Materials @ www.rejinpaul.com 1

Artificial Intelligence www.studymaterial

Category Organization FOL and Categories

ƒ Relation = inheritance: ƒ An object is a member of a category
• MemberOf(BB12,Basketballs)
• All instance of food are edible, fruit is a subclass of ƒ A category is a subclass of another category
food and apples is a subclass of fruit then an apple is • SubsetOf(Basketballs,Balls)
edible. ƒ All members of a category have some properties
ƒ Defines a • ∀ x (MemberOf(x,Basketballs)
(MemberOf(x Basketballs) ⇒ Round(x))
ƒ All members of a category can be recognized by
taxonomy some properties
• ∀ x (Orange(x) ∧ Round(x) ∧
Diameter(x)=9.5in ∧ MemberOf(x,Balls) ⇒
MemberOf(x,BasketBalls))
ƒ A category as a whole has some properties
• MemberOf(Dogs,DomesticatedSpecies)
7 8

Relations Between Categories Relations Between Categories

ƒ Two or more categories are disjoint if they ƒ A partition is a disjoint exhaustive
have no members in common: decomposition:
• Disjoint(s)⇔(∀ c1,c2 c1 ∈ s ∧ c2 ∈ • Partition(s,c) ⇔ Disjoint(s) ∧
s ∧ c1 ¹ c2 ⇒ Intersection(c1,c2) E.D.(s,c)
={})
• Example:
• Example; Disjoint({animals,
Disjoint({animals P titi ({M l
Partition({Males,Females},Persons)
F l } P )
vegetables})
ƒ Is ({Americans, Canadian,
ƒ A set of categories s constitutes an
exhaustive decomposition of a category c if Mexicans}, NorthAmericans) a
all members of the set c are covered by partition?
categories in s: ƒ Categories can be defined by providing
• E.D.(s,c) ⇔ (∀ i, i ∈ c ⇒ ∃ c2 c2 necessary and sufficient conditions for
∈ s ∧ i ∈ c2) 9
membership
10

E l E h ti D iti

Natural Kinds Physical Composition

ƒ Many categories have no clear-cut definitions ƒ One object may be part of another:
(chair, bush, book) • PartOf(Bucharest,Romania)
• PartOf(Romania,EasternEurope)
ƒ Tomatoes: sometimes green, red, yellow, black. • PartOf(EasternEurope,Europe)
Mostly round. ƒ The PartOf predicate is transitive (and irreflexive),
ƒ One solution: category Typical(Tomatoes). so we can infer that PartOf(Bucharest,Europe)
• ∀ x,
x x ∈ Typical(Tomatoes) ⇒ Red(x) ∧ ƒ More generally:
Spherical(x). • ∀ x PartOf(x,x)
ƒ We can write down useful facts about categories • ∀ x,y,z PartOf(x,y) ∧ PartOf(y,z) ⇒ PartOf(x,z)
without providing exact definitions. ƒ Often characterized by structural relations among
parts.
ƒ What about “bachelor”? Quine challenged the • E.g. Biped(a) ⇒
utility of the notion of strict definition. We might
question a statement such as “the Pope is a
bachelor” 11 12

Download Useful Materials @ www.rejinpaul.com 2

Artificial Intelligence www.studymaterial

Measurements Actions, Events and Situations

ƒ Objects have height, mass, cost, .... ƒ Reasoning about the
Values that we assign to these are measures outcome of actions is
ƒ Combine Unit functions with a number: central to KB-agent
Length(L1) = Inches(1.5) = Centimeters(3.81)
ƒ How can we keep track of
ƒ Conversion between units:
∀ i Centimeters(2.54
i ( x i) = Inches(i).
h (i)
location in FOL?
• Remember the multiple copies
ƒ Some measures have no scale: Beauty, in PL.
Difficulty, etc.
• Most important aspect of measures: is that they are ƒ Representing time by
orderable situations (states resulting
• Don't care about the actual numbers from the execution of
E.g. An apple can have deliciousness .9 or .1.
actions).
13 14
• Situation calculus

Situation Calculus ƒ Results of action

ƒ Actions are logical terms sequences are determined
ƒ Situations are logical terms by the individual actions
consiting of
ƒ Projection task: an SC
• The initial situation I
• All situations resulting from the agent should be able to
action on I ((=Result(a
Result(a,s))
s)) deduce the outcome of a
ƒ Fluents are functions and sequence of actions
predicates that vary from
one situation to the next. ƒ Planning task: find a
• E.g. ¬Holding(G1, S0) sequence that achieves a
ƒ Eternal predicates are also desirable effect
allowed
• E.g. Gold(G1) 15 16

Describing Change
ƒ Simplest Situation calculus requires two
axioms to describe change:
• Possibility axiom: when is it possible to do the action
- At(Agent,x,s) ∧ Adjacent(x,y) ⇒
Poss(Go(x,y),s)
• Effect axiom: describe changes due to action
- P ) ) ⇒
Poss(Go(x,y),s)
(G (
At(Agent,y,Result(Go(x,y),s))
ƒ What stays the same?
• Frame problem: how to represent all things that stay
the same?
• Frame axiom: describe non-changes due to actions
- At(o,x,s) ∧ (o ≠ Agent) ∧ ¬Holding(o,s)
⇒ At(o,x,Result(Go(y,z),s))
17 18

Download Useful Materials @ www.rejinpaul.com 3

Artificial Intelligence www.studymaterial

The Frame Problem Other Problems

ƒ For F fluents and A actions we need ƒ How to deal with secondary (implicit)
A*F frame axioms to describe other objects effects?
which are stationary unless they are held • If the agent is carrying the gold and the agent
• Must write down the effect of each action moves then the gold moves too.
• Ramification problem
ƒ Solution - describe how each fluent changes
over time ƒ How to decide efficiently whether fluents
• Successor-state axiom:
hold in the future?
- Pos(a,s) ⇒ (At(Agent,y,Result(a,s)) ⇔ • Inferential frame problem
(a = Go(x,y)) ∨ (At(Agent,y,s) ∧ a ≠
Go(y,z))
ƒ Extensions
• Note that next state is completely specified by current
• Event calculus (when actions have a duration)
state. 19 • Process categories 20

E h ti ff t i ti d l

Mental Events and Objects The Internet Shopping World

ƒ So far, KB agents can have beliefs and deduce new ƒ A Knowledge Engineering example
beliefs
ƒ What about knowledge about beliefs? ƒ An agent that helps a buyer to find product
ƒ What about knowledge about the inference proces? offers on the internet.
• Requires a model of the mental objects in someone’s head and the • IN = product description (precise or ¬precise)
processes that manipulate those objects
ƒ Relationships between agents and mental objects: • OUT = list of webpages that offer the product for sale
believes, knows, wants, …
• Believes(Lois,Flies(Superman)) with Flies(Superman) being a
ƒ Environment = WWW
function … a candidate for a mental object (reification)
ƒ Percepts = web pages (character strings)
ƒ Agent can now reason about the beliefs of agents
• Extracting useful information required

21 22

24

Find Relevant Product Offers Find Relevant Product Offers, cont

ƒ RelevantOffer(page,url,query) ⇔ ƒ Require rich category vocabulary
Relevant(page, url, query) ∧
• synonymy
Offer(page)
• ambiguity
ƒ Write axioms to define Offer(x)
ƒ How to retrieve pages: GetPage(url) ?
ƒ Find relevant pages: Relevant(x,y,z)
Relevant(x y z) ? • Procedural attachment
• Start from an initial set of stores.
ƒ Compare offers (information extraction)
• What is a relevant category?
• What are relevant connected pages?

23

Download Useful Materials @ www.rejinpaul.com 4

Artificial Intelligence www.studymaterial

Reasoning systems for Categories Semantic Networks

How to organize and reason with ƒ Logic vs. semantic networks
categories? ƒ Many variations
ƒ Semantic networks • All represent individual objects, categories of
• Visualize knowledge-base objects and relationships among objects.
• Efficient algorithms for category membership
inference ƒ Allows for inheritance reasoning
ƒ Description logics • Female persons inherit all properties from person.
• Formal language for constructing and combining • Cfr. OO programming.
category definitions ƒ Inference of inverse links
• Efficient algorithms to decide subset and superset • SisterOf vs. HasSister
relationships between categories.
25 26

Semantic Network Example Semantic networks

ƒ Drawbacks
• Links can only assert binary relations
• Can be resolved by reification of the proposition
as an event
ƒ Representation
R t ti off default
d f lt values
l
• Enforced by the inheritance mechanism

27 28

Description Logics Reasoning with Default Information

ƒ Are designed to describe defintions and ƒ “The following courses are offered: ICS101,
properties about categories ICS110, ICS111, EE160”
• A formalization of semantic networks ƒ Four (db)
ƒ Principal inference task is • Assume that this information is complete (not asserted
ground atomic sentences are false)
• Subsumption: checking if one category is the
• = CLOSED WORLD ASSUMPTION
subset of another by comparing their definitions
• Assume that distinct names refer to distinct objects
• Classification: checking whether an object belongs
• = UNIQUE NAMES ASSUMPTION
to a category.
• Consistency: whether the category membership ƒ Between one and infinity (logic)
criteria are logically satisfiable • Does not make these assumptions
29 • Requires completion. 30

Download Useful Materials @ www.rejinpaul.com 5

Artificial Intelligence www.studymaterial

Truth Maintenance Systems Summary

ƒ Many inferences have default status ƒ Ontological engineering
rather than being absolutely certain ƒ Categories and objects
• Inferred facts can be wrong and need to be
retracted = belief revision.
ƒ Actions, situations and events
• Assume KB contains sentence P and we want ƒ Mental events and mental objects
to execute TELL(KB, ¬P)
ƒ The internet shopping world
- To avoid contradiction: RETRACT(KB,P)
- But what about sentences inferred from P? ƒ Reasoning systems for categories
ƒ Truth maintenance systems are designed ƒ Reasoning with default information
to handle these complications. ƒ Truth maintenance systems
31

Download Useful Materials @ www.rejinpaul.com 6

 www.studymaterial

First-Order Logic

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial

Difference between
Propositional Logic and First-Order Logic

• Ontological commitment
(i.e.) what it assumes about the nature
of reality
– Propositional logic  facts that either hold or
do not hold.
– First-order logic  objects with relations
among them that hold or do not hold.

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial
Pros and cons of propositional
logic
Propositional logic is declarative
Propositional logic allows partial/disjunctive/negated information
– (unlike most data structures and databases)
Propositional logic is compositional:
– meaning of B1,1  P1,2 is derived from meaning of B1,1 and of P1,2
Propositional logic is context-independent
– (unlike natural language, where meaning depends on context)
Propositional logic has very limited expressive power
– (unlike natural language)
– E.g., cannot say "pits cause breezes in adjacent squares“
• except by writing one sentence for each square

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial

Special purpose logics

• Make further ontological commitment
• Special purpose logic
– Temporal logic (Facts that hold at a particular
time)
– Higher-order logic (functions and relations by
first-order logic with objects)
Epistemological commitments possible states (
true, false, unknown)

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial

Different Logics
LANGUAGES Ontological Commitment Epistemological
Commitment

PROPOSITIONAL Facts True / False / Unknown

LOGIC
FIRST-ORDER LOGIC Facts, Objects, Relations True / False / Unknown

TEMPORAL LOGIC Facts, Objects, Relations, True / False /Unknown

Times

PROBABILITY Facts Degree of belief (0,1)

THEORY
FUZZY LOGIC Facts with degree of truth Known interval value

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial
First-order logic
• Whereas propositional logic assumes the world
contains facts,
• first-order logic (like natural language)
assumes the world contains

– Objects: people, houses, numbers, colors, baseball

games, wars, …
– Relations: red, round, prime, brother of, bigger
than, part of, comes between, …
– Functions: father of, best friend, one more than,
plus, …

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial

Syntax of FOL: Basic elements

• Constants KingJohn, 2,...
• Predicates Brother, >,...
• Functions Sqrt, LeftLegOf,...
• Variables x, y, a, b,...
• Connectives , , , , 
• Equality =
• Quantifiers , 

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial

Models for FOL: Example

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial

Atomic sentences
Atomic sentence = predicate (term1,...,termn)
or term1 = term2

Term = function (term1,...,termn)

or constant or variable

• E.g., Brother(John,Richard)

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial

Complex sentences
• Complex sentences are made from atomic sentences using
connectives
S, S1  S2, S1  S2, S1  S2, S1  S2,

E.g. Sibling(John,Richard)  Sibling(Richard,John)

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial

Symbols and Interpretations

• Sentences are true with respect to a model and an interpretation

• Model contains objects (domain elements) and relations among

them

• Interpretation specifies referents for

constant symbols → objects

predicate symbols → relations

function symbols → functional relations

• An atomic sentence predicate(term1,...,termn) is true

iff the objects referred to by term1,...,termn
are in the relation referred to by predicate
Download Useful Materials @ www.rejinpaul.com
 www.studymaterial

Universal quantification
• <variables> <sentence>
Everyone in cse is smart:
x (x,cse)  Smart(x)

• x P is true in a model m iff P is true with x being each

possible object in the model

• Roughly speaking, equivalent to the conjunction of

instantiations of P
in(John,cse)  Smart(KingJohn)
 in(Richard,cse)  Smart(Richard)
 in(Sam,cse)  Smart(Sam)

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial

A common mistake to avoid

• Typically,  is the main connective with 

• Common mistake: using  as the main connective

with :
x in(x,cse)  Smart(x)
means “Everyone is in cse and everyone is smart”

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial

Existential quantification
• <variables> <sentence>
• Someone in cse is smart:
• x (x,cse)  Smart(x)

• x P is true in a model m iff P is true with x being some

possible object in the model
•

• Roughly speaking, equivalent to the disjunction of

instantiations of P
•
in(KingJohn,cse)  Smart(John)
 in(Richard,cse)  Smart(Richard)
 in(Sam,cse)  Smart(Sam)
Download Useful Materials @ www.rejinpaul.com
 www.studymaterial

Another common mistake to avoid

• Typically,  is the main connective with 

• Common mistake: using  as the main connective

with :
x in(x,cse)  Smart(x)

is true if there is anyone who is not in cse!

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial

Properties of quantifiers

• x y is the same as y x

• x y is the same as y x

• x y is not the same as y x

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial

Nested Quantifiers
• x y Loves(x,y)
– “There is a person who loves everyone in the
world”
–
• y x Loves(y,x)
– “Everyone in the world is loved by at least one
person”

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial

Quantifier duality

Quantifier duality: each can be expressed using

the other

• x Likes(x,IceCream) = x Likes(x,IceCream)

• x Likes(x,icecream) = x Likes(x,icecream)

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial
Unification
• We can get the inference immediately if we can find a substitution θ
such that King(x) and Greedy(x) match King(John) and Greedy(y)

θ = {x/John,y/John} works

• Unify(p,q) = θ where SUBST(θ,p)= SUBST(θ,q)

p q θ
Knows(John,x) Knows(John,Jane) {x/Jane}
Knows(John,x) Knows(y,Bill) {x/Bill,y/John}
Knows(John,x) Knows(y,Mother(y)) {y/John,x/Mother(John)}
Knows(John,x) Knows(x,Elizabeth) fail

• Standardizing apart eliminates overlap of variables, e.g.,

Knows(z,Elizabeth)

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial

Unification
• To unify Knows(John,x) and Knows(y,z),

θ = {y/John, x/z } or θ = {y/John, x/John, z/John}

• The first unifier is more general than the second.

• There is a single most general unifier (MGU) that is

unique up to renaming of variables.
• MGU = { y/John, x/z }

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial

The unification algorithm

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial

The unification algorithm

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial

Generalized Modus Ponens

p1', p2', … , pn', ( p1  p2  …  pn q) where pi'θ = pi θ for all i

qθ
p1' is King(John) p1 is King(x)
p2' is Greedy(y) p2 is Greedy(x)
θ is {x/John,y/John} q is Evil(x)
q θ is Evil(John)

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial

Forward Chaining
• What is Forward Chaining?
-Start with the atomic sentences in the KB
and apply modus ponens in the forward
direction
-Known facts to goal
• Modus ponens, α β , α
β
From the given sentence α, β can be inferred.

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial

• Definite Clauses
Ex:- situation Response
Definite clauses are useful for systems that
make inferences in responses to newly arrived
information.

• To build a KB using definite clauses

• Reasoning with forward chaining is much
more efficient.

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial

• To apply forward chaining to first-order

definite clauses
• First-order definite clauses are disjunctions or
conjunctions of literals.

ex:- for first-order definite clauses

King (x)  Greedy (x)  Evil(x)
King(john)
Greedy(y)

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial

• Datalog KB: set of first-order definite clauses

with no function symbols or quantifiers.
– The absence of function symbols makes inference
much easier.

• What will the forward chaining algorithm do?

-Starts from the known facts,
-it triggers all the rules whose premises are
satisfied,
- adding their conclusions to the known facts.

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial

• The process repeats until the query is answered

or no new facts are added.
• Renaming – one sentence is a renaming of
another if they are identical except the name of
the variables.
Ex:- likes(x,icecream)
likes(y,icecream)

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial

Forward chaining algorithm

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial

Example knowledge base

• The law says that it is a crime for an American
to sell weapons to hostile nations. The country
Nono, an enemy of America, has some
missiles, and all of its missiles were sold to it
by SAM, who is American.

• Prove that SAM is a criminal

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial

Example knowledge base contd.

... it is a crime for an American to sell weapons to hostile nations:
American(x)  Weapon(y)  Sells(x,y,z)  Hostile(z)  Criminal(x)

Nono … has some missiles, i.e., x Owns(Nono,x)  Missile(x):

Owns(Nono,M1) and Missile(M1)

… all of its missiles were sold to it by SAM

Missile(x)  Owns(Nono,x)  Sells(sam,x,Nono)

Missiles are weapons:

Missile(x)  Weapon(x)

An enemy of America counts as "hostile“:

Enemy(x,America)  Hostile(x)

SAM, who is American …

American(sam)

The country Nono, an enemy of America …

Enemy(Nono,America)
Download Useful Materials @ www.rejinpaul.com
 www.studymaterial

Forward chaining proof Tree

American(x)  Weapon(y)  Sells(x,y,z)  Hostile(z)  Criminal(x)

Missile(x)  Owns(Nono,x)  Sells(sam,x,Nono)
Download Useful Materials @ www.rejinpaul.com
 www.studymaterial

Properties of forward chaining

• Sound and complete for first-order definite clauses

• Datalog:
– Datalog= first-order definite clauses + no functions

• Termination –
-FC terminates for Datalog in finite number of iterations
- May not terminate in general if sentence α is not
entailed

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial

Efficiency of forward chaining

Incremental forward chaining: no need to match a rule
on iteration k if a premise wasn't added on iteration k-
1  match each rule whose premise contains a newly added
positive literal

Matching itself can be expensive:

Database indexing allows O(1) retrieval of known facts

– e.g., query Missile(x) retrieves Missile(M1)

Forward chaining is widely used in deductive databases

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial

RESOLUTION
Resolution:- It means to show that a given
sentence is entailed by the set of sentences in
a KB.
completeness theorem:- any entailed
sentence has a finite proof.

Incompleteness theorem:- sentences that are

entailed, but have no finite proof within the
system.
Download Useful Materials @ www.rejinpaul.com
 www.studymaterial
Conjunctive Normal Form
Everyone who loves all animals is loved by someone.

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial

The resolution inference rule

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial

BINARY RESOLUTION
• Resolution rule that resolves exactly two
literals.
• The binary resolution rule by itself does not
yield a complete inference procedure.

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial

Completeness of Resolution
• Resolution is refutation-complete, which
means that if a set of sentences is
unsatisfiable, then resolution will always be
able to derive a contradiction

• Completeness:-Resolution can be used to

establish that a given sentence is entailed by
the set of sentences.

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial

Resolution Strategies
• Resolution inference rule will eventually find a
proof if one exists

• Resolution Strategies that help find proofs

efficiently.
1. Unit preference
2. Set of support
3. Input resolution
4. Sub-sumption

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial

Unit Preference
• Prefer inferences that produce shorter
clauses.

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial

Set of support

• It starts by identifying a subset of the sentences

called the set of support.
• Every resolution combines a sentence from the
set of support with another sentence and adds
the resolvent into the set of support.
• If the set of support is small relative to the
whole knowledge base, the search space will
be reduced dramatically.

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial

Input Resolution
• Every resolution combines one of the input
sentences (from the KB or the query) with
some other sentence.
• The linear resolution strategy is a slight
generalization that allows P and Q to be
resolved together either if P is in the original
KB or if P is an ancestor of Q in the proof tree.
Linear resolution is complete.

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial

Subsumption
• The subsumption method eliminates all
sentences that are subsumed by an existing
sentence in the KB.
• For example, if P(x) is in the KB, then there is
no sense in adding P(A) and even less sense in
adding P(A) V Q(B).
• Subsumption helps keep the KB small, and
thus helps keep the search space small.

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial

Backward Chaining
• Goal to known facts
• List of goals – stack
• For first goal , the BC algorithm finds every
clause in the KB whose positive literal unifies
with the goal.
• Each such clause creates a new recursive call
in which the premise of the clause is added to
the goal stack.

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial

Backward chaining algorithm

-Algorithm uses composition of substitutions

SUBST(COMPOSE(θ1, θ2), p) = SUBST(θ2, SUBST(θ1, p))

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial

Backward chaining example

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial

Backward chaining example

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial

Backward chaining example

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial

Backward chaining example

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial

Backward chaining example

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial

Backward chaining example

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial

Backward chaining example

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial

Properties of backward chaining

• Depth-first recursive proof search: space is linear in
size of proof.
• Incomplete due to infinite loops
–  fix by checking current goal against every goal on stack
• Inefficient due to repeated sub-goals (both success
and failure)
–  fix using caching of previous results (extra space)
• Widely used for logic programming

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial

Logic programming: Prolog

• Algorithm = Logic + Control

• Program = set of clauses = head :- literal1, … literaln.

criminal(X) :- american(X), weapon(Y), sells(X,Y,Z), hostile(Z).

• Depth-first, left-to-right backward chaining

• Built-in predicates for arithmetic etc., e.g., X is Y*Z+3
• Built-in predicates that have side effects (e.g., input and output
• predicates, assert/retract predicates)
• Closed-world assumption ("negation as failure")
– e.g., given alive(X) :- not dead(X).
– alive(joe) succeeds if dead(joe) fails

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial

Prolog
• Appending two lists to produce a third:
append([],Y,Y).
append([X|L],Y,[X|Z]) :- append(L,Y,Z).

• query: append(A,B,[1,2]) ?

• answers: A=[] B=[1,2]

A=[1] B=[2]
A=[1,2] B=[]

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial

Knowledge
representation
2

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial

Knowledge Representation
using structured objects

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial

Knowledge Representation
using structured objects
Semantic nets

Download Useful Materials @ www.rejinpaul.com

 c o v e re d _ b y www.studymaterial
a n im a l
tr a v e ls _ b y s k in
fly in g
is a is a
tr a v e ls _ b y
b ir d fis h
fe a th e rs
c o v e re d _ b y s w im m in g

is a is a is a is a

o s tr ic h p e n g u in c a n a ry r o b in

tr a v e ls _ b y c o lo u r
c o lo u r

w a lk in g y e llo w
tr a v e ls _ b y

re d

in s ta n c e _ o f

in s ta n c e _ o f
O pus
T w e e ty

c o lo u r
w h ite
Download Useful Materials @ www.rejinpaul.com
 www.studymaterial

Semantic nets

knowledge is represented as a collection

of concepts, represented by nodes
(shown as boxes in the diagram),
connected together by relationships,
represented by arcs (shown as arrows in
the diagram).
certain arcs - particularly isa arcs - allow
inheritance of properties.

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial

Semantic nets

Problems with semantic nets

logical inadequacy - vagueness about what
types and tokens really mean.
heuristic inadequacy – finding a specific
piece of information could be chronically
inefficient.
trying to establish negation is likely to lead
to a combinatorial explosion.
"spreading activation" search is very
inefficient, because it is not knowledge-
guided.
Download Useful Materials @ www.rejinpaul.com
 www.studymaterial

Semantic nets

Attempted improvements
building search heuristics into the
network.
more sophisticated logical structure,
involving partitioning.
these improvements meant that the
formalism’s original simplicity was
lost.

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial

Semantic nets

Developments of the semantic nets idea:

psychological research into whether
human memory really was organised in
this way.
used in the knowledge bases in certain
expert systems: e.g. PROSPECTOR.
special-purpose languages have been
written to express knowledge in semantic
nets.
Download Useful Materials @ www.rejinpaul.com
 www.studymaterial

Knowledge Representation
using structured objects
Frames

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial

Frames

Devised by Marvin Minsky, 1974.

Incorporates certain valuable human
thinking characteristics:
Expectations, assumptions, stereotypes.
Exceptions. Fuzzy boundaries between
classes.
The essence of this form of knowledge
representation is typicality, with
exceptions, rather than definition.
Download Useful Materials @ www.rejinpaul.com
 www.studymaterial

Frames

The idea of frame hierarchies is very

similar to the idea of class hierarchies
found in object-orientated programming.

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial

How frames are organised

A frame system is a hierarchy of frames

Each frame has:
a name.
slots: these are the properties of the entity
that has the name, and they have values.
A particular value may be:
a default value
an inherited value from a higher frame
a procedure, called a daemon, to find a
value
a specific value, which might represent
an exception.
Download Useful Materials @ www.rejinpaul.com
 www.studymaterial

How frames are organised

In the higher levels of the frame

hierarchy, typical knowledge about the
class is stored.
The value in a slot may be a range or a
condition.
In the lower levels, the value in a slot
may be a specific value, to overwrite the
value which would otherwise be
inherited from a higher frame.
Download Useful Materials @ www.rejinpaul.com
 www.studymaterial

How frames are organised

An instance of an object is joined to its

class by an 'instance_of' relationship.
A class is joined to its superclass by a
'subclass_of' relationship.
Frames may contain both procedural
and declarative knowledge.
Slot values normally amount to declarative
knowledge, but a daemon is in effect a
small program. So a slot with a daemon in it
amounts to procedural knowledge.
Download Useful Materials @ www.rejinpaul.com
 www.studymaterial

How frames are organised

Note that a frames system may allow

multiple inheritance but, if it does so, it
must make provision for cases when
inherited values conflict.

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial

Frames: some examples

The following is an exploration of

Minsky's frames, using simple
examples.
Note that this is not exactly the way
Minsky described frames in his original
paper, but it is the way the idea has
come to be used in practice.

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial

Frames: some examples

First I provide a verbal description of some

concept;
then I show a diagram representing the
resulting frame;
beside this is a bit of programming which
implements this frame in a suitable
knowledge representation programming
language - in this case, the MIKE package
written by knowledge engineering
specialists at the Open University.
Download Useful Materials @ www.rejinpaul.com
 www.studymaterial

Frames: some examples

We will start with a simple piece of

information: there is a category of things
called cars.
Given this information, we can start to
build a frame:

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial

Name: car Subclass of: thing

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial

More information: a car has 4 wheels, is

moved by an engine, and runs on petrol
or diesel.
We can now add three slots to the
frame.
The last of these has a restriction rather
than a specific value.

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial
“a car has 4 wheels, is moved by an engine,
and runs on petrol or diesel.”

car subclass_of
Name: car Subclass of: thing thing
with
Slots:
wheels: 4,
Name: Value: Restrictions:
moved_by:
engine,
wheels 4 fuel:
moved by engine [value:
unknown,
fuel ? petrol or diesel type:
[petrol,diesel]].
Download Useful Materials @ www.rejinpaul.com
 www.studymaterial

More information: there is a particular

type of car called a VW, manufactured in
Germany.
We can add a second frame to our
system, with one slot. We don’t need to
repeat the slots and values in the
previous frame: they will be inherited.

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial
“there is a particular type of car called a VW,
manufactured in Germany.”

‘VW’ subclass_of
Name: VW Subclass of: car car
with
Slots:
made_in:
Name: Value: Restrictions:
‘Germany’.
made in Germany

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial

More information: there is a particular

type of VW called a Golf, which has a
sun-roof.
We can add a third frame to our system,
with one slot. Once again, we don’t
repeat the slots in the previous frames,
because they will be inherited.

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial
“there is a particular type of VW called a Golf,
which has a sunroof.”

‘Golf’ subclass_of
Name: Golf Subclass of: VW VW
with
Slots:
top: sunroof.
Name: Value: Restrictions:

top sunroof

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial

More information: there is a particular

type of Golf called a TDi, which runs on
diesel. A TDi has 4 cylinders, and an
engine capacity of 1.8 litres.
We can add a fourth frame to our
system, with three slots. One of the slots
(fuel) was already in the system, but
appears here because it now has a
specific value rather than a restriction.
Download Useful Materials @ www.rejinpaul.com
“there is a particular type of Golf www.studymaterial

which runs on diesel, has 4 cylinders, and has a 1.8

litre engine.”

‘TDi’ subclass_of
Name: TDi Subclass of: Golf ‘Golf’
with
Slots:
fuel: diesel,
Name: Value: Restrictions:
engine_capacity:
fuel diesel 1.8,
cylinders: 4.
engine
capacity 1.8 litres

cylinders 4

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial

More information: my car, called

C637SRK, is a Golf Tdi. It hasn’t got a
sun-roof.
We can add a fifth frame to our system,
with two slots. Unlike all the previous
frames, this one is an instance frame.
One of the slots (top) was already in the
system, but appears here because the
value contradicts (overwrites) the value
which would otherwise be inherited.
Download Useful Materials @ www.rejinpaul.com
 www.studymaterial
“There is a car called C637SRK which is an
instance of a Golf TDi etc.”

‘C637SRK’
Name: C637SRK Instance of: TDi instance_of
‘TDi’
Slots: with
Name: Value: Restrictions: owner: jp,
top:
owner jp
no_sun_roof.

top no sun-roof

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial

More information: to calculate a car’s

cylinder size, you divide the total engine
capacity by the number of cylinders.
We can now add another slot to the car
frame. It’s a daemon, which discovers
the information required, if the user asks
what the value in the slot is.

Download Useful Materials @ www.rejinpaul.com

“to calculate a car’s cylinder size, you www.studymaterial
total engine capacity by the number of
cylinders.”

car subclass_of thing

with
Name: car Subclass of: thing wheels: 4,
moved_by: engine,
Slots: fuel: [value: unknown,
type: [petrol,diesel]],
Name: Value: Restrictions: cylinder_size:
wheels 4 [value: unknown,
moved by engine access_rule:
fuel ? petrol or diesel. (if the
cylinder size ? find total engine engine_capacity of
?self is E & the
capacity, find cylinders of ?self is
no.of cylinders, C & Ans := E/C
divide first by then make_value
second.
Download Useful Materials Ans)].
@ www.rejinpaul.com
 www.studymaterial

Knowledge Representation
using structured objects
Other varieties of structured object

Download Useful Materials @ www.rejinpaul.com

Other varieties of structured www.studymaterial

object
Knowledge representation researchers -
particularly Roger Schank and his
associates - devised some interesting
variations on the theme of structured
objects.
In particular, they invented the idea of
scripts (1973).
A script is a description of a class of
events in terms of contexts, participants,
and sub-events.
Download Useful Materials @ www.rejinpaul.com
 www.studymaterial

Scripts

Rather similar to frames: uses inheritance

and slots; describes stereotypical
knowledge, (i.e. if the system isn't told
some detail of what's going on, it assumes
the "default" information is true), but
concerned with events.
Somewhat out of the mainstream of
expert systems work. More a development
of natural-language-processing research.
Download Useful Materials @ www.rejinpaul.com
 www.studymaterial

Scripts

Why represent knowledge in this way?

Because real-world events do follow
stereotyped patterns. Human beings use
previous experiences to understand verbal
accounts; computers can use scripts instead.
Because people, when relating events, do
leave large amounts of assumed detail out of
their accounts. People don't find it easy to
converse with a system that can't fill in
missing conversational detail.
Download Useful Materials @ www.rejinpaul.com
 www.studymaterial

Scripts

Scripts predict unobserved events.

Scripts can build a coherent account from
disjointed observations.

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial

Scripts

Commercial applications of script-like

structured objects: work on the basis that a
conversation between two people on a pre-
defined subject will follow a predictable
course.
Certain items of information need to be
exchanged.
Others can be left unsaid (because both people
know what the usual answer would be, or can
deduce it from what's been said already), unless
Download Useful Materials @ www.rejinpaul.com
 www.studymaterial

Scripts

This sort of knowledge representation has

been used in intelligent front-ends, for
systems whose users are not computer
specialists.
It has been employed in story-
understanding and news-report-
understanding systems.

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial

Conceptual graphs

In 1984, John Sowa published his

conceptual graph approach.
This is something like the semantic net
approach described earlier.
It is more sophisticated in the way in
which it represents concepts
It is much more precise regarding what
the objects in the graphs mean in real-
world terms.
Download Useful Materials @ www.rejinpaul.com
 www.studymaterial

Knowledge Representation
using structured objects
Practical applications

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial

KBSs using structured objects

Two more examples of KBSs using

structured objects for knowledge
representation

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial

Explorer

Explorer is a natural-language interface

for a database relating to oil-wells:
Contains geological knowledge,
organised as structured objects.
Interfaced to a graphics package that
can draw oil-exploration maps.
The geologist can converse with the
program in unconstrained natural
language, and in geological jargon.
Download Useful Materials @ www.rejinpaul.com
 Bank customer advisory www.studymaterial

system
Bank customer advisory system:
Advises customer on opening a bank
account, based on customer's
needs/characteristics. This is a task that
can consume a lot of the bank staff's time.
Deduces customer's intended meaning.
Does not annoy customer by asking for
every detail; uses its knowledge about
people to deduce what the answers must
be. Download Useful Materials @ www.rejinpaul.com
 www.studymaterial

Knowledge Representation
using structured objects
The CYC project

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial

The CYC Project

A very large scale commercial research

project designed to represent a large
body of common-sense knowledge.
Headed by Lenat and Guha, and based
in Austin, Texas
Has been making steady progress since
1984

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial

The CYC Project

It now ‘knows’ a huge collection of

fragments of real-world knowledge such
as:
Mothers are older than their children.
You have to be awake to eat.
You can usually know people’s noses, but
not their hearts.
If you cut a lump of peanut butter in half,
each half is a lump of peanut butter, but if
you cut a table in half, neither half is a table.
Download Useful Materials @ www.rejinpaul.com
 www.studymaterial

The CYC Project

The ultimate objective is to give it enough

knowledge to understand ordinary books,
so that it can read them and expand its
own knowledge.
So far, it’s got to a stage where, when
asked to find photos of “risky activities”, it
located photos of people climbing
mountains and doing white-water rafting.

Download Useful Materials @ www.rejinpaul.com

 www.studymaterial

Knowledge Representation
using structured objects
Final comments

Download Useful Materials @ www.rejinpaul.com

 Structured objects: final www.studymaterial

comments
A knowledge-representation system with
property inheritance should make a
distinction between essential properties
('universal truths') and accidental properties.
Some don't: some allow unrestrained overwriting
of inherited properties.
It's possible to generate incoherent structures in
any of these systems.
It's particularly easy when multiple inheritance is
involved.
Download Useful Materials @ www.rejinpaul.com
 Structured objects: final www.studymaterial

comments
At best, structured objects provide data
& control structures which are more
flexible than those in conventional
languages, and so more suited to
simulating human reasoning.

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com

LECTURE 2:
AGENT
ARCHITECTURES

Artificial Intelligence II – Multi-Agent Systems

Introduction to Multi-Agent Systems
URV, Winter - Spring 2010

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
Outline of the talk

„ Intelligent agent – reactivity

„ Environments

„ Agent architectures

‰ Reactive
‰ Deliberative

‰ Hybrid

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
Agent definitions
„ "An agent is anything that can be viewed as
perceiving its environment through sensors and
acting upon that environment through effectors.”
„ "Autonomous agents are computational systems
that inhabit some complex dynamic environment,
sense and act autonomously in this environment,
and by doing so realize a set of goals or tasks for
which they are designed."
„ “An autonomous agent is a system situated within
and a part of an environment that senses that
environment and acts on it, over time, in pursuit of
its own agenda and so as to affect what it senses
in the future.”

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
Basic abstract view of an agent

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
Reactivity
„ An agent has to be able to react [adapt its
behaviour] in an appropriate way to the dynamic
changes in its “environment”
‰ Other computational agents
‰ Human agents/users
‰ External information sources (e.g. sensors)
‰ Physical objects (e.g. robots)
‰ Internet
‰ ...

„ This is one of several properties that an intelligent

agent should have ... [more on that next week]

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
Kinds of environments (I)

„ Accessible vs inaccessible

‰ An accessible environment is one in which the

agent can obtain complete, accurate, up-to-date
information about the environment’s state.
‰ Most moderately complex environments (including,
for example, the everyday physical world and the
Internet) are inaccessible.
‰ The more accessible an environment is, the simpler
it is to build agents to operate in it.

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
Kinds of environments (II)
„ Deterministic vs non-deterministic
„ A deterministic environment is one in
which any action has a single guaranteed
effect — there is no uncertainty about the
state that will result from performing an
action.
„ The physical world can to all intents and
purposes be regarded as non-
deterministic.
„ Non-deterministic environments present
greater problems for the agent designer.

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
Kinds of environments (III)
‰ Episodic vs non-episodic

„ In an episodic environment, the performance of

an agent is dependent on a number of discrete
episodes, with no link between the performance
of an agent in different scenarios.
„ Episodic environments are simpler from the
agent developer’s perspective because the
agent can decide what action to perform based
only on the current episode — it need not reason
about the interactions between this and future
episodes.

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
Kinds of environments (IV)
‰ Static vs dynamic

„ A static environment is one that can be assumed to

remain unchanged except by the performance of
actions by the agent.
„ A dynamic environment is one that has other
processes operating on it, and which hence
changes in ways beyond the agent’s control.
„ The physical world is a highly dynamic
environment.

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
Kinds of environments (V)

‰ Discrete vs continuous

„ An environment is discrete if there are a fixed,

finite number of actions and percepts in it.
„ The real world is a continuous environment.

Practical exercise, step 1: think about the kind of environment

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
Agent architectures
„ An architecture proposes a particular
methodology for building an autonomous
agent
‰ How the construction of the agent can be
decomposed into the construction of a set of
component modules
‰ How these modules should be made to interact
‰ These two aspects define how the sensor data
and the current internal state of the agent
determine the actions (effector outputs) and
future internal state of the agent

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
From perception to action

f = state update function

s = internal state
g = output function

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
Main kinds of agent architectures

„ Reactive architectures
‰ Focused on fast reactions/responses to changes
detected in the environment
„ Deliberative architectures (symbolic)
‰ Focused on long-term planning of actions, centred
on a set of basic goals
„ Hybrid architectures
‰ Combining a reactive side and a deliberative side

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
Reactive vs Deliberative: example
„ Robot that has to reach a certain point
‰ Reactive
„ Sensor in the front of the robot
„ Change movement right/left when sensor detects obstacle
‰ Minimal computation based on current location and destination
point
‰ Deliberative
„ Explicit representation of the environment (map)
„ Planning procedure that finds the minimal route between the
current position and the destination
‰ High computational cost
‰ Possible dynamic re-plannings needed

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
Reactive Architectures
„ There are many unsolved (some would say
insoluble) problems associated with
symbolic AI
‰ Computational cost, brute search

‰ Problems below the 100 ms threshold

„ For example, face recognition

„ These problems have led some researchers to question the

viability of the whole paradigm, and to the development of
reactive architectures
„ Although united by a belief that the assumptions
underpinning mainstream AI are in some sense wrong,
reactive agent researchers use many different techniques

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
Reactive agents – basic ideas
‰ Reactive agents have
„ at most a very simple internal representation of
the world,
„ but provide tight coupling of perception and
action
‰ Behaviour-based paradigm
‰ Intelligence is a product of the interaction

between an agent and its environment

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
Classic example: ant colony
„ A single ant has very
little intelligence,
computing power or
reasoning abilities
„ The union of a set of
ants and the
interaction between
them allows the
formation of a highly
complex, structured
and efficient system.

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
Main characteristics (I)

„ Emergent functionality
‰ Simple agents
‰ Simple interaction
‰ Complex behaviour patterns appear as a result of
the dynamic interactions
‰ The global behaviour of the system is not
specified a priori
„ Dynamic movement of robots, depending on obstacles

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
Main characteristics (II)

„ Task decomposition
‰ Agents composed of autonomous modules
‰ Each module manages a given task
„ Sensor, control, computations
‰ Minimal, low-level communication between
modules
‰ There isn’t any world global model
‰ There isn’t any “planning/controller agent”

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
Main characteristics (III)

„ Raw data
‰ Basic data from sensors
‰ There isn’t any complex symbolic
management of data as in classical AI
„ Refusal of the Hypothesis of the physic symbols
system [basic pillar of symbolic AI]
‰ “Intelligent behaviour can only be obtained in symbol-
processing systems”

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
Basic concept
‰ Each behaviour continually maps perceptual
input to action output
‰ Reactive behaviour: action rules: S A
where S denotes the states of the environment,
and A the primitive actions the agent is capable of
performing.
‰ Example:
Heater off, if temperature is OK in state s
action(s) =
Heater on, otherwise

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
Basic schema of reactive architecture
Agent

Stimulus-response behaviours

State
State11 Action
Action11 EE
SS ff
ee ff
nn State
State22 Action
Action22 ee
ss cc
oo .. .. tt
rr . . oo
ss rr
ss
State
Statenn Action
Actionnn

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
Rodney Brooks

Director of the
Computer
Science and
Artificial
Intelligence Lab
(MIT)
1997-2007

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
Brooks refutal of symbolic AI
„ Brooks has put forward three theses:
1. Intelligent behaviour can be generated without
explicit representations of the kind that symbolic
AI proposes
2. Intelligent behaviour can be generated without
explicit abstract reasoning of the kind that
symbolic AI proposes
‰ Reduced computation on sensor-like data
3. Intelligence is an emergent property of certain
complex systems

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
Brooks – key ideas (I)
‰ Situatedness: ‘Real’ intelligence is situated
in the world
„ The world is its best model
„ The world is always up-to-date
„ A model is an abstraction, a simplification of
the world, considering a particular set of
characteristics and disregarding others

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
Brooks – key ideas (II)
‰ Embodiment: ‘Real’ intelligence requires a
physical body, and cannot be found in
disembodied systems such as theorem
provers or expert systems
„ Physical robots
‰ Intelligence and emergence: ‘Intelligent’
behavior arises as a result of an agent’s
interaction with its environment. Also,
intelligence is ‘in the eye of the beholder’; it is
not an innate, isolated property

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
Brooks – behaviour languages
„ To illustrate his ideas, Brooks built some
systems based on his subsumption
architecture
„ A subsumption architecture is a hierarchy
of task-accomplishing behaviours
„ Each behaviour is a rather simple rule-like
structure
„ Each behaviour ‘competes’ with others to
exercise control over the agent, as different
behaviours may be applicable at the same
time

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
Behaviour layers
„ Lower layers represent more primitive kinds of
behaviour (such as avoiding obstacles)
„ Higher layers represent more complex
behaviours (e.g. identifying an object)
„ Lower layers have precedence over layers
further up the hierarchy
„ The resulting systems are, in terms of the
amount of computation they do, extremely
simple
„ Some of the robots do tasks that would be
impressive if they were accomplished by
symbolic AI systems

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
Decomposition of a Mobile Robot
Control System into Functional Modules

Modules in Classical AI systems

From Brooks, “A Robust Layered Control System for a Mobile Robot”, 1985

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
Decomposition Based on
Task Achieving Behaviours

From Brooks, “A Robust Layered Control System for a Mobile Robot”, 1985
Download Useful Materials @ www.rejinpaul.com
 www.rejinpaul.com
Layered Control in the Subsumption
Architecture

From Brooks, “A Robust Layered Control System for a Mobile Robot”, 1985
Download Useful Materials @ www.rejinpaul.com
 www.rejinpaul.com
Schematic of a Module

From Brooks, “A Robust Layered Control System for a Mobile Robot”, 1985
Download Useful Materials @ www.rejinpaul.com
 www.rejinpaul.com
Levels 0, 1, and 2 Control Systems

Avoid
collisions

Random
movement

Pick up
objects,
monitor,
explore

From Brooks, “A Robust Layered Control System for a Mobile Robot”, 1985
Download Useful Materials @ www.rejinpaul.com
 www.rejinpaul.com

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
Steels’ Mars Explorer

„ Steels’ Mars explorer system, using the

subsumption architecture, achieves
near-optimal cooperative performance in
simulated ‘rock gathering on Mars’
domain:
The objective is to explore a distant
planet, and in particular, to collect
sample of a precious rock. The location
of the samples is not known in advance,
but it is known that they tend to be
clustered.

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
Steels’ Mars Explorer Rules (I)
„ For individual (non-cooperative) agents, the lowest-level
behaviour, (and hence the behaviour with the highest
“priority”) is obstacle avoidance:
if detect an obstacle then change direction (1)
„ Any samples carried by agents are dropped back at the
mother-ship:
if carrying samples and at the base
then drop samples (2)
„ Agents carrying samples will return to the mother-ship:
if carrying samples and not at the base
then travel up gradient (3)

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
Steels’ Mars Explorer Rules (II)

„ Agents will collect samples they find:

if detect a sample then pick sample up (4)
„ An agent with “nothing better to do” will explore
randomly:
if true then move randomly (5)

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
Situated Automata
„ A sophisticated theoretical approach is that of
Rosenschein and Kaelbling
„ In their situated automata paradigm, an agent is
specified in a rule-like (declarative) language, and
this specification is then compiled down to a digital
machine, which satisfies the declarative
specification
„ This digital machine can operate in a provable
time bound
„ Reasoning is done off line, at compile time, rather
than online at run time

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
Situated Automata components

„ An agent is specified in terms of two

components: perception and action
„ Two programs are then used to synthesize
agents
‰ RULER is used to specify the perception
component of an agent
‰ GAPPS is used to specify the action component

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
Circuit Model of a Finite-State Machine

f = state update function

s = internal state
g = output function

From Rosenschein and Kaelbling,

“A Situated View of Representation and Control”, 1994
Download Useful Materials @ www.rejinpaul.com
 www.rejinpaul.com
Circuit Model of a Finite-State Machine

GAPPS
RULER

From Rosenschein and Kaelbling,

“A Situated View of Representation and Control”, 1994
Download Useful Materials @ www.rejinpaul.com
 www.rejinpaul.com
RULER – Situated Automata
„ RULER takes as its input three components
‰ The semantics of the agent's inputs (‘whenever bit 1 is on,
it is raining’)
‰ A set of static facts (‘whenever it is raining, the ground is
wet’)
‰ A specification of the state transitions of the world (‘if the
ground is wet, it stays wet until the sun comes out’).
„ The programmer then specifies the desired
semantics for the output (‘if this bit is on, the ground
is wet’)
„ The compiler designs a circuit whose output will
have the correct semantics

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
GAPPS – Situated Automata
„ The GAPPS program takes as its input
‰ A set of goal reduction rules,
„ Rules that encode information about how goals can be
achieved in a given state
‰ A top level goal
„ Then it generates a program that can be
translated into a digital circuit in order to
realize the goal
„ The generated circuit does not represent or
manipulate symbolic expressions; all symbolic
manipulation is done at compile time

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
Advantages of Reactive Agents
„ Simplicity of individual agents
„ Flexibility, adaptability
‰ Ideal in very dynamic and unpredictable environments
„ Computational tractability
‰ Avoiding complex planning/reasoning procedures
‰ Avoiding continuous model update
„ Robustness against failure
‰ No central planning component (e.g. ant colony)
„ Elegance

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
Limitations of Reactive Agents (I)
„ Agents without environment models must have
sufficient information available from local
environment
„ If decisions are based on local environment,
how can we take into account non-local
information?
‰ “Short-term” view
„ No long-term planning capabilities
„ Limited applicability
‰ Games, simulations, basic robots (insects)

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
Limitations of Reactive Agents (II)

„ Difficult to make reactive agents that learn

‰ Dynamic evolution of rules?
„ Since behaviour emerges from component
interactions plus environment, it is hard to
see how to engineer specific agents (no
principled methodology exists)
„ It is hard to engineer agents with large
numbers of behaviours (dynamics of
interactions become too complex to
understand)

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
Deliberative agent architecture

‰ Explicit symbolic model of the world

‰ Decisions are made via logical reasoning,
based on pattern matching and symbolic
manipulation
‰ Sense-plan-act problem-solving paradigm
of classical AI planning systems

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
Basic deliberative architecture

Agent

EE
SS ff
ee ff
nn ee
ss World
World Plan
Plan cc
Planner
Planner
oo Model
Model executor
executor tt
rr oo
ss rr
ss

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
Belief-Desire-Intention (BDI) model
„ A theory of practical
reasoning.
„ Originally developed by
Michael E. Bratman in his
book ”Intentions, Plans,
and Practical Reason”,
(1987).
„ Concentrates in the roles
of the intentions in
practical reasoning.

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
Practical reasoning
„ Reasoning directed towards actions — the
process of figuring out what to do:
“Practical reasoning is a matter of weighing conflicting
considerations for and against competing options,
where the relevant considerations are provided by
what the agent desires and what the agent believes.”
(Bratman)

”We deliberate not about ends, but about means. We

assume the end and consider how and by what means
it is attained.” (Aristotle)

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
Human practical reasoning
„ Human practical reasoning consists of two
activities:
‰ Deliberation, deciding what state of affairs
we want to achieve
„ the outputs of deliberation are intentions
‰ Means-ends reasoning, deciding how to
achieve these states of affairs
„ the outputs of means-ends reasoning are
plans

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
Belief-Desire-Intention paradigm
„ Desires:
‰ Follow from the beliefs.
„ Beliefs:
Desires can be unrealistic
‰ Agent’s view of the and inconsistent.
environment/world. „ Goals:
‰ A subset of the desires.
Realistic and consistent.
Determine potential
processing.
„ Plans: „ Intentions:
‰ A subset of the goals. A
‰ Sequences of actions that
goal becomes an
are needed to achieve the intention when an agent
intentions, given the decides to commit to it
agent’s beliefs (e.g. by assigning
priorities to goals)

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
BDI plans
„ In BDI implementations plans usually have:
‰ a name
‰ a goal
invocation condition that is the triggering event for the plan
‰ a pre-condition list
list of facts which must be true for plan to be executed
‰ a delete list
list of facts that are no longer true after plan is performed
‰ an add list
list of facts made true by executing the actions of the plan
‰ a body
list of actions

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
Belief-Desire-Intention architecture

Desires

Goals
Knowledge

Beliefs

Plans
Intentions

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
Intention is choice with commitment
(Cohen & Levesque)
„ There should be ”rational balance” among the beliefs, goals,
plans, intentions, commitments and actions of autonomous
agents.
„ Intentions play a big role in maintaining ”rational balance”
„ An autonomous agent should act on its intentions, not in spite of
them
‰ adopt intentions that are feasible

‰ drop the ones that are not feasible

‰ keep (or commit to) intentions, but not forever

‰ discharge those intentions believed to have been satisfied

‰ alter intentions when relevant beliefs change

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
Using plans to constrain reasoning

„ An agent’s plans serve to frame its subsequent

reasoning problems so as to constrain the amount
of resources needed to solve them
‰ Agents commit to their plans

‰ Their plans tell them what to reason about, and

what not to reason about

‰ Plans can help reasoning in differents levels of

abstraction

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
Intention reconsideration
„ Intentions (plans) enable the agent to be goal-driven
rather than event-driven.
„ By committing to intentions the agent can pursue long-
term goals.
„ However, it is necessary for a BDI agent to reconsider
its intentions from time to time:
‰ The agent should drop intentions that are no longer

achievable.
‰ The agent should adopt new intentions that are

enabled by opportunities.

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
Problems in the deliberative approach
„ Dynamic world
‰ Update symbolic world model
‰ World changes while planning is being done
„ Representation language
‰ Expressive enough to be useful in any domain
‰ Limited enough to be computationally tractable
„ Classical planning => complete, optimal solutions
‰ High computational cost
‰ Sometimes a sub-optimal low-cost fast reaction can be
effective

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
Hybrid Approaches
„ Many researchers have argued that neither a
completely deliberative nor a completely reactive
approach are suitable for building agents
„ They have suggested using hybrid systems, which
attempt to marry classical and alternative approaches
„ An obvious approach is to build an agent out of two
(or more) subsystems:
‰ a deliberative one, containing a symbolic world model, which
develops plans and makes decisions in the way proposed by
symbolic AI
‰ a reactive one, which is capable of reacting quickly to events
without complex reasoning

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
Hybrid agent architecture
Agent
Deliberative component

World
World Plan
Plan
Planner
Planner
Model
Model executor
executor

EE
SS observations modifications ff
ee ff
nn ee
ss Reactive component cc
oo tt
oo
rr rr
ss State1 Action1 ss

State2 Action2
.. ..
. .
Staten Actionn

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
Layered Architectures
„ Often, the reactive component is given some
kind of precedence over the deliberative one
„ This kind of structuring leads naturally to the
idea of a layered architecture, of which
TOURINGMACHINES and INTERRAP are
examples
„ In such an architecture, an agent’s control
subsystems are arranged into a hierarchy,
with higher layers dealing with information at
increasing levels of abstraction

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
Layering techniques
„ A key problem in such architectures is what kind of
control framework to embed the agent’s subsystems
in, to manage the interactions between the various
layers.
„ Horizontal layering
Each layer is directly connected to the sensory input
and action output.
In effect, each layer itself acts like an agent,
producing suggestions as to what action to perform.
„ Vertical layering
Sensory input and action output are dealt with by at
most one layer each.

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
Horizontal layering
m possible actions suggested by each layer, n layers

O(mn) possible
options to be
considered

Introduces bottleneck
in central control system

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
Vertical layering
m possible actions suggested by each layer, n layers

O(mn) interactions
between layers

Not fault tolerant to

layer failure

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
Ferguson – TOURINGMACHINES

„ The TOURINGMACHINES architecture

consists of perception and action
subsystems, which interface directly with the
agent’s environment, and three control
layers, embedded in a control framework,
which mediates between the layers

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
Ferguson – TOURINGMACHINES

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
Ferguson – TOURINGMACHINES
„ The reactive layer is implemented as a set of
situation-action rules, a la subsumption architecture

rule-1: obstacle-avoidance
if
is-in-front(Obstacle, Observer) and
speed(Observer) > 0 and
separation(Obstacle, Observer) < ThreshHold
then
change-orientation(ObstacleAvoidanceAngle)

„ The planning layer constructs plans and selects

actions to execute in order to achieve the agent’s
goals

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
Ferguson – TOURINGMACHINES
„ The modeling layer contains symbolic
representations of the ‘cognitive state’ of
other entities in the agent’s environment
„ The three layers communicate with each
other and are embedded in a control
framework, which use control rules
censor-rule-1:
if entity(obstacle-6) in perception-buffer
then remove-sensory-record(layer-R, entity(obstacle-6))
[Prevents the reactive layer from reacting in front of obstacle-6]

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
Müller –InteRRaP
„ Vertically layered, two-pass architecture

cooperation layer social knowledge

plan layer planning knowledge

behaviour layer world model

world interface

perceptual input action output

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
Behaviour layer
„ Reactive part of the architecture
„ Works with the world model (beliefs on the
world state)
„ Only level that interacts with the real world
„ Has a set of “situation action” rules
‰ Fast recognition of situations that deserve a
quick reaction
„ Makes routine tasks efficiently, without
complex symbolic planning

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
Planning layer

„ Works with the mental model (beliefs on the

own agent)
„ Standard deliberative level
„ Implements local behaviour guided towards
certain goals

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
Cooperative planning layer

„ Works with the social model (beliefs on

other agents of the system)
„ Allows planning and cooperation with other
agents
‰ Global plans of action
‰ Conflict resolution

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
Critiques to hybrid architectures

„ Lack of general design guiding

methodologies
„ Very specific, application dependent
„ Unsupported by formal theories

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
Readings for this week

„ M.Wooldridge: An introduction to
MultiAgent Systems – chapter 5
(reactive, hybrid)
„ A.Mas: Agentes software y sistemas
multi-agente: conceptos,
arquitecturas y aplicaciones –
chapter 2
„ Articles/book chapters on Moodle
web site

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
Reactive agents, mind map (Wooldridge)

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
Hybrid agents, mind map (Wooldridge)

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com

LECTURE 5:
Agent Communication

Artificial Intelligence II – Multi-Agent Systems

Introduction to Multi-Agent Systems
URV, Winter-Spring 2010

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
Multi-agent systems course (I)
„ First part of the course: agents
‰ Introduction to agent technology
‰ Agent architectures
„ Reactive / Deliberative / Hybrid
‰ Agent properties
„ Autonomy / Communication / Learning / Reactivity /
Mobility / Reasoning / Proactivity / ...
‰ Agent types
„ Cooperative / Interface / Information / Mobile / Reactive /
Hybrid / Heterogeneous systems

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
Multi-agent systems course (II)
„ Second part of the
course: Multi-Agent
Systems
‰ Agent communication
‰ Coordination
‰ Negotiation
‰ Distributed planning
‰ Coalition formation
‰ Application domains

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
Overview of today’s lecture

„ Collaborative agents
„ Distributed problem solving
„ Agent communication
‰ Blackboard systems
‰ Message passing
‰ FIPA standards
‰ Basic communication protocols

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com

Collaborative Agents (I)

„ A multi-agent system (MAS)
may be seen as a collection of
collaborative agents
„ They can communicate and
cooperate with other agents,
while keeping their autonomy
„ They usually negotiate with
their peers to reach mutually
acceptable agreements during
cooperative problem solving

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
Collaborative Agents (II)
„ They normally have limited learning
capabilities
„ Collaborative agents are usually
deliberative agents (e.g. BDI model), with
some reasoning capabilities
‰ Reactive agents can hardly communicate
and collaborate (only through actions that
modify the common environment)
„ They are usually static, complex agents

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com

Collaborative Agents: Motivations (I)

„ To solve problems that are too large for a
single centralised agent
‰ Huge amount of knowledge to be considered
‰ Many computational resources needed to
solve the problem
‰ Risk of having a centralised system

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
Collaborative Agents: Motivations (II)
„ To create a system that functions beyond the
capabilities of any of its members
‰ Added value of a MAS

„ To allow for the interconnection and inter-

operation of existing legacy systems
[ Recall agentification mechanisms seen on the
previous lecture: translators, wrappers]
‰ DBs, expert systems, electronic equipment, sensors

Example: organ transplant coordination

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
Finding a transplant receiver
Spanish Model
Process defined by the
Spanish Transplant
Organisation (ONT)

0-Emergency Patients in all Spain

Donor’s hospital

Other hospitals in the same city

Other cities of the same region
Other regions of the same zone
Rest of Spain
Download Useful Materials @ www.rejinpaul.com
 www.rejinpaul.com
Architecture of the MAS
National Level National
Coordinator

Zonal Level ZI ZII ZIII ZIV ZV ZVI

Balearic Catalonia Aragon

Regional Level
Islands

Barcelona

H1 H2 ... Hn
Hospital Level

Urgency
Emergency Level
Agent

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com

Collaborative Agents: Applications

„ Provide solutions to physically distributed
problems
‰ Disaster in a city (police, firemen, ambulances)
„ Provide solutions to problems with
distributed data sources
‰ Sensor network monitoring a given area
„ Provide solutions that need distributed
expertise
‰ Health care provision (family doctors, nurses,
specialists, laboratory analysis, …)

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
Benefits of Multi-Agent Systems (I)
„ Modularity
‰ Each agent is specialised in the solution of a
particular kind of problems (leading also to
reusability)
‰ The complexity of the construction of agents is
reduced
‰ The process of solving a complex problem is
reduced to solving easier subproblems

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
Benefits of Multi-Agent Systems (II)

„ Efficiency
‰ Problems can be solved more quickly, due to the
inherent concurrency/parallelism
‰ Different agents are working at the same time in
different parts of a problem
„ These subproblems can be independent or (slightly)
dependent
‰ Share partial results
‰ Coordinate the use of shared resources

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
Benefits of Multi-Agent Systems (III)

„ Reliability
‰ Avoid single point of failure in centralised
systems
‰ We can have redundancy
„ Different agents of the same type
„ Different agents that can do a certain task
‰ If an individual agent fails, the other agents
can take its work and re-distribute it
dynamically

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
Benefits of Multi-Agent Systems (IV)

„ Flexibility
‰ Agents can be created/deleted dynamically,
depending on the amount of work to be done, the
available resources,etc
‰ Agents can dynamically generate subtasks and
look for helping agents
‰ Agents with different skills may dynamically form
teams/coalitions to work together

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
Distributed problem solving
„ Agents work together to solve
problems that require collective
effort
„ Requires:
‰ Coherence
„ Need to want to work together
‰ Competence
„ Need to know how to work together
„ Distributed Planning
‰ Distributed Problem Solving in
which the agents work together to
construct a plan

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
Distributed problem solving steps

1. Task Decomposition
2. Task Allocation
3. Task Accomplishment
4. Result Synthesis

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
1-Task decomposition
„ Divide a complex problem into a set of tasks (e.g.
a hierarchy of subproblems)
„ Decompose (recursively, if necessary) large
tasks into subtasks that can be tackled by
different agents
„ Tasks can be totally independent or can have
some relationships
„ This decomposition may be done by the user, by
a central coordinator agent or may be negotiated
at run time by all the agents of the system

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
Example of hierarchical task
decomposition in an AND-OR tree

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
2-Task allocation
„ Assign the subproblems to different agents
„ Easy case: there is only one agent capable of
solving each subproblem
‰ No flexibility
„ Interesting case: the same subproblem
can be solved by different agents
‰ Dynamic mechanisms of run-time task allocation
(e.g. Contract Net)
‰ Possibility of recovering from
individual failures

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
3-Task accomplishment
„ Each agent solves (on its own) its assigned
subproblems
„ This phase can be done, in principle, without
communication (if the problems are
independent)
„ There may be dependencies between
subproblems
‰ Potential conflicts may appear

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
Conflict management

„ Examples
‰ Two sub-solutions are incompatible
‰ Conflicts in the use of shared resources
„ Agents have to communicate with each other to
solve these situations
„ There may be different solutions to the same
subproblem

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
Agent support in task execution (I)
„ Task sharing
‰ An agent can request the help of other agents to
solve a particular task
„ Too complex / expensive for the agent to do individually
„ It can know that other agents have the appropriate
knowledge/skills to solve that task
„ It can know that other agents already have to solve that
task
‰ Problem of task assignment
„ Who can I ask for help?
„ How do I know what tasks can other agents do?

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
Agent support in task execution (II)
„ Result sharing
‰ Use intermediate results obtained by other agents
[ For example, traffic routes to certain points of the city ]
‰ Agents can provide intermediate subsolutions to help other
agents in their work
‰ That allows a fast recognition of
„ Incorrect solutions
‰ An agent, on the basis of its knowledge, can detect an error on
the results of other agents
„ Conflictive solutions
‰ An agent can detect possible conflicts between its results and
subsolutions of other agents
‰ Cooperation/Negotiation to solve these problems

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
4-Result synthesis

„ Put together the results of all agents to

find the complete solution
„ Who makes it?
„ How is it made?
„ If each subproblem has a unique
solution, it is a relatively easy step
„ Otherwise, there may be need of
conflict detection, task re-assignment,
...

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
Summary: collaborative MAS (I)

„ Who decomposes the problem?

‰ One agent / all agents / user
„ How is the problem decomposed?
‰ 1 level / n levels
‰ Static – beginning / Dynamic – execution time
„ Who assigns tasks to agents? How is the
assignment made?
‰ Static – beginning / Dynamic – execution time

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
Summary: collaborative MAS (II)
„ What kind of interactions can exist between
agents when they are solving tasks?
‰ Conflict detection
‰ Conflict resolution
„ How are tasks shared?
„ How are results shared?
„ Who/how merges the subresults to get the
final result?

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com

Why do we need Agent Communication?

„ Multi agent systems allow distributed
problem solving
„ This requires the agents to coordinate
their actions
„ Agent communication facilitates this by
allowing individual agents to interact
‰ allows cooperation
‰ allows information sharing

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com

Communication Categories

„ We can classify communication in a

mechanistic manner
‰ via the type of sendee-addressee link
‰ via the nature of the medium
„ or in a higher level meaning-based
manner
‰ via the type of intention

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com

The Sendee - Addressee Link

„ Communication can be
‰ Point to Point
„ An agent talks directly to another agent
‰ Broadcast
„ An agent sends some information to a group
of agents
‰ Mediated
„ The communication between two agents is
mediated by a third party
„ Example: facilitators

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
Communication via facilitator (I)
Agent 1 Agent 2

5 1
3
2 4
Facilitator

1. Agent_2 tells the facilitator the services it provides

2. Agent_1 asks to the facilitator who can provide a certain
service with some conditions
3. The facilitator requests the service to agent_2
4. Agent_2 provides the answer
5. The facilitator sends the answer to agent_1

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
Communication via facilitator (II)
4
Agent 1 Agent 2
5
3 1
2
Facilitator

1. Agent_2 tells the facilitator the services it provides

2. Agent_1 asks to the facilitator who can provide a certain
service with some conditions
3. The facilitator tells agent_1 that agent_2 can do that
service
4. Agent_1 requests the service from agent_2
5. Agent_2 sends the answer to agent_1

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com

Locating other agents

„ Unless we use some broadcast techniques
(e.g. blackboard systems), agents must
know the address of other agents - possible
solutions are
‰ Complete internal directory
‰ Partial/hierarchical internal directory
‰ Mediated (e.g. JADE’s DF)

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
JADE – Directory Facilitator

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com

Nature of the medium

„ Direct routing
‰ Message sent directly to other agent(s) with no interception
or attenuation in strength
„ Signal propagation routing
‰ Commonly used by reactive agents
‰ Agent sends signal whose intensity decreases according to
distance (e.g. physical robots)
„ Public notice routing
‰ Blackboard systems

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
Agent communication

„ Basic options used in MAS

‰ Blackboard systems
‰ Direct message passing

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
Blackboard Systems

agent
agent agent

agent blackboard agent

agent agent
agent

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
Blackboard-based communication
„ Each agent can put
information/data/knowledge on the
common information space
„ Each agent can read from the
blackboard at any moment
„ There is no direct communication
between agents

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
Information in blackboard
„ Data of the common problem
„ Current state of the solution
„ Next subproblems to be solved
„ Requests of help
„ Present task of each agent
„ Intermediate results

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
Uses of blackboard
‰ Detect conflicts
„ Different agents that want to perform the same task
‰ Notice incompatible solutions
„ Solutions using a shared resource at the same time
‰ Share results
„ Agents can use partial/complete results obtained by
other agents
‰ Share tasks
„ Agents can request help in solving sub-tasks

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
Advanced blackboard systems (I)
„ Moderator agent
‰ Advertises in the blackboard the next problems to
be solved
‰ Checks which agents offer to solve them
‰ Assigns the pending problems to the agents
„ It has internal domain and system knowledge to make
this assignment
„ The blackboard is also used to communicate the
assignments

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
Advanced blackboard systems (II)
„ Dispatcher agent
‰ Tells the agents registered in a blackboard
about the changes produced on it that can be
interesting / relevant for them
„ Example: new problem announcement => tell the
agents that can be potentially interested in solving it
‰ Agents do not need to be continuously
checking the blackboard

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
Blackboard systems summary (I)

„ Positive aspects
‰ Flexible mechanism that allows
communication/cooperation
„ E.g. n blackboards
‰ Independent of cooperation
strategy
‰ It does not place any
restriction on the agents’
internal architecture

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
Blackboard systems summary (II)

„ Negative aspects
‰ Centralised structure
‰ System bottleneck
‰ Everyone has to write
info on the blackboard
‰ Everyone has to read
from the blackboard
‰ Single point of failure

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com

Message passing

Agent 1 message Agent 2

(Sender) (Receiver)

„ Information is passed from one agent to

another. The nature of this information can be
very varied. Speech acts provide one way to
describe this variety

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
Speech Acts
„ Most treatments of communication in
multi-agent systems borrow their
inspiration from speech act theory
„ Speech act theories are pragmatic
theories of language, i.e., theories of
language use: they attempt to account for
how language is used by people every
day to achieve their goals and intentions
„ The origin of speech act theories is
usually traced to Austin’s 1962 book,
How to Do Things with Words

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
Speech Act Theory
„ Austin noticed that some utterances are like
‘physical actions’ that appear to change the state
of the world
„ Paradigmatic examples would be:
‰ Declaring war
‰ ‘I now pronounce you man and wife’
„ But more generally, everything we utter is
uttered with the intention of satisfying some goal
„ A theory of how utterances are used to achieve
intentions is a speech act theory

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com

Speech Acts

„ A speech act is an act of communication

„ Speech does not imply any particular
communication media
„ There are various types of speech act
„ By using the various types of speech act,
agents can interact effectively

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
Types of speech act

„ inform other agents about some data

„ query others about their current situation
„ answer questions
„ request others to act
„ promise to do something
„ offer deals
„ acknowledge offers and requests
„ …

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
Speech acts types – Searle (I)
„ Representative
‰ Communicate some state of affairs

„ Informing, asserting, claiming, describing,

…
„ Commissive
‰ Commit the speaker to some future

course of action
„ Promising, agreeing, threatening, inviting,
offering, swearing, volunteering, ...

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
Speech acts types – Searle (II)
‰ Directive
„ Intention to get the receiver to carry out some action
‰ Requesting, commanding, daring, asking, begging,
forbidding, advising, ...
‰ Declaration
„ Bring about a state of affairs
‰ Arresting, marrying, declaring, ...
‰ Expressive
„ Indicate the speaker’s psychological state or mental
attitude
‰ Thanking, greeting, congratulating, apologizing, ...

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
Speech Act components

„ In general, a speech act can be seen to have

two components:
‰ a performative verb:
(e.g., request, inform, promise, … )
‰ propositional content:
(e.g., “the door is closed”)

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
Communicative Acts
„ Is the door open? „ query
„ Open the door (for me) „ request
„ OK! I’ll open the door „ agree
„ The door is open „ inform
„ I am unable to open the door „ failure
„ I don’t want to open the door „ refuse
„ Tell me when the door becomes open „ subscribe
„ Does anyone want to open the door? „ cfp
„ I can open the door for you... at a price „ propose
„ Door? What’s that? Don’t
understand... „ not-understood

Same content of the message, different meanings

(depending on the used performative)

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com

Conversations

„ Speech acts are rarely carried out in isolation

„ Speech acts must be considered in terms of the other
speech acts of the conversation
„ Speech acts therefore can have different meanings
based on the previous discourse
„ Establishing meaning is thus more than understanding
individual utterances

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com

Communication Standards
„ Agents must understand each other
even if running on different machines
and/or different operating systems
„ Standards
‰ allow different groups to write cooperating
agents
‰ help abstract out communication, by
defining high-level general languages and
protocols

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
FIPA

„ Foundation for Intelligent Physical Agents

„ Beginning (1996): stand-alone non-profit
organisation
„ Now: IEEE Computer Society standards committee
„ Mission: develop and promote agent standards
‰ MAS architecture
‰ Agent communication
language (FIPA-ACL)
‰ Communication protocols

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com

Components of a FIPA-ACL message

ACL message
(inform Message content
Begin message structure :sender agent1 expression
:receiverhpl-auction-server
:content
Communicative act type (price (bid good02) 150)
:in-reply-to round-4 Parameter
:reply-withbid04 expression
:language sl
Message parameters :ontology hpl-auction
)

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
Parameters in a FIPA ACL message
„ :sender - who sends the message
„ :receiver - who is the recipient of the message
„ :content - content of the message
„ :reply-with - identifier of the message
„ :reply-by - deadline for replying the message
„ :in-reply-to - identifier of the message being replied
„ :language – language in which the content is written
„ :ontology - ontology used to represent the domain
concepts
„ :protocol - communication protocol to be followed
„ :conversation-id - identifier of conversation

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
FIPA-ACL performatives

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
Inform
„ Content: statement
„ The sender informs the receiver that a
given proposition is true
„ The sending agent:
‰ holds that some proposition is true
‰ intends that the receiving agent also comes
to believe that the proposition is true
‰ does not already believe that the receiver
has any knowledge of the truth of the
proposition

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
Inform example

(inform
:sender (agent-identifier :name i)
:receiver (agent-identifier :name j)
:content ”door( now, open )"
:language Prolog)
)

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
Request
„ Content: action
„ The sender requests the receiver to
perform some action
„ The sending agent:
‰ intends the action content to be performed
‰ believes recipient is capable of performing
this action
‰ does not believe that receiver already
intends to perform action

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
Request example
(request
:sender (agent-identifier :name i)
:receiver (agent-identifier :name j)
:content (action (agent-identifier :name j)
open_the_door)
:language fipa-sl
)

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
Communication protocols
„ There are many situations in which agents
engaged in a dialogue with a certain purpose
exchange the same sequence of messages
‰ When an agent makes a question to another
‰ When an agent requests a service from another
‰ When an agent looks for help from other agents
„ To ease the management of this typical
message interchanges we can use
predefined protocols

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
FIPA-Request
protocol

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
FIPA-Query
protocol

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com

FIPA-
Contract Net
protocol

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
FIPA Contract Net

Agent 1) cfp
3) propose agent
5) reject-proposal

1) cfp

6) inform 2) propose
4) propose 1) cfp
5) reject-proposal
1) cfp 5) accept-proposal agent

agent agent

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
Readings for this week

„ FIPA web page http://www.fipa.org

‰ FIPA specifications

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com

Download Useful Materials @ www.rejinpaul.com