Informed Search and Exploration

Sections 3.5 and 3.6

Ch. 03 – p.1/51
Outline

Best-first search
A∗ search
Heuristics, pattern databases
IDA∗ search
(Recursive Best-First Search (RBFS), MA∗ and
SMA∗ search)

Ch. 03 – p.2/51
Best-first search

Idea: use an evaluation function for each node

The evaluation function is an estimate of “desirability”
Expand the most desirable unexpanded node
The desirability function comes from domain knowledge
Implementation:
The frontier is a queue sorted in decreasing order of desirability
Special cases:
greedy best first search
A∗ search

Ch. 03 – p.3/51
Romania with step costs in km

Oradea
71
Neamt

Zerind 87
75 151
Iasi
Arad
140
92
Sibiu Fagaras
99
118
Vaslui
80
Rimnicu Vilcea
Timisoara
142
111 Pitesti 211
Lugoj 97
70 98
85 Hirsova
Mehadia 146 101 Urziceni
75 138 86
Bucharest
Dobreta 120
90
Craiova Eforie
Giurgiu

Sample straight line distances to Bucharest:

Arad: 366, Bucharest: 0, Sibiu: 253, Timisoara: 329.

Ch. 03 – p.4/51
Greedy best-first search

Evaluation function h(n) (heuristic) = estimate of cost from n to

the closest goal
E.g., hSLD (n) = straight-line distance from n to Bucharest
Greedy best-first search expands the node that appears to be
closest to goal

Ch. 03 – p.5/51
Greedy best-first search example

Arad

Ch. 03 – p.6/51
After expanding Arad

Arad

140 75
118

Sibiu Timisoara Zerind

253 329 374

Ch. 03 – p.7/51
After expanding Sibiu

Arad

140 75
118

Sibiu Timisoara Zerind

80 329 374
140
99 151

Arad Fagaras Oradea Rimnicu V.

366 176 380 193

Ch. 03 – p.8/51
After expanding Fagaras

Arad

140 75
118

Sibiu Timisoara Zerind

80 329 374
140
99 151

Arad Fagaras Oradea Rimnicu V.

366 380 193

99 211

Sibiu Bucharest

253 0
The goal Bucharest is found with a cost of 450. However, there is a
better solution through Pitesti (h = 417).

Ch. 03 – p.9/51
Properties of greedy best-first search

Complete No — can get stuck in loops

For example, going from Iasi to Fagaras,
Iasi → Neamt → Iasi → Neamt → . . .
Complete in finite space with repeated-state checking
Time O(bm ), but a good heuristic can give dramatic improvement
(more later)
Space O(bm )—keeps all nodes in memory
Optimal No
(For example, the cost of the path found in the previous slide was
450. The path Arad, Sibiu, Rimnicu Vilcea, Pitesti, Bucharest has
a cost of 140+80+97+101 = 418.)

Ch. 03 – p.10/51
A∗ search

Idea: avoid expanding paths that are already expensive

Evaluation function f (n) = g(n) + h(n)
g(n) = exact cost so far to reach n
h(n) = estimated cost to goal from n
f (n) = estimated total cost of path through n to goal
A∗ search uses an admissible heuristic
i.e., h(n) ≤ h∗ (n) where h∗ (n) is the true cost from n.
(Also require h(n) ≥ 0, so h(G) = 0 for any goal G.)
Straight line distance (hSLD (n)) is an admissible heuristic
because never overestimates the actual road distance.

Ch. 03 – p.11/51
A∗ search example

Arad

366=0+366

Ch. 03 – p.12/51
After expanding Arad

Arad

Sibiu Timisoara Zerind

393=140+253 447=118+329 449=75+374

Ch. 03 – p.13/51
After expanding Sibiu

Arad

Sibiu Timisoara Zerind

447=118+329 449=75+374

Arad Fagaras Oradea Rimnicu V.

646=280+366 415=239+176 671=291+380 413=220+193

Ch. 03 – p.14/51
After expanding Rimnicu Vilcea

Arad

Sibiu Timisoara Zerind

447=118+329 449=75+374

Arad Fagaras Oradea Rimnicu V.

646=280+366 415=239+176 671=291+380

Craiova Pitesti Sibiu

526=366+160 417=317+100 553=300+253

Ch. 03 – p.15/51
After expanding Fagaras

Arad

Sibiu Timisoara Zerind

447=118+329 449=75+374

Arad Fagaras Oradea Rimnicu V.

646=280+366 671=291+380

Sibiu Bucharest Craiova Pitesti Sibiu

591=338+253 450=450+0 526=366+160 417=317+100 553=300+253

Remember that the goal test is performed when a node is selected for
expansion, not when it is generated.
Ch. 03 – p.16/51
After expanding Pitesti

Arad

Sibiu Timisoara Zerind

447=118+329 449=75+374

Arad Fagaras Oradea Rimnicu V.

646=280+366 671=291+380

Sibiu Bucharest Craiova Pitesti Sibiu

591=338+253 450=450+0 526=366+160 553=300+253

Bucharest Craiova Rimnicu V.

418=418+0 615=455+160 607=414+193

Ch. 03 – p.17/51
Optimality of A∗ for trees

Theorem: A∗ search is optimal.

Note that, A∗ search uses an admissible heuristic by definition.
Suppose some suboptimal goal G2 has been generated and is in the
queue. Let n be an unexpanded node on a shortest path to an optimal
goal G1 .

Ch. 03 – p.18/51
 Optimality of A∗ for trees (cont’d)

start

G1
G2

f (n) = g(n) + h(n) by definition

f (G1 ) = g(G1 ) because h is 0 at a goal
f (G2 ) = g(G2 ) because h is 0 at a goal
f (n) ≤ f (G1 ) because h is admissible (never overestimates)
f (G1 ) < f (G2 ) because G2 is suboptimal
f (n) < f (G2 ) combine the above two
Since f (n) < f (G2 ), A∗ will never select G2 for expansion.

Ch. 03 – p.19/51
Progress of A∗ with an inconsistent heuristic

I
g=0, h=6, f=6

2 2

g=2, h=5, f=7 g=2, h=2, f=4

1 2
g=3, h=1, f=4 g=4, h=1, f=5

g=7, h=0, f=7 g=8, h=0, f=8

G

Note that h is admissible, it never overestimates.

Ch. 03 – p.20/51
Progress of A∗ with an inconsistent heuristic

I
g=0, h=6, f=6

2 2

g=2, h=5, f=7 g=2, h=2, f=4

1 2
g=3, h=1, f=4 g=4, h=1, f=5

g=7, h=0, f=7 g=8, h=0, f=8

G
The root node was expanded. Note that f decreased from 6 to 4.

Ch. 03 – p.21/51
Progress of A∗ with an inconsistent heuristic

I
g=0, h=6, f=6

2 2

g=2, h=5, f=7 g=2, h=2, f=4

1 2
g=3, h=1, f=4 g=4, h=1, f=5

g=7, h=0, f=7 g=8, h=0, f=8

G
The suboptimal path is being pursued.

Ch. 03 – p.22/51
Progress of A∗ with an inconsistent heuristic

I
g=0, h=6, f=6

2 2

g=2, h=5, f=7 g=2, h=2, f=4

1 2
g=3, h=1, f=4 g=4, h=1, f=5

g=7, h=0, f=7 g=8, h=0, f=8

G
Goal found, but we cannot stop until it is selected for
expansion.

Ch. 03 – p.23/51
Progress of A∗ with an inconsistent heuristic

I
g=0, h=6, f=6

2 2

g=2, h=5, f=7 g=2, h=2, f=4

1 2
g=3, h=1, f=4 g=4, h=1, f=5

g=7, h=0, f=7 g=8, h=0, f=8

G
The node with f = 7 is selected for expansion.

Ch. 03 – p.24/51
Progress of A∗ with an inconsistent heuristic

I
g=0, h=6, f=6

2 2

g=2, h=5, f=7 g=2, h=2, f=4

1 2
g=3, h=1, f=4 g=4, h=1, f=5

g=7, h=0, f=7 g=8, h=0, f=8

G
The optimal path to the goal is found.

Ch. 03 – p.25/51
Consistency

A heuristic is consistent if n
h(n) ≤ c(n, a, n′ ) + h(n′ )
If h is consistent, we have c(n, a, n’) h(n)

f (n′ ) = g(n′ ) + h(n′ ) n’

= g(n) + c(n, a, n′ ) + h(n′ ) h(n’) G
≥ g(n) + h(n)
= f (n)

I.e., f (n) is nondecreasing along any path.

Ch. 03 – p.26/51
Optimality of A∗ for graphs

Lemma: A∗ expands nodes in order of increasing f value

Gradually adds “f -contours” of nodes
(cf. breadth-first adds layers)
Contour i has all nodes with f = fi , where fi < fi+1
With uniform-cost search (A* search with h(n)=0) the bands are
“circular”.
With a more accurate heuristic, the bands will stretch toward the
goal and become more narrowly focused around the optimal
path.

Ch. 03 – p.27/51
F-contours

Z N

A I
380 S
F
V
400
T R
L P

H
M U
420 B
D
C E
G
Ch. 03 – p.28/51
Performance of A∗

The absolute error of a heuristic is defined as

∆ ≡ h∗ − h
The relative
∗
error of a heuristic is defined as
ǫ ≡ h h−h
∗

Complexity with constant step costs: O(bǫd )

Problem: there can be exponentially many states with f (n) < C ∗
even if the absolute error is bounded by a constant

Ch. 03 – p.29/51
Properties of A∗

Complete Yes, unless there are infinitely many nodes with

f ≤ f (G)
Time Exponential in
(relative error in h × length of solution)
Space Keeps all nodes in memory
Optimal Yes—cannot expand fi+1 until fi is finished
A∗ expands all nodes with f (n) < C ∗
A∗ expands some nodes with f (n) = C ∗
A∗ expands no nodes with f (n) > C ∗

Ch. 03 – p.30/51
Admissible heuristics

E.g., for the 8-puzzle:

h1 (n) = number of misplaced tiles
h2 (n) = total Manhattan distance
(i.e., no. of squares from desired location of each tile)
7 2 4 1 2

5 6 3 4 5

8 3 1 6 7 8

Start State Goal State

h1 (S) = ??
h2 (S) = ??

Ch. 03 – p.31/51
Dominance

If h2 (n) ≥ h1 (n) for all n (both admissible)

then h2 dominates h1 and is better for search
Typical search costs:
d = 14 IDS = 3,473,941 nodes
A∗ (h1 ) = 539 nodes
A∗ (h2 ) = 113 nodes
d = 24 IDS ≈ 54,000,000,000 nodes
A∗ (h1 ) = 39,135 nodes
A∗ (h2 ) = 1,641 nodes

Ch. 03 – p.32/51
Effect of Heuristic on Performance

The effect is characterized by the effective branching factor (b∗ )

If the total number of nodes generated by A∗ is N and
the solution depth is d,
then b is branching factor of a uniform tree, such that
N + 1 = 1 + b + (b)2 + + (b)d
A well designed heuristic has a b close to 1.

Ch. 03 – p.33/51
Using relaxed problems to find heuristics

Admissible heuristics can be derived from the exact solution cost

of a relaxed version of the problem
If the rules of the 8-puzzle are relaxed so that a tile can move
anywhere, then h1 (n) gives the shortest solution
If the rules are relaxed so that a tile can move to any adjacent
square, then h2 (n) gives the shortest solution
Key point: the optimal solution cost of a relaxed problem is no
greater than the optimal solution cost of the real problem

Ch. 03 – p.34/51
Relaxed problems (cont’d)

Well-known example: travelling salesperson problem (TSP)

Find the shortest tour visiting all cities exactly once

Minimum spanning tree can be computed in O(n2 )

and is a lower bound on the shortest (open) tour

Ch. 03 – p.35/51
Pattern databases

Admissible heuristics can also be generated from the solution

cost of sub- problems.
For example, in the 8-puzzle problem a sub-problem of getting
the tiles 2, 4, 6, and 8 into position is a lower bound on solving
the complete problem.
Pattern databases store the solution costs for all the sub-problem
instances.
The choice of sub-problem is flexible:
for the 8-puzzle a subproblem for 2,4,6,8 or 1,2,3,4 or 5,6,7,8, . .
. could be created.

Ch. 03 – p.36/51
Iterative Deepening A* (IDA*)

Idea: perform iterations of DFS. The cutoff is defined based on

the f -cost rather than the depth of a node.
Each iteration expands all nodes inside the contour for the
current f -cost, peeping over the contour to find out where the
contour lies.

Ch. 03 – p.37/51
Iterative Deepening A* (IDA*)

function IDA* (problem)

returns a solution sequence

inputs: problem, a problem

local variables:
f-limit, the current f -C OST limit
root, a node

root ← M AKE -N ODE (I NITIAL -S TATE[problem])

f-limit ← f -C OST(root)
loop do
solution, f-limit ← DFS-C ONTOUR(root, f-limit)
if solution is non-null then return solution
if f-limit = ∞ then return failure

Ch. 03 – p.38/51
Iterative Deepening A* (IDA*)

function DFS-C ONTOUR (node, f-limit)

returns a solution sequence and a new f -C OST limit

inputs: node, a node

f-limit, the current f -C OST limit
local variables:
next-f, the f -C OST limit for the next contour, initally ∞

if f -C OST[node] > f-limit then return null, f -C OST[node]

if G OAL -T EST[problem](S TATE[node]) then return node, f-limit
for each node s in S UCCESSORS(node) do
solution, new-f ← DFS-C ONTOUR(s, f-limit)
if solution is non-null then return solution, f-limit
next-f ← M IN(next-f, new-f)
return null, next-f

Ch. 03 – p.39/51
How would IDA* proceed?

Arad h=366

140 75
118

Sibiu h=253 f=393 Timisoara Zerind

f=118+329=447 f=75+374=449
140 80
99 151
Arad Fagaras Oradea Rimnicu V.
f=415 f=413
f=280+366=646 f=291+380=671

Sibiu Bucharest Craiova Pitesti Sibiu

f=417
f=338+253=591 f=450+0=450 f=366+160=526 f=300+253=553
f−limits:
366 (Arad), 393 (Sibiu),
413 (RV), 417 (Pitesti) Bucharest Craiova Rimnicu V.
418 (Bucharest, goal)
f=418+0=418 f=455+160=615 f=414+193=607

The blue nodes are the ones A* expanded. For IDA∗ ,

they define the new f-limit.

Ch. 03 – p.40/51
Properties of IDA*

Complete Yes, similar to A*.

Time Depends strongly on the number of different
values that the heuristic value can take on.
8-puzzle: few values, good performance
TSP: the heuristic value is different for every state.
Each contour only includes one more state than the
previous contour. If A* expands N nodes, IDA*
expands 1 + 2 + . . . + N = O(N 2 ) nodes.
Space It is DFS, it only requires space proportional
to the longest path it explores. If δ is the smallest
operator cost, and f ∗ is the optimal solution cost,
then IDA* will require bf ∗ /δ nodes.
Optimal Yes, similar to A*
Ch. 03 – p.41/51
Recursive Best-First Search (RBFS)

Idea: mimic the operation of standard best-first

search, but use only linear space
Runs similar to recursive depth-first search, but
rather than continuing indefinitely down the current
path, it uses the f-limit variable to keep track of the
best alternative path available from any ancestor of
the current node.
If the current node exceeds this limit, the recursion
unwinds back to the alternative path. As the
recursion unwinds, RBFS replaces the f-value of
each node along the path with the best f-value of its
children. In this way, it can decide whether it’s worth
reexpanding a forgotten subtree.
Ch. 03 – p.42/51
RBFS Algorithm

function R ECURSIVE -B EST-F IRST-S EARCH (problem)

returns a solution or failure
return RBFS(problem, M AKE -N ODE(problem.I NITIAL -S TATE), ∞)

Ch. 03 – p.43/51
RBFS Algorithm (cont’d)

function RBFS (problem, node, f-limit)

returns a solution or failure and a new f -cost limit
if problem.G OAL -T EST(node.S TATE) then return S OLUTION(node)
successors ← [ ]
for each action in problem.ACTIONS(node.S TATE) do
add C HILD -N ODE(problem, node, action) into successors
if successors is empty then return failure, ∞
for each s in successors do
/* update f with value from previous search, if any */
s.f ← max (s.g + s.h, node.f ))
loop do
best ← the lowest f -value in successors
if best.f > f-limit then return failure, best.f
alternative ← the second lowest f-value among successors
result, best.f ← RBFS (problem, best, min(f-limit,alternative))
if result 6= failure then return result

Ch. 03 – p.44/51
Progress of RBFS

(a) After expanding Arad, Sibiu, Rimnicu Vilcea Arad 366

447
Sibiu Timisoara Zerind
393
447 449
415
Arad Fagaras Oradea Rimnicu Vilcea
413
646 415 526

Craiova Pitesti Sibiu

526 417 553

(b) After unwinding back to Sibiu Arad

and expanding Fagaras 366

447
Sibiu Timisoara Zerind
393 447 449
417
Arad Fagaras Oradea Rimnicu Vilcea
646 415 526 413 417

Sibiu Bucharest
591 450

(c) After switching back to Rimnicu Vilcea Arad

and expanding Pitesti 366

447
Sibiu Timisoara Zerind
393 447 449
447
Arad Fagaras Oradea Rimnicu Vilcea
646 526 417
415 450
447
Craiova Pitesti Sibiu
526 417 553

Bucharest Craiova Rimnicu Vilcea

418 615 607

Ch. 03 – p.45/51
Progress of RBFS (cont’d)

Stage (a): The path via Rimnicu Vilcea is followed

until the current best leaf (Pitesti) has a value that is
worse than the best alternative path (Fagaras).
Stage (b): The recursion unwinds and the best leaf
value of the forgotten subtree (417) is backed up to
Rimnicu Vilcea; then Fagaras is expanded, revealing
a best value of 450.
Stage (c): The recursion unwinds and the best value
of the of the forgotten subtree (450) is backed up to
Fagaras; then Rimnicu Vilcea is expanded. This
time, because the best alternative path through
Timisoara costs at least 447, the expansion
continues to Bucharest.
Ch. 03 – p.46/51
Properties of RBFS

Complete Yes, similar to A*.

Time The time complexity is difficult to characterize:
it depends both on the accuracy of the heuristic
function and on how often the best path changes as
nodes are expanded. Each mind change
corresponds to an iteration of IDA∗ , and could
require many reexpansions of forgotten nodes to
recreate the best path and extend it one more node.
RBFS is somewhat more efficient than IDA∗ , but still
suffers from excessive node regeneration.

Ch. 03 – p.47/51
Properties of RBFS (cont’d)

Space IDA∗ and RBFS suffer from using too little

memory. Between iterations, IDA∗ retains only a
single number: the current f-cost limit. RBFS retains
more information in memory, but only uses O(bd)
memory. Even if more memory is available, RBFS
has no way to make use of it.
Optimal Yes, similar to A*.

Ch. 03 – p.48/51
MA* and SMA*

Idea: use all the available memory

IDA* remembers only the current f -cost limit
RBFS uses linear space
Proceeds just like A*, expanding the best leaf until
the memory is full. When the memory if full, drops
the worst leaf node.

Ch. 03 – p.49/51
Summary

The evaluation function for a node n is:

f (n) = g(n) + h(n)
If only g(n) is used, we get uniform-cost search
If only h(n) is used, we get greedy best-first search
If both g(n) and h(n) are used we get best-first
search
If both g(n) and h(n) are used with an admissible
heuristic we get A∗ search
A consistent heuristic is admissible but not
necessarily vice versa

Ch. 03 – p.50/51
Summary (cont’d)

Admissibility is sufficient to guarantee solution

optimality for tree search
Consistency is required to guarantee solution
optimality for graph search
If an admissible but not consistent heuristic is used
for graph search, we need to adjust path costs when
a node is rediscovered
Heuristic search usually brings dramatic
improvement over uninformed search
Keep in mind that the f-contours might still contain
an exponential number of nodes

Ch. 03 – p.51/51