CMP 446 BASIC SEARCH
STRATEGIES
DR. S.A. AREKETE
COURSE CONTENTS
• Problem spaces (states, goals and operators), problem solving by
search, Factored representation (factoring state into variables),
Uninformed search (breadth-first, depth-first, depth-first with
iterative deepening), Heuristics and informed search (hill-
climbing, generic best-first, A*), Space and time efficiency of
search, Two-player games (Introduction to mini-max search),
Constraint satisfaction (backtracking and local search methods),
Review of propositional and predicate logic (cross-reference
DS/Basic Logic), Resolution and theorem proving (propositional
logic only), Forward chaining, backward chaining, Review of
probabilistic reasoning, Bayes theorem
Work Plan
Week# Topics Activity/Assignments
1 Problem spaces (states, goals and operators), problem solving by search, Factored
representation (factoring state into variables)
2-4 Uninformed search (Random Search, Generate and test breadth-first, depth-first, depth-first Assignment 1
with iterative deepening)
5-6 Heuristics and informed search (hill-climbing, generic best-first, A*), Space and time
efficiency of search, Two-player games (Introduction to mini-max search)
7 Continuous Assessment Test 1
8 Student Presentations
9 Space and time efficiency of search, Two-player games (Introduction to mini-max search), Assignment 2
Constraint satisfaction (backtracking and local search methods)
10 Constraint satisfaction (backtracking and local search methods), Review of propositional and
predicate logic (cross-reference DS/Basic Logic),
11-12 Resolution and theorem proving (propositional logic only), Forward chaining, backward Assignment 3
chaining, Review of probabilistic reasoning, Bayes theorem
13 Revision
Blind Search Algorithms
• Blind search algorithms are explained in Section 2.3. A blind, or
uninformed, search algorithm is one that makes no use of
problem domain knowledge.
• Suppose, for example, that you are finding your way through a
maze.
• In a blind search you might always choose the far left route
• regardless of any other alternatives.
Generate-and-Test Paradigm
• A straightforward way to solve
a problem is to propose
possible solutions and then to
check whether each proposal
constitutes a solution.
• This is referred to as the
generate-and-test paradigm;
the approach is illustrated with
the n-Queens Problem, shown
in Figure 2.3.
Generate-and-Test Paradigm..
• The n-Queens Problem involves
placing n Queens on an n × n
chessboard so that
i. no two Queens are attacking.
ii. That is, no two Queens should
occupy the same row, column,
or diagonal on the board.
• These conditions are referred to as
the constraints of the problem.
• The proposed solutions in Figure
2.4(a) – (c) violate various
constraints of the problem.
• A solution to the 4-Queens
problem appears in Figure 2.4(d).
Generate-and-Test Paradigm..
• In this problem four queens need to
be placed on a 4 × 4 chessboard.
• There are a total of 16C4, or 1820,
ways to accomplish this.
• As Figure 2.4 illustrates, many of
these proposed solutions violate
one or more of the problem
constraints.
• However, if a solution is not to be
missed, then a reliable generator
must propose every subset of size
four that satisfies the problem
constraints.
Generate-and-Test Paradigm..
• More generally, a reliable generator is
complete if it proposes every possible
solution.
• Furthermore, if a proposed solution is
rejected, it should not be proposed again (in
fact, even successful proposals should be
made only once).
• In other words, a good generator should be
nonredundant.
• Finally, recall that there are 1820 ways to
place four Queens on a 4 × 4 chessboard.
Generate-and-Test Paradigm..
• The generator would be more efficient
if it did not propose solutions that
would obviously fail.
• Figure 2.4(a) shows an example in
which all problem constraints are
violated.
• We say that a generator is informed if it
possesses some information that allows
it to limit its proposals.
Generate-and-Test Paradigm..
• A procedure for the generate-and-test paradigm would look like the
following:
{While no solution is found and more candidates remain
[Generate a possible solution
Test if all problem constraints are satisfied]
End While}
If a solution has been found, announce success and output it.
Else announce no solution found.
Generate-and-Test Paradigm..
Example 2.1: Generate and Test for Prime Numbers
• Suppose you must determine whether a given number between 3 and 100,
inclusive, is a prime.
• Recall that an integer N ≥ 2 is prime if its only factors are 1 and itself.
• So 17 and 23 are prime whereas 33 is not, because it is the product of 3 and
11.
• Assume that you must solve this problem without benefit of a computer or
pocket calculator.
• Your first attempt at a solution, using the generate-and-test approach,
might look like the following pseudocode:
Generate-and-Test Paradigm..
Example 2.1: Generate and Test for Prime Numbers..
Generate-and-Test Paradigm..
Example 2.1: Generate and Test for Prime Numbers..
• If possible factor equals Number,
• Then return Number is prime
• If Number is equal to 85, then the Test fails for possible factors
of 2, 3, and 4.
• However, 85/5 yields 17, so we can declare that 85 is not prime.
If number is equal to 37, then we exit the While Loop with
possible factors equal to 37 as well and return that 37 is prime.
Generate-and-Test Paradigm..
Example 2.1: Generate and Test for Prime Numbers..
• A more informed generator checks only possible factors up to
floor (square root (Number)).
• Recall that floor of a number is the largest integer ≤ that
number; for example floor (3.14) = 3, floor (2) = 2, and floor
(−5.17 ) = −6.
• For Number equal to 37 in the previous example, the informed
generator returns that 37 is prime after only checking 2, 3, 4, 5,
and 6 as possible factors.
• More informed generators lead to vast savings in time and
complexity.
The Greedy Algorithm
• The greedy algorithm is
another classic search method,
which also operates by first
dividing a problem into stages.
• A greedy algorithm always
contains an objective function
that must be optimized (i.e.,
either maximized or minimized).
• Typical objective functions
might be distance travelled, cost
expended, or time elapsed.
• Figure 2.13 (a) represents a map
of North-Eastern China.
The Greedy Algorithm..
• Suppose that a salesperson is starting
from Chengdu and wants to find the
shortest path to Haerbin that passes
only through the cities that are
circled: Chengdu (V1), Beijing (V2),
Haerbin (V3), Hangzhou (V4), and
Xi’an (V5).
• Distances between these five cities
are shown in kilometres.
• In Stage 1, a greedy approach to this
problem proceeds from Chengdu to
Xi’an because, at a distance of only
606 km, Xi’an is the closest city.
• Subsequent steps in the algorithm are
explained in Figure 2.14.
The Greedy Algorithm..
• In Stage 1, the path from V1 to
V5 is taken because Xi’an is the
closest city to Chengdu.
• 2. We may consider paths that
go through vertices only if they
have been visited previously.
• In Stage 2, the next path
generated goes directly from V1
to V2; its cost (distance) is 1518
km.
• This direct route is less
expensive than the path going
through V5, which would cost
606 + 914 = 1520 km.
The Greedy Algorithm..
• 3. The least expensive path from
V1 to V3 is constructed from the
least expensive path from V1 to
an intermediate node (Vi) plus
the least costly path from Vi to
V3. Here i equals V2; the least
costly path from V1 to V3
passes through V2 and has a
cost of 1518 + 1061 = 2579 km.
• The direct path from V1 to V4,
however, is less costly (1539).
• We are going to V4 (Hangzhou).
The Greedy Algorithm..
• Stage 4: We are searching for the next
least expensive path from V1 to
anywhere.
• We already have the least expensive
path from V1 to V5, at a cost of 606
km. The second least expensive path
is the direct one from V1 to V2, with a
cost of 1518 km.
• The direct path from V1 to V4 (1539
km) is less costly than either the one
that first passes through V5 (606 +
1150 = 1756 km) or V2 (1518 + 1134 =
2652 km).
• Hence, the next least costly path is
the one to V3 (2579).
• There are several possibilities:
The Greedy Algorithm..
• V1 to V5: cost = 606; then V5
to V2: cost = 914; the cost of
going from V1 to V2 through
V5 is 1520.
• You then need to get from V2
to V3; this distance is 1061.
The path from V1 to V3 that
passes through V5 and V2 has
a total cost of 1520 + 1061 =
2581.
The Greedy Algorithm..
• V1 to V2: cost = 1518; then
V2 to V3: cost = 1061; total
cost = 2579 km.
• V1 to V4: cost = 1539; then
V4 to V3: cost = 1822; this
total cost is 3361 km.
• We are taking the path from
V1 to V3, which first passes
through V2. Its total cost is
2579 km.
The Greedy Algorithm..
The Greedy Algorithm..
The Greedy Algorithm..
The Greedy Algorithm..
• The least costly path from V1 to V3 passes through V2 and has a
cost of 1518 + 1061.
• The direct path from V1 to V4 is less costly.
The Greedy Algorithm..
The Traveling Salesperson Problem
• In the Traveling Salesperson Problem (TSP), you are given n
vertices in a weighted graph (i.e., a graph with costs on the
edges).
• You must find the shortest circuit that starts at some vertex Vi,
passes through each vertex in the graph once and only once, and
then returns to Vi.
• The previous example concerning five cities from China is
employed.
• Suppose that our salesperson resides in Xi’an, and must
therefore visit each of Chengdu, Beijing, Hangzhou, and Haerbin,
in some order, and then return home to Xi’an.
The Traveling Salesperson Problem..
• The least expensive such circuit is
sought.
• A greedy-based solution to the TSP
always visits the closest city next, as
shown in Figure 2.15.
• The greedy algorithm visits Chengdu,
Beijing, Haerbin, Hangzhou, and then
finally returns to Xi’an.
• The cost of this circuit is 606 + 1518 +
1061 + 1822 + 1150 = 6157 km.
The Traveling Salesperson Problem..
• If the salesperson visits Beijing, Haerbin,
Hangzhou, Chengdu, and then returns
to Xi’an, the total accrued cost is 914 +
1061 + 1822 + 1539 + 606 = 5942 km.
• Clearly, the greedy algorithm has failed
to find an optimal route for the TSP.
• A variation on breadth first search, in
which nodes are explored in terms of
nondecreasing cost, is branch and
bound.
• Branch and bound is also referred to as
uniform cost search and is effective for
TSP.
Depth First Search
• Depth first search (dfs), as the
name suggests, attempts to
plunge as deeply into a tree as
quickly as possible.
• Whenever the search can make
a choice, it selects the far left
(or far right) branch (though it
usually selects the far left
branch).
• As an example of dfs, consider
the tree in Figure 2.16.
Depth First Search
• We note that tree traversal
algorithms will often “visit” a
node several times, for
example, in Figure 2.16 a dfs
encounters nodes in the
order:
• A,B,D,B,E,B,A,C,F,C,G.
• It is traditional to announce
only the first visit as shown in
the caption (A,B,D,E,C,F,G.).
Depth First Search
Depth First Search
Solving a 3-Puzzle Using dfs
• To find a dfs solution for the
3-puzzle, start by defining the
start state and goal state, as
shown in Figure 2.19.
Solving a 3-Puzzle Using dfs..
• In Figure 2.19(a) the 1 tile is free to
move south one square and the 2 tile
can move east.
• It is, however, better to assume that
the blank moves.
• In Figure 2.19(a), the blank can move
north or west.
• Four operators can change the state of
the puzzle—the blank can move
north, south, east, or west.
• Because possible moves must be tried
in this order, we will represent moves
by an arrow pointing in the
appropriate direction: N, S, E, and W.
Solving a 3-Puzzle Using DFS..
• This order is arbitrary, though
some order must be specified.
• A dfs is employed to solve this
instance of the 3-puzzle.
• The results of this search are
shown in Figure 2.20.
Solving a 3-Puzzle Using dfs..
• Each step in the search applies
the first operator from the set
{N, S, E, W}.
• No effort is expended trying to
determine which move arrives at
a solution fastest—in this sense,
the search is blind.
• However, the search avoids
repeated states.
Solving a 3-Puzzle Using dfs..
• Starting at the root and applying
N and then S, you arrive at the
state marked with a * in Figure
2.20.
• As we shall see in Section 2.4,
avoiding repeated states is an
essential feature of many
efficient search algorithms.
Assignment
• Use the DFS to solve the 8-puzzle
Breadth First Search
• A second blind search approach
is provided by breadth first
search (bfs).
• In bfs, nodes are visited level by
level from the top of the tree to
the bottom, in left to right
fashion (or right to left, though
left to right is more traditional).
• All nodes at level i are visited
before any nodes at level i+1
are encountered.
• Figure 2.21 shows a bfs.
Breadth First Search
Solving a 3-Puzzle Using bfs
• To find a bfs solution for the 3-
puzzle we will once again solve
the instance of the puzzle
shown in Figure 2.19.
• This time bfs will be employed.
• See Figure 2.22.
• Note that a solution is found at
depth 4 (where, as is usual, the
root is considered to be at
depth 0), which means that four
moves of the blank are required
to reach the goal.
Implementing a Depth First Search
Solution
• The various search algorithms vary greatly in the way a tree is
inspected.
• Each algorithm, however, shares one attribute in that two lists are
maintained: an open list and a closed list.
• An open list contains all nodes in the tree that are still being explored
(or expanded); the closed list contains those nodes that have already
been explored and are no longer under consideration.
• Recall that a dfs moves deeply into a search tree as quickly as
possible.
• As the code for dfs in Figure 2.25 illustrates, this is accomplished by
maintaining the open list as a stack.
• A stack is a last in, first out (LIFO) data structure.
Implementing a Depth First Search Solution
Implementing a Depth First Search Solution
• As soon as a node is visited, it moves to the front of the open
list, ensuring that its children will be generated next.
• This algorithm is applied to the search tree in Figure 2.26.
Implementing a Depth First Search Solution
Implementing a Depth First Search Solution
• As soon as a node is visited, it moves to the front of the open
list, ensuring that its children will be generated next.
• This algorithm is applied to the search tree in Figure 2.26.
• This tree is redrawn in Figure 2.27 without heuristic estimates
and node to node distances because dfs does not use these
metrics.
• Depth first search is applied to the tree in Figure 2.27.
• The results of this search (Figures 2.26 and 2.27) are given in
Figure 2.28.
Implementing a Breadth First Search Solution
• Breadth first search explores all nodes close to the root before
plunging deeper into the search tree.
• The code for bfs is shown in Figure 2.29.
• Breadth first search maintains the open list as a queue.
• A queue is a FIFO (first in, first out) data structure.
• Once a node is expanded, its children move to the rear of the
open list: hence these children are explored only after every
other node at its parent’s level has been visited.
• Figure 2.30 traces the steps of bfs on the tree in Figure 2.27.
Implementing a Breadth First Search Solution
Measuring Problem-Solving Performance
• To determine which solution works best for a particular problem,
we can compare dfs and bfs.
• Before doing so, it is helpful to provide metrics to measure these
and other search algorithms.
• Four measures are described in the following sections.
Measuring Problem-Solving Performance
1. Completeness
• A search algorithm is said to be complete when it is guaranteed to
find a solution when there is one.
• Using the generate-and-test paradigm introduced earlier in this
chapter, suppose we are trying to identify all integers x between 100
and 1000, inclusive, that are perfect cubes.
• In other words, we want to know all x with 100 ≤ x ≤ 1000 such that x
= y3 with y an integer.
• If our generator checks every integer between 100 and 1000,
inclusive, then this search would be complete.
• In fact, the results would be such that 125, 216, 343, 512, 729, and
1000 are perfect cubes.
Measuring Problem-Solving Performance
2. Optimality
• A search algorithm is said to be optimal if it provides the lowest-
cost path among all solutions.
• Figure 2.20 depicts a dfs solution for an instance of the 3-puzzle.
• A solution is found whose path length is eight.
• Figure 2.22 illustrates a bfs solution for this same instance, with
a path length of four.
• Therefore, dfs is not an optimal search strategy.
Measuring Problem-Solving Performance
3. Time Complexity
• The time complexity of a search algorithm concerns how long it
takes to find a solution.
• Time is measured in terms of the number of nodes generated (or
expanded) during the search.
Measuring Problem-Solving Performance
4. Space Complexity
• The space complexity of a search algorithm measures how much
memory is required to perform the search.
• We must determine the maximum number of nodes stored in
memory.
Measuring Problem-Solving Performance
Complexity
• Complexity in AI is expressed in terms of
three parameters:
1. The branching factor (b) of a node is
the number of branches emanating
from it (see Figure 2.31).
2. The parameter (d) measures depth of
the shallowest goal node.
3. 3. The parameter (m) measures the
maximum length of any path in the
state space.
• If every node in a search tree has a
branching factor of b, then the
branching factor of the tree equals b.
Measuring Problem-Solving Performance
Comparing dfs and bfs
• We have encountered two blind search algorithms — dfs and
bfs. Which is preferable?
• First, let’s clarify the criteria. By preferable, do we mean which
algorithm requires less work to find a path? Or do we mean
which algorithm will find a shorter path?
• In both cases, as expected, the answer is: It depends.
• Depth first search is preferred if
• the tree is deep
• the branching factor is not excessive and
• solutions occur relatively deep in the tree.
Measuring Problem-Solving Performance
Comparing dfs and bfs..
• Breadth first is preferred if
• the branching factor of the search tree is not excessive (reasonable b)
• a solution occurs at a reasonable level in the tree (d is reasonable), and
• no path is excessively deep.
Measuring Problem-Solving Performance
Comparing dfs and bfs..
• Depth first search has modest memory requirements.
• For a state space with branching factor b and maximum depth
m, dfs requires only b * m + 1 nodes, which is O(b*m).
• Backtracking is actually a variant of dfs in which only one
successor of a node is generated at a time (for example: in which
row should the third Queen be placed?).
• Backtracking requires only O(m) memory.
Measuring Problem-Solving Performance
Comparing dfs and bfs..
• Is dfs complete? Consider the search
space in Figure 2.32. As this figure
shows, dfs is not complete.
• The search might get lost in relatively
long or even infinite paths in the left
portion of the search space, while a
goal node remains unexplored in the
upper-right portion of the tree.
• Recall also that dfs is also not
optimal. (Review Figures 2.20 and
2.22.)
Measuring Problem-Solving Performance
Comparing dfs and bfs..
• If you are assured that the branching factor of a search space is
finite, then bfs is complete.
• In bfs, you first search all b children of the root, then all b2
grandchildren, and finally all bd nodes at level d.
• This latter argument should also convince the reader that bfs will
find the “shallowest” goal node first; however, this does not
necessarily mean that bfs is optimal.
• If the path cost is a nonincreasing function of the depth of the
node, then bfs is optimal.
Measuring Problem-Solving Performance
Comparing dfs and bfs..
• The time complexity (t(n)) for bfs grows exponentially.
• If the branching factor of a search tree equals b, then the root
node will have b children. Each of these b descendants will have
b children of their own.
• In fact, one will need to expand all but the last node at level d for
a total of
Measuring Problem-Solving Performance
Comparing dfs and bfs..
• Because every node that is generated must remain in memory,
the space complexity (S(n)) for bfs is also O(bd+1).
• Actually: S(n) = t(n)+1 because the root of the search tree must
also be stored.
• The harshest critique for bfs is that it requires exponential space
complexity.
• For even modest problem sizes, bfs quickly becomes infeasible.
