Chapter 3

Brute Force

Copyright 2007 Pearson Addison-Wesley. All rights reserved.

Brute Force
A straightforward approach, usually based directly on the problems statement and definitions of the concepts involved

Examples: 1. Computing an (a > 0, n a nonnegative integer)

2.

Computing n! Multiplying two matrices Searching for a key of a given value in a list

3.

4.

Brute-Force Sorting Algorithm

Selection Sort
1.

2.

3.

Scan the array to find its smallest element and swap it with the first element. Then, starting with the second element, scan the elements to its right to find the smallest among them and swap it with the second elements. Generally, on pass i (0 i n-2), find the smallest element in A[i..n-1] and swap it with A[i]:

A[0] . . . A[i-1] | A[i], . . . , A[min], . . ., A[n-1] in their final positions

Selection Sort & Bubble Sort

Selection Sort
1. 2. 3.

Start with the 1st element, scan the entire list to find its smallest element and exchange it with the 1st element Start with the 2nd element, scan the remaining list to find the the smallest among the last (N-1) elements and exchange it with the 2nd element

Example:

89 45 68 90 29 34 17 17 | 45 68 90 29 34 89 29 | 68 90 45 34 89 34 | 90 45 68 89 45 | 90 68 89 68 | 90 89 89 | 90 90

Selection Sort
Algorithm
for i 0 to N-2 do min i; N-2 i=0 N-1 c j=i+1

for j i+1 to N-1 do if (A[j] < A[min]) min j; } swap A[i] and A[min] ; }

Analysis: N-2 N-1 N-2 2 T(N) = c = c (N-1-i) = c (N-1)N / 2 in O(N )

i=0 j=i+1 i=0

Bubble Sort
Bubble Sort
In each pass, we compare adjacent elements and swap them if they are out of order until the end of the list. By doing so, the 1st pass ends up bubbling up the largest element to the last position on the list 2. The 2nd pass bubbles up the 2nd largest, and so on until, after N-1 passes, the list is sorted. Example: Pass 1 89 | 45 68 90 29 34 17 Pass 2 45 | 68 89 29 34 17 45 89 | 68 90 29 34 17 45 68 | 89 29 34 17 68 89 | 90 29 34 17 68 89 | 29 34 17 89 90 | 29 34 17 29 89 | 34 17 29 90 | 34 17 34 89 | 17 34 90 | 17 17 89 17 90 45 68 89 29 34 17 90 45 68 29 34 17 89 largest 2nd largest
1.

Bubble Sort
Algorithm
for i 0 to N-2 do N-2

N-2-i for j 0 to N-2-i do c j=i+1 if (A[j+1] < A[j]) swap A[j] and A[j+1] ; } }

i=0

Analysis: N-2 N-2-i N-2 T(N) = c = c (N-1-i) = c (N-1)N / 2 in O(N2 )

i=0 j=0 i=0

Sec 3.2 Sequential Search and String Matching

Sequential search
Compare successive elements of a given list with a search key until 1. either a match is encountered 2. or the list is exhausted without a match. 0 1 N-1 N Algorithm: SequentialSearch(A[0..N], key){ A[N] key; i 0; while (i<N and A[i] != k ) do { i = i + 1}; if i < n return i; Best case: O(1) else return 1; Analysis: Worst case: O(N) } Avg. case: O(N)

extra position to store the key

Brute-Force String Matching

pattern: a string of m characters to search for text: a (longer) string of n characters to search in problem: find a substring in the text that matches the pattern

Brute-force algorithm
Step 1 Align pattern at beginning of text Step 2 Moving from left to right, compare each character of pattern to the corresponding character in text until all characters are found to match (successful search); or a mismatch is detected Step 3 While pattern is not found and the text is not yet exhausted, realign pattern one position to the right and repeat Step 2

Examples of Brute-Force String Matching

1.

Pattern: 001011 Text: 10010101101001100101111010

2.

Pattern: happy Text: It is never too late to have a happy childhood.

Brute-force String Matching

Pattern: Text: p[0] p[1] p[m-1] m characters (m <= n) t[0] t[1] t [n-1] n characters

Purpose: to find a substring of the text that matches the pattern. More precisely, to find position k such that t[k] = p[0], t[k+1] = p[1], t[k+j]= p[j], , t[k+m-1] = p[m-1]
Brute-force algorithm: for k = 0 to n-m do j 0; while (j < m and p[j] = t[k+j]) do j j+1; if j = m return k; // matched } (assuming m << n) return 1; // ( no match) Complexity

Worst case: m (n m + 1) in (nm) Avg. case: (n + m) = (n) (for random text)

Brute-Force Polynomial Evaluation

Problem: Find the value of polynomial p(x) = anxn + an-1xn-1 + + a1x1 + a0 at a point x = x0

Brute-force algorithm
p 0.0; for i n downto 0 do { power 1; for j 1 to i do //compute xi power power x; p p + a[i] power; } return p;

Efficiency:

Polynomial Evaluation: Improvement

We can do better by evaluating from right to left:
Better brute-force algorithm p a[0] power 1 for i 1 to n do { power power x p p + a[i] power } return p Efficiency:

Closest-Pair Problem
Find the two closest points in a set of n points (in the two-dimensional Cartesian plane).

Brute-force algorithm
Compute the distance between every pair of distinct points and return the indexes of the points for which the distance is the smallest.

Closest-Pair Brute-Force Algorithm (cont.)

Efficiency:
How to make it faster?

Brute-Force Strengths and Weaknesses

Strengths

wide applicability simplicity yields reasonable algorithms for some important problems (e.g., matrix multiplication, sorting, searching, string matching)

Weaknesses

rarely yields efficient algorithms some brute-force algorithms are unacceptably slow not as constructive as some other design techniques

Exhaustive Search
A brute force solution to a problem involving search for an element with a special property, usually among combinatorial objects such as permutations, combinations, or subsets of a set. Method: generate a list of all potential solutions to the problem in a systematic manner (see algorithms in Sec. 5.4)

evaluate potential solutions one by one, disqualifying infeasible ones and, for an optimization problem, keeping track of the best one found so far

when search ends, announce the solution(s) found

Example 1: Traveling Salesman Problem

Given n cities with known distances between each pair, find the shortest tour that passes through all the cities exactly once before returning to the starting city Alternatively: Find shortest Hamiltonian circuit in a weighted connected graph Example:
2

a
5 8 3

b
4

TSP by Exhaustive Search

Tour abcda abdca acbda acdba adbca adcba More tours? Less tours? Efficiency: Cost 2+3+7+5 = 17 2+4+7+8 = 21 8+3+4+5 = 20 8+7+4+2 = 21 5+4+3+8 = 20 5+7+3+2 = 17

Example 2: Knapsack Problem

Given n items: weights: w1 w2 wn values: v1 v2 vn a knapsack of capacity W Find most valuable subset of the items that fit into the knapsack Example: Knapsack capacity W=16 item weight value 1 2 $20 2 5 $30 3 10 $50 4 5 $10

Knapsack Problem by Exhaustive Search

Subset Total weight {1} 2 {2} 5 {3} 10 {4} 5 {1,2} 7 {1,3} 12 {1,4} 7 {2,3} 15 {2,4} 10 {3,4} 15 {1,2,3} 17 {1,2,4} 12 {1,3,4} 17 {2,3,4} 20 {1,2,3,4} 22 Total value $20 $30 $50 $10 $50 $70 $30 $80 $40 $60 not feasible $60 not feasible not feasible not feasible

Efficiency:

Sec 3.4 Exhaustive Search

Assignment Problem
1. Assign N jobs to N people, one person per job. 2. Find an assignment such that the total cost is smallest.

Example:
Let C[i, j] be the cost of assigning job j to person i. J1 J2 J3 J4 jobs P1 9 2 7 8 C = P2 6 4 3 7 P3 5 8 1 8 cost P4 7 6 9 4 persons * Select one element in each row so that all selected elements are in different columns and the total sum of the cost is the smallest. * n-tuple notation: t = <t[1], t[2], t[3], t[4]> t[i] = the column (job) # of the element selected in row i * <2, 3, 4, 1> C(1,2) + C(2,3) + C(3,4) + C(4,1) = 2 + 3 + 8 + 7 = 20

Few iterations
Person 1 2 3 4 ======== < 1, 2, 3, 4 > < 1, 2, 4, 3 > < 1, 3, 2, 4 > < 1, 3, 4, 2 > < 1, 4, 2, 3 > < 1, 4, 3, 2 > 9 + 4 + 1 + 4 = 18 9 + 4 + 8 + 9 = 30 9 + 3 + 8 + 4 = 24 9 + 3 + 8 + 6 = 26 9 + 7 + 8 + 9 = 33 9 + 7 + 1 + 6 = 23

< 2, 1, 3, 4 > 2 + 6 + 1 + 4 = 13 < 2, 1, 4, 3 > 2 + 6 + 8 + 9 = 25 etc. j1 j2 j3 jN Total # of permutations = N ! p1 p2 p3 pN

Final Comments on Exhaustive Search

Exhaustive-search algorithms run in a realistic amount of time only on very small instances

In some cases, there are much better alternatives!

Euler circuits shortest paths minimum spanning tree assignment problem

In many cases, exhaustive search or its variation is the only known way to get exact solution