Analysis of Algorithms

Recurrences

Instructor: Dr. Ahmad Qawasmeh

Note: Slides were written by Dr. Monica Nicolescu at University of Nevada, Reno
 Methods for Solving Recurrences

• Iteration method

• Substitution method

• Recursion tree method

• Master method

2
The recursion-tree method

Convert the recurrence into a tree:

– Each node represents the cost incurred at that level
of recursion

– Sum up the costs of all levels

– Useful to when the recurrence describes the Running
time of a divide-and-conquer algorithm

Used to “guess” a solution for the recurrence

3
 Example 1
W(n) = 2W(n/2) + n2

• Subproblem size at level i is: n/2i

• Subproblem size hits 1 when 1 = n/2i  i = lgn
• Cost of the problem at level i = (n/2i)2 No. of nodes at level i = 2i
• Total cost: lg n−1 2
n lg n−1
 1 
i 
1
i

2   2    2 
1
W (n) = lg n
+ 2 W (1) = n
2 2
+nn2
+ O(n) =n + O(n) = 2n 2
i=0
i
i=0 i=0 1− 1
2
 W(n) = O(n2)
4
 Example 2
E.g.: T(n) = 3T(n/4) + cn2

• Subproblem size at level i is: n/4i

• Subproblem size hits 1 when 1 = n/4i  i = log4n
• Cost of a node at level i = c(n/4i)2
• Number of nodes at level i = 3i  last level has 3log n = nlog 3 nodes
4 4
• Total cost:
) ( ) ( )
i
 3 2
( 3 2
log 4 n−1  i

    cn + O n log4 3 =
1 cn 2 + O n log4 3
T (n) = 
i=0
  cn + O n
 16 
log4 3

i=0  16 
3
= O(n 2 )
1−
16
 T(n) = O(n ) 2 5
 Example 2
E.g.: T(n) = 3T(n/4) + cn2

) ( ) ( )
i
 3 2
( 3 2
log 4 n−1  i


1
T (n) =   cn + O n
log4 3
    cn + O n log4 3 = cn 2 + O n log4 3 = O(n 2 )
i=0  16  i=0  16 
3
1−
16 6
 Example 2 - Substitution
T(n) = 3T(n/4) + cn2
• Guess: T(n) = O(n2)
– Induction goal: T(n) ≤ dn2, for some d and n ≥ n0
– Induction hypothesis: T(n/4) ≤ d (n/4)2
• Proof of induction goal:
T(n) = 3T(n/4) + cn2
≤ 3d (n/4)2 + cn2
= (3/16) d n2 + cn2
= d n2 + cn2 + (3/16) d n2 - d n2
= d n2 + cn2 - (13/16) d n2
≤ d n2 if: cn2 - (13/16) d n2 ≤ 0
d ≥ (16/13)c
Therefore: T(n) = O(n2)
7
 Example 3
W(n) = W(n/3) + W(2n/3) + n

• The longest path from the root to

a leaf is:

n → (2/3)n → (2/3)2 n → … → 1
• Subproblem size hits 1 when
1 = (2/3)in  i=log3/2n

• Cost of the problem at level i = n

• Total cost:
log3 / 2 n log3 / 2 n

n = n 
lg n 1
W (n)  n + n + ... = 1 = n log 3/ 2 n = n = n lg n
i=0 i=0 lg 3 / 2 lg 3 / 2
for all levels
 W(n) = O(nlgn) 8
 Example 3 - Substitution
W(n) = W(n/3) + W(2n/3) + n
• Guess: W(n) = O(nlgn)
– Induction goal: W(n) ≤ dnlgn, for some d and n ≥ n0
– Induction hypothesis: W(k) ≤ d klgk for any k < n
(n/3, 2n/3)
• Proof of induction goal:
Try it out as an exercise!!

9
 Master’s method
• “Cookbook” for solving recurrences of the form:

T (n) = aT   + f (n)
n
b
where, a ≥ 1, b > 1, and f(n) > 0

Idea: compare f(n) with nlogb a

• f(n) is asymptotically smaller or larger than nlogb a by

a polynomial factor n

• f(n) is asymptotically equal with nlogba

10
 Master’s method
• “Cookbook” for solving recurrences of the form:

T (n) = aT   + f (n)
n
b
where, a ≥ 1, b > 1, and f(n) > 0

Case 1: if f(n) = O(nlogba -) for some  > 0, then: T(n) = (nlogba)

Case 2: if f(n) = (nlogba), then: T(n) = (nlogba lgn)

Case 3: if f(n) = (nlogba +) for some  > 0, and if

af(n/b) ≤ cf(n) for some c < 1 and all sufficiently large n, then:

T(n) = (f(n))
regularity condition
11
 Why nlogba? T (n) = aT   + f (n)
n
b

•
T (n) = aT  
n Case 1:
b – If f(n) is dominated by n logba:
a 2T  2 
n
• T(n) = (nlog bn)
b 

a 3T  3 
n
• Case 3:
b 
– If f(n) dominates nlogba:
T (n) = aiT  i  i • T(n) = (f(n))
n
b 

• Case 2:
• Assume n = bk  k = logbn – If f(n) = (nlog ba):
• T(n) = (nlog a logn)
• At the end of iterations, i = k: b

T (n) = a log b n  bi 
( ) (
T  i  = a logb n T (1) =  a logb n =  n logb a )
b 
12
 Examples

T(n) = 2T(n/2) + n

a = 2, b = 2, log22 = 1

Compare nlog 2 with f(n) = n

2

 f(n) = (n)  Case 2

 T(n) = (nlgn)

13
 Examples
T(n) = 2T(n/2) + n2
a = 2, b = 2, log22 = 1
Compare n with f(n) = n2
 f(n) = (n1+) Case 3  verify regularity cond.
a f(n/b) ≤ c f(n)
 2 n2/4 ≤ c n2  c = ½ is a solution (c<1)
 T(n) = (n2)

14
 Examples (cont.)

T(n) = 2T(n/2) + n

a = 2, b = 2, log22 = 1

Compare n with f(n) = n1/2

 f(n) = O(n1-) Case 1

 T(n) = (n)

15
 Examples

T(n) = 3T(n/4) + nlgn

a = 3, b = 4, log43 = 0.793
Compare n0.793 with f(n) = nlgn

f(n) = (nlog 3+) Case 3

4

Check regularity condition:

3(n/4)lg(n/4) ≤ (3/4)nlgn = c f(n), c=3/4

T(n) = (nlgn)
16
 Examples

T(n) = 2T(n/2) + nlgn

a = 2, b = 2, log22 = 1

• Compare n with f(n) = nlgn

– seems like case 3 should apply

• f(n) must be polynomially larger by a factor of n

• In this case it is only larger by a factor of lgn

17
Readings

• Chapter 4

18