Recurrences

an algorithm contains a recursive call to itself, its running time can be

described by a recurrence
a recurrence is an equation or inequality that describes a function in
terms of its value on smaller inputs
worst case running time T (n) of merge sort can be described by

Θ(1) if n = 1
T (n) =
2T (n/2) + Θ(n) if n > 1

Solution T (n) = Θ(n log2 n)

Three methods to solve recurrences
substitution method, recursion-tree method, master method

M. A. Zaveri, SVNIT, Surat () Algorithm Analysis and Design 1 / 12

 Recurrence solving methods

substitution method: guess a bound and then use mathematical

induction to prove our guess correct
recursion-tree method: converts the recurrence into a tree whose
nodes present the costs incurred at various levels of the recursion
master method: provides bounds for recurrences of the form

T (n) = aT (n/b) + f (n)

a ≥ 1 b > 1 and f (n) given function

M. A. Zaveri, SVNIT, Surat () Algorithm Analysis and Design 2 / 12

 Substitution method

two steps
guess the form of the solution
use mathematical induction to find the constants and show that the
solution works
can be applied when it is easy to guess the form of the answer
can be used to establish either upper or lower bounds on a recurrence
say, determine an upper bound on the recurrence

T (n) = 2T (bn/2c) + n

guess the solution T (n) = O(n lg n)

prove that T (n) ≤ cn lg n for an appropriate choice of constant c > 0

M. A. Zaveri, SVNIT, Surat () Algorithm Analysis and Design 3 / 12

 Substitution method

assume that this bound holds for bn/2c that

T (bn/2c) ≤ cbn/2c lg(bn/2c)

T (n) ≤ 2(cbn/2c lg(bn/2c)) + n

≤ cn lg(n/2) + n
= cn lg n − cn lg 2 + n
= cn lg n − cn + n ≤ cn lg n

it holds when c ≥ 1
need to show it also holds for boundary conditions
for n = 1 T (1) = 0 which is at odds with T (1) = 1; base case fails
requires to prove T (n) ≤ cn lg n for n ≥ n0
appropriate value of c and n0 must be chosen, here c ≥ 2 and
n ≥ n0 (= 2)

M. A. Zaveri, SVNIT, Surat () Algorithm Analysis and Design 4 / 12

 Substitution method
no general way to guess the correct solutions to recurrences
little algebraic manipulation can make an unknown recurrences similar
to one that have a solution
√
T (n) = 2T (b nc) + lg n
√
with n it is difficult, change variable m = lg n

T (2m ) = 2T (2m/2 ) + m

again rename S(m) = T (2m )

S(m) = 2S(m/2) + m

new recurrence has solution: S(m) = O(m lg m)

changing back from S(m) to T (n)

T (n) = T (2m ) = S(m) = O(m lg m) = O(lg n lg lg n)

M. A. Zaveri, SVNIT, Surat () Algorithm Analysis and Design 5 / 12

 Recursion tree method

substitution method provides a succinct proof that a solution to a

recurrence is correct but difficult to come up with a good guess
recursion tree - each node represents the cost of a single subproblem
somewhere in the set of recursive function invocations
sum the costs within each level of the tree to obtain a set of per-level
costs
sum all the per-level costs to determine the total cost of all levels of
the recursion
recursion tree useful when the recurrence describes the running time
of a divide-and-conquer algorithm
it may be used to generate a good guess, then use substitution
method
it may be used as a direct proof of a solution to a recurrence

M. A. Zaveri, SVNIT, Surat () Algorithm Analysis and Design 6 / 12

 Recursion tree method

say, find upper bound for the solution

T (n) = 3T (bn/4c) + Θ(n2 )

floors and ceilings are insubstantial in solving recurrences

T (n) = 3T (n/4) + c(n2 )

assume n is power of 4
cn2 term at the root represents the cost at the top level of recursion
three subtrees of the root represent the cost incurred by the
subproblems of size n/4
continue expanding each node in the tree by breaking it into its
constituent parts determined by the recurrence

M. A. Zaveri, SVNIT, Surat () Algorithm Analysis and Design 7 / 12

 Recursion tree method

subproblem sizes decrease as move further from the root, must reach
a boundary condition
the subproblem size for a node at depth i is n/4i
the subproblem size hits n = 1 when n/4i = 1 or when i = log4 n
tree has log4 n + 1 levels (0, 1, 2, . . . , log4 n)
the cost at each level of tree has three times more nodes than the
level above
the number of nodes at depth i is 3i
subproblem sizes reduce by a factor of 4 for each level move down
from the root
each node at depth i (i = 0, 1, 2, . . . , log4 n − 1) has cost of c(n/4i )2

M. A. Zaveri, SVNIT, Surat () Algorithm Analysis and Design 8 / 12

 Recursion tree method
the total cost over all nodes at depth i (i = 0, 1, 2, . . . , log4 n − 1) is
3i c(n/4i )2 = (3/16)i cn2
the last level at depth log4 n has 3log4 n = nlog4 3 nodes, each
contributing cost T (1), for a total cost of nlog4 3 T (1) and which is
Θ(nlog4 3 )
add up the costs over all levels to determine the cost for the entire
tree T (n)
 2  log4 n−1
3 2 3 3
= cn2 + cn + cn2 + · · · + cn2 + Θ(nlog4 3 )
16 16 16
log4 n−1  i
X 3
= cn2 + Θ(nlog4 3 )
16
i=0
(3/16)log4 n − 1 2
= cn + Θ(nlog4 3 )
(3/16) − 1

M. A. Zaveri, SVNIT, Surat () Algorithm Analysis and Design 9 / 12

 Recursion tree method
T (n) cn2
.↓&
T ( n4 ) T ( n4 ) T ( n4 )

cn2
.↓&
c( n4 )2 c( n4 )2 c( n4 )2
.↓& .↓& .↓&
n n n n n n n n
T ( 16 ) T ( 16 ) T ( 16 ) T ( 16 ) T ( 16 ) T ( 16 ) T ( 16 ) T ( 16 )
.↓& .↓& .↓& .↓& .↓& .↓& .↓& .↓&

.. .. .. .. .. .. .. .. .. .. .. ..
............
T (1) T (1) T (1) T (1) T (1) T (1) T (1) T (1) T (1) T (1) T (1) T (1) T (1)
last formula looks messy, take advantage of small amounts of
sloppiness

M. A. Zaveri, SVNIT, Surat () Algorithm Analysis and Design 10 / 12

 Recursion tree method
use an infinite decresing geometric series as an upper bound

log4 n−1 
3 i 2
X 
T (n) = cn + Θ(nlog4 3 )
16
i=0
∞ 
3 i 2
X 
< cn + Θ(nlog4 3 )
16
i=0
1
= cn2 + Θ(nlog4 3 )
1 − (3/16)
16 2
= cn + Θ(nlog4 3 ) = O(n2 )
13

derived a guess of T (n) = O(n2 ) for original recurrence

root’s contribution to the total cost is cn2 , it is a constant fraction of
the total cost
total cost of the tree is dominated by the cost of the root
M. A. Zaveri, SVNIT, Surat () Algorithm Analysis and Design 11 / 12
 Recursion tree method
if O(n2 ) is indeed an upper bound for the recurrence then it must be
a tight bound
use the substitution method to verify that our guess is correct, i.e.,
T (n) = O(n2 ) is an upper bound for the recurrence
T (n) = 3T (bn/4c) + Θ(n2 )
need to show that T (n) ≤ dn2 for some constant d > 0

T (n) ≤ 3T (bn/4c) + cn2

≤ 3dbn/4c2 + cn2
≤ 3d(n/4)2 + cn2
3 2
= dn + cn2
16
≤ dn2

the last step holds as long as d ≥ (16/13)c

M. A. Zaveri, SVNIT, Surat () Algorithm Analysis and Design 12 / 12