Dr. Afzal A. L.

Dept. of Computer Science and Engineering

College of Engineering Muttathara.
 Recurrence Relation
• A function may contain a call to that function itself. such functions are called recursive
functions.
• Certain problems can be solved easily by using recursive functions
– Eg:- Factorial, Fibonacci number, Merge Sort, Quick Sort, Linear and Binary Search,
DFS ……
• Consider Sum of first n natural numbers.
– Sum(n) = 1+2+3+…… +n // Iterative method for adding n numbers
– Sum(n) = n+ Sum(n-1) // Recursive method for adding n numbers.
• Let n = 5
• Sum(5) = 5 +sum(4) -> 5+4+sum(3) -> 5+4+3+sum(2) -> 5+4+3+2+sum(1) -> 5+4+3+2+1+sum(0) -
> 5+4+3+2+1+0
• All the recursive algorithm should terminate after a particular number of instances.
• Base condition – Termination Condition is called base condition
• Above example , base condition is sum(n) = 1 when n =1 . No more instances of invocation. 2
 Properties of Recursive functions
• A Recursive function involve a call to that function itself.
• To avoid infinite running of recursive function, there are two properties that a recursive function
must have −
• Base criteria − There must be at least one base criteria or condition, such that, when this
condition is met the function stops calling itself recursively. It will be the termination condition.
• Progressive approach − The recursive calls should progress in such a way that each time a
recursive call is made it comes closer to the base criteria. Each instance of recursive call should
approaches to the base condition.
• Ex:-
int fact(int n)
{ if (n < = 1) // base case
return 1;
3
else return n*fact(n-1); }
 Recurrence Equations
• Many algorithms, particularly divide and conquer algorithms are designed in a recursive
manner.
• They have time complexities which are naturally modeled by recurrence relations.
• A recurrence relation is an equation which is defined in terms of itself.
• A recurrence is an equation or inequality that describes a function in terms of its values on
smaller inputs.
• Example : Let T(n) be the complexity of a recursive algorithm .
• Then the recursive equation for the complexity of this algorithm may be
– T(n) = C + T(n-1) or T(n) = 2T(n/2) + C or ……

• The worst case complexity of Merge sort Algorithm can be expressed as :

T (n) = 1 if n=1 // Base Condition
2T (n/2) + n if n>1 4
 Solving Recurrence
• To solve a Recurrence Relation means to obtain a function defined on the natural
numbers that satisfy the recurrence.
• There are four methods to solve Recurrence relation.
1. Iteration Method
2. Substitution Method
3. Recursion Tree Method
4. Master Method

5
• Solve : T (n) = 1 if n=1
= 2T (n-1) if n>1
Iteration Method
• This method iteratively unfold the recurrence relation until we get a pattern
T(n) = 2T(n-1)
T(n-1 ) = 2T(n-1-1) Substitute this
=2[2T(n-2)] = 4T(n-2) = 22 T(n-2)
=4[2T(n-3)] = 8T(n-3) = 23 T(n-3)
…………….
=2i T(n-i) ----- Equation 1
• Find a value such a way that the recursive in the pattern becomes base condition
• n-i = 1 🡪 i = n-1
• Put n-i = 1 in Equation 1 : 2i T(1) = 2i
• Convert the pattern variable back to n 6

• = 2n-1 = O( 2n )
• Solve T (n) = T (n-1) +1 and T (1) = θ (1).
• T (n) = 1+ T (n-1)
= 1+ [1+ T (n-2)] = 2 + T(n-2)
= 2+[1+T(n-3)] = 3+T(n-3)
….
= k + T(n-k) put n- k =1
= k + T(1)
= K + θ (1) Substitute k = n -1
= n-1 + θ (1)
= θ (n).

7
• T(n) = 8T(n/2) +n2 , T(1) = 1
T(n) = n2 + 8T(n/2)
= n2 +8 [(n/2)2 +8T(n/4)]
= n2 + 8 n2 /4 + 82 T(n/ 4)
= n2 +2 n2 + 82 [(n/4)2 + 8 T(n/8) ]
= n2 +2 n2 + 82 n2 / 42 + 83 T(n/ 8 )
= n2 +2 n2 + 22 n2 + 83 T(n/ 8 )
= n2 +2 n2 + 22 n2 + 83 T(n/ 23 )
…….
=n2 +2 n2 + 22 n2 +….. + 2k-1 n2 + 8k T(n/ 2k)

8
•

Sum of terms in GP:

Sn = a[(rn – 1)/(r – 1)] if r ≠ 1 and r > 1
9
 Homework
1. T(n) = T(n-1) + n4 T(1) = 1
2. T(n) = 2T(n/2)+2 if n > 2 , T(n) = 1 if n =2
3. Solve using Iteration method T(n)=2T(n/2)+n, T(1)=1
4. Using iteration solve the following recurrence equation
T(n)= 2 if n=1 else T(n)= 2T(n/2)+2n+3

https://youtu.be/UXNRcFIZsjo

10
 Recursion Tree Method
• visual representation of recursive call hierarchy where each node represents the cost of a single
subproblem
• For e.g., T(n) = a T(n/b) + f(n) where a > 1 ,b > 1 and f(n) is the non recursive function then :
• F(n) is the cost of splitting or combining the sub problems.
• A recursion tree is a graphical representation of iteration method.
• In this method, we draw a recurrence tree and calculate the time taken by every level of tree.
• In general, we consider the second term (non recurrence ) in recurrence equation as root.
• Find the sum of cost in every level of tree
• Finally, we sum the work done at all levels.
• To draw the recurrence tree, we start from the given recurrence and keep drawing till we find a
pattern among levels.
11
• The pattern is typically a arithmetic or geometric series.
 • Solve T(n) = 2T(n/2) + n : using recursion tree method

• Here n is the cost of recursion at root level (non-recursive part)

• Tree start with n as root node.

• There are two recursive part in this equation. Add them as

children of root node
n

T(n/2) T(n/2) 12
• Solve T(n) = 2T(n/2) + n : using recursion tree method
n

T(n/2) T(n/2)

• Apply n/2 in our equation :

• T(n/2) = 2T(n/2 /2) + n/2
• = 2T(n/4) + n/2
• Cost corresponding to T(n/2) is n/2 and the recursive parts are T(n/4),
T(n/4)

13
T(n/2) = 2T(n/4) + n/2

n
n
n/2 n/2
T(n/2) T(n/2)
T(n/4) T(n/4) T(n/4) T(n/4)
• Apply n/4 in our equation :
• T(n/4) = 2T(n/4 /2) + n/4 = 2T(n/8) + n/4
• Cost corresponding to T(n/2) is n/2 and the recursive parts are T(n/8), T(n/8)

14
 n n

n/2 n/2 n

n/4 n/4
n/4 n/4 n

T(n/8) T(n/8) T(n/8) T(n/8) T(n/8) T(n/8) T(n/8) T(n/8) n

…

…
…

…
…

…
…
T(1) T(1) T(1) T(1) T(1)
T(1) T(1) T(1) n

15
• Subproblem size at level i is: n/2i
• Subproblem size hits 1 when n/2i =1 ⇒ n = 2i ⇒ logn = log 2i ⇒ i = log n
• There are log n levels
• Cost of the problem at level i = n
• Total cost will be logn times n
• Solution is : O(n logn)

16
• Solve T(n) = 2T(n/2) + c : using recursion tree method
c

T(n/2) T(n/2)

• Apply n/2 in our equation :

• T(n/2) = 2T(n/2 /2) + c = 2T(n/4) + c
• Cost corresponding to T(n/2) is c and the recursive parts are T(n/4), T(n/4)

17
 c

c
c c

T(n/2) T(n/2)
T(n/4) T(n/4) T(n/4) T(n/4)
• Apply n/4 in our equation :
• T(n/4) = 2T(n/4 /2) + C = 2T(n/8) + C
• Cost corresponding to T(n/2) is C and the recursive parts are T(n/8), T(n/8)

18
 C
C

C 2C
C

C C
C C 4C

T(n/8) T(n/8) T(n/8) T(n/8) 8C

T(n/8) T(n/8) T(n/8) T(n/8)
…

…
…

…
…

…
…
T(1) T(1) T(1) T(1) T(1)
T(1) T(1) T(1)
19
•

20
•

21
•

• r =½<1
• Then GP is 1/ (1-r)

22
Solve T(n) = T(n/3) + T(2n/3) + cn

23
cn

cn

24
• Subproblem size at level i is: n/3i and 2n/3i
• n/3i converges to 1 before 2n/3i
• For the best case we can take n/3i as base case
• Subproblem size hits 1 when 1 = n/3i ⇒ i = log3n
• Or Elements from shortest path are being divided by 3, so length of this path will be equal
to log3n
• Sum will be cn+cn+cn … , log3n times
• = c n log3n
• =Ω(n logn)

25
 • Solve T(n) = T(n/4) + T(n/2) + n2: using recursion tree method

• Here n2 is the cost of recursion at root level

• Tree start with n2 as root node.
n2

• There are two recursive part in this equation. Add them as

children of root node
n2

T(n/4) T(n/2) 26
• Solve T(n) = T(n/4) + T(n/2) + n2: using recursion tree method
n2

T(n/4) T(n/2)

• Apply n/4 in our equation :

• T(n/4) = T(n/4 /4) + T(n/4 /2) +( n/4 )2 = T(n/16) + T(n/8) + ( n/4 )2
• Cost corresponding to T(n/4) is ( n/4 )2 and the recursive parts are T(n/16), T(n/8)
• Apply n/2 in our equation
• T(n/2) = T(n/2/4) + T(n/2/2) +( n/2)2 = T(n/8) + T(n/4) + ( n/2)2
• Cost corresponding to T(n/2) is ( n/2 )2 and the recursive parts are T(n/8) , T(n/4)
Redraw the recursive nodes with their corresponding cost and children
27
 n
n2 2
(n/4) (n/2)
2 2
T(n/4) T(n/2)
T(n/16) T(n/8) T(n/8) T(n/4)

28
 n
2
(n/4) (n/2)
2 2
(n/16)2 (n/8)2 (n/8)2 (n/4)2
…

…
…
…

T(1) T(1) T(1) T(1)

29
•

30
31
 Homework
Using Recursion Tree method, solve. Assume constant time for small values of n.
T(n)= 2T(n/10)+ T(9n/10)+n

Recurrence Relation: Recursion Tree method - Examples: Set 2 - YouTube

DESIGN AND ANALYSIS OF ALGORITHMS, LECTURE - #7, Recursion tree method to solve
recurrence relations - YouTube
32
 Substitution Method
• The substitution method for solving recurrences is described using two steps:
– Guess the form of the solution.
– Use induction to show that the guess is valid.
• For example consider the recurrence T(n) = 2T(n/2) + n.
• We guess the solution as T(n) = O(nLogn).
• Now we use induction to prove our guess.
• We need to prove that T(n) <= cnLogn.
Here, T(n) = 2T(n/2) + n [ T(n/2) = c n/2 Log(n/2) ]
= 2 [ c * n/2 * log(n/2)]+n
= c*n*log(n/2)+n
= cnLogn - cnLog2 + n

= cnLogn - cn + n
<= c nLogn 33

O(nLogn)
• T (n) = T +n

Let T (n) = O (log n)

We have to show that for some constant c
T (n) ≤c logn.
T(n) = c log(n/2)+n
= c*logn – c* log2 +n
= c*logn –c +n
<= c* logn
Hence T(n) = O (log n)

34
 Exercise -3
•

35
• T(n) = T(n/2)+f(n) =O(logn)
• T(n) = 2T(n/2)+f(n) =O(nlogn)

36
 Master Theorem
• The master theorem is used for calculating the time complexity of recurrence relations (divide and
conquer algorithms) in a simple and quick way.
• The master method is a formula for solving recurrence relations of the form:
• T(n) = aT(n/b) + f(n), where,
n = size of input
a = number of subproblems in the recursion
n/b = size of each subproblem. All subproblems are assumed to have the same size.
f(n) = cost of the work done outside the recursive call, which includes the cost of dividing the problem
and the cost of merging the solutions
Here, a ≥ 1 and b > 1 are constants, and f(n) is an asymptotically positive function. An asymptotically
positive function means that for a sufficiently large value of n, we have f(n) > 0.
Then T (n) has the following asymptotic bounds:

37
 Masters Theorem
Case 1: if f(n) = O(nlogba -ε) for some ε > 0, then: T(n) = Θ(nlogba)

Case 2: if f(n) = Θ(nlogba ), then: T(n) = Θ(nlogba log n)

Case 3: if f(n) = Ω(nlogba +ε) for some ε > 0, and if

af(n/b) ≤ cf(n) for some c < 1 and all sufficiently large n, then:

T(n) = Θ(f(n))

38
 Master Method
• Provides a “cookbook” method for solving recurrences
• Recurrence must be of the form: T(n) = aT(n/b) + f(n)
• where a≥1 and b>1 are constants and f(n) is an asymptotically positive function.
Idea: compare f(n) with nlogba

• f(n) is asymptotically smaller or larger than nlogba by a polynomial factor nε

• f(n) is asymptotically equal with nlogba logkn

f(n) < nlogba -> T(n) = Θ(nlogba)
• K is usually 0 f(n) = nlogba -> Θ(nlogba lgn)
f(n) > nlogba -> T(n) = Θ(f(n)), Regularity
39
1. Solve T(n) = .5 T(n/2)+n Using master’s theorem
Here a = .5 which is less than 1. Hence Master’s theorem can not be applied here

2. Solve T(n) = 2n T(n/2) + n

Here a is not a constant. hence Master’s theorem can not be applied here.
3. Solve T(n) = 2T(n/2) – nlogn
Here f(n) is not a positive function. hence can not be applied.

40
• Solve T(n) = 2T(n/2) + n using Master method
• a =2 , b = 2 , f(n) = n
f(n) < nlogba 🡪 T(n) = Θ(nlogba)
• nlogba = nlog22
f(n) = nlogba 🡪 Θ(nlogba lgn)
= n*1 = n
n log a f(n) > nlogba 🡪 T(n) = Θ(f(n)), Regularity
Compare b & f(n)
• n log a
b = f(n) = n 🡪 Case 2
• T(n) = Θ(nlogba lgn)
= Θ(n lgn)

41
 Solve T(n) = 2T(n/2) + n2 using master method
a = 2, b = 2,
nlogba = nlog22 = n*1 = n
Compare nlogba with f(n) = n2
n2 ?n

⇒ f(n) = Ω(n1+ε) Case 3 ⇒ verify regularity cond.

a f(n/b) ≤ c f(n)
⇔ 2 n2/4 ≤ c n2 ⇒ c = ½ is a solution (c<1)
42
⇒ T(n) = Θ(n2)
•

43
 Example -4
• Solve 9T(n/3)+ n
• a =9 , b =3 , f(n) = n
• nlogba = nlog39 = nlog332 f(n) < nlogba 🡪 T(n) = Θ(nlogba)
= n2 log33 [log33 =1] f(n) = nlogba 🡪 Θ(nlogba lgn)
= n2
f(n) > nlogba 🡪 T(n) = Θ(f(n)), Regularity
• Compare f(n) and nlogba
• n < n2 , f(n) < nlogba case 1.
• T(n) = Θ(nlogba)
• T(n) = Θ (n2 )

44
•

45
•

46
 Tutorial
•

47