Recursions and Their Complexity Analysis

Chapter Objectives
Learn about recursive definitions n Explore the base case and the general case of a recursive definition n Explore how to use recursive functions to implement recursive algorithms n Complexity analysis of recursive functions n Recursion and backtracking
n

Recursive Definitions
n

Recursion
Process of solving a problem by reducing it to smaller versions of itself n It can be either
n n

Reduce and conquer

n Reduce a problem of size n into a problem of size n1

Divide and conquer

n Reduce a problem of size n into one ore more problem of size n/b

Recursive Definitions
Recursive algorithm n Algorithm that finds the solution to a given problem by reducing the problem to smaller versions of itself n Has one or more base cases n Implemented using recursive functions n Recursive function n A function that calls itself is called a recursive function
n

Recursive Definitions
Direct Recursion : a function that calls itself n Indirect Recursion : a function that calls another function and eventually results in the original function call n Receive(buffer)Decode(buffer) Store(b uffer)Receivi(buffer). n Nested Recursion: the argument of the function is a smaller recursive call of the same function n F(n) = F( F(n-1) + 2);
n

Infinite Recursion : the case where every recursive call results in another recursive call

Recursive Definitions
Anchor, ground, or base case: n Case in recursive definition in which the solution is obtained directly n Stops the recursion n General case: n Case in recursive definition in which a smaller version of itself is called n Must eventually be reduced to a base case
n

Recursion: Example
Classic linear recursion example--the factorial function: n n! = 1 2 3 (n-1) n n Recursive definition:
n

In this example, factorial(0) is called the base case (there should be at least one) and factorial(n) for n > 0 is called recursive case.

Recursive Factorial Function

int recursiveFactorial(int num) { if(num == 0) return 1; else return num * recursiveFactorial(num 1); }

Visualizing Factorial How does the OS Perform Recursions?

Designing Recursive Functions: What you Should

Do
Understand problem requirements n Identify base cases n Provide direct solution to each base case n Identify general case(s) n Provide solutions to general cases in terms of smaller versions of the problem itself
n

Adding the Numbers in an Array: Divide and Conquer Approach

n n n n
Understand problem requirements Identify base cases Provide direct solution to each base case Identify general case(s)

n Provide solutions to general cases in terms of smaller versions of itself

Adding all the Numbers in an Integer Array A: Binary

Recursive Method
n

Binary recursion occurs whenever there are two recursive calls for each non-base case.
Algorithm BinaryRecSum(A, i, n): Input: An array A and integers i and n Output: The sum of the n integers in A starting at index i BinaryRecSum(A, 0, 5) = 59 if n = 1 then return A[i] return BinaryRecSum(A, i, Ceil(n/ 2)) + BinaryRecSum(A, i + ceil(n/ 2), floor(n/ 2))

Binary Recursive Trace

Example: BinaryRecSum(A, 0, 8): A =

Adding the Numbers in an Array: Reduce and Conquer Approach

n Understand problem requirements n Identify base cases

n Provide direct solution to each base case n Identify general case(s) n Provide solutions to general cases in terms of smaller versions of itself

Adding all the Numbers in an Integer Array A: Linear Recursive Method

n

Unary recursion occurs whenever there is one recursive call for each non-base case.
Algorithm LinearRecSum(A, i, n): Input: An array A and integers i and n Output: The sum of the n integers in A starting at index i LinearSum(A, 0, 5) = 59 if n = 1 then return A[i ] Else return A[i]+ LinearRecSum(A, i+1, n-1)

Linear Recursive Trace

Example: LinearRecSum(A, 0, 8): A =

Space Complexity
What is the size OS stack required to execute LinearRecSum and BinaryRecSum, given that the size of the array is n?? (Hint: use the size of the activation block)

A =

Complexity Analysis of Recursive Algorithms

Steps in mathematical analysis of worst case recursive algorithms with an input of size n: n Decide on parameter n indicating input size n Identify algorithms basic operation (Not the base case) n Compute the number of Basic operations C(.) for the base case and for the general case. (typically it is a recurrence function for the general case) n Solve the recurrence to obtain a closed form or estimate the order of magnitude of the solution(using the Master Method). (To get a closed form of a recurrence relation,

you can use by backward/forward substitutions or another method).

Recursive Factorial Function

int recursiveFactorial(int num) { if(num == 1) return 1; else return num * recursiveFactorial(num 1); }

Complexity Analysis of Recursive Algorithms: Factorial

Steps in mathematical analysis of recursive algorithms: n Decide on parameter n indicating input size n Identify algorithms basic operation n Determine worst , average and best case for input of size n
n Compute the number of Basic operations C(n) for the

base case and for the general case. (typically it is a recurrence function for the general case)

n Solve the recurrence to obtain a closed form or estimate

the order of magnitude or complexity of the solution.

Recurrence Relationship for the Number of Basic Operations vs. the Original Recursive Function n The recurrence function for the number of basic operations is different from the original recursive function n Factorial
F(n) := 1 if n = 1 n F(n) := F(n-1) * n if n >1
n n

Recurrence for number of multiplications to compute n!:

C(1) = 0; if n = 1 n C(n) = C(n-1) +1; if n >1
n

Solving Recurrence Relation Using Forward Substitution: Factorial

C(n) = C(n-1) + 1 C(1) = 0

= C(n-2) + 1 +1 = C(n-3) + 1 + 1+ 1 = C(n-4) + 1 +1 +1+ 1 : : = C(1) +1 +1 +1 +1..+1 = 0 + n-1 O(n)

Solving Recurrence Relation Using Forward Substitution.

n

Ex 1 n C(n) = C(n-1) +4
n n n n n n n

C(1) = 2

C(n-1) = C(n-2) + 4 C(n-2) = C(n-3) + 4 : : C(1) = 2 C(n) = (c(1) + 4)+4)+4+4) + 4 C(n) = ((((( 2) + 4)+4)+4+4) + 4 //with n-

1 4s n C(n) = (n-1) * 4 + 2 = 4n -2

Solving Recurrence Relation Using Backward Substitution: Factorial

C(n) = C(n-1) + 1 C(1) = 0 C(2) = C(1) + 1 = 0 + 1 = 1 C(3) = C(2) + 1 = 1 + 1 = 2 C(4)= C(3) + 1 =2+1=3 : : C(n)= n-1 O(n)

Important Reduce-andConquer Recurrence Types

n

A recursive call of the function reduce the

size of the problem by one, and performs b basic operations.

C(n) = C(n-1) + b C(1) = d Solution: C(n) = (n-1)b + d linear
n

Ex. Printing a list in reverse order ReversePrint (A,n) if n=1 then print A[n-1] // the last element is at index n1 else ReversePrint(A,n-1); print last element in A, A[n-1]; print Next Element

Adding all the Numbers in an Integer Array A: Linear Recursive Method

Algorithm LinearRecSum(A, i, n): Input: An array A and integers i and n

Output: The sum of the n integers in A starting at index i LinearSum(A, 0, 5) = 59 if n = 1 then return A[i ] Else return A[i]+ LinearRecSum(A, i+1, n-1) Solution

C(n) = C(n-1) + b C(1) = d b=1, d = 0 Solution: C(n) = (n-1)b + d = n-1

Important Reduce-andConquer Recurrence Types

n

A recursive call of the function reduce the size of the problem by one, and performs b*n basic operations.
C(n) = C(n-1) + bn C(1) = d Solution: C(n) = [n(n+1)/2 1] b + d quadratic

Ex. Recursive Selectionsort

RecSelectionSort(A,n)
if n>1 then
e = FindLargestElement(A,n); A[0]e; RecSelectionSort(A,n-1);

Approximating the Order of Growth of the Recurrence Relation: Master Method Apply to Divide-and-Conquer Cases
If a problem of size n is solved recursively by diving the problem into a sub-problems, each of size n/b, and if the amount of work required to divided the problem and to combine the solutions is f(n) then we can say that:
C(n) = aC(n/b) + f (n)
n If a < bk then n If a = bk then n If a > bk then

where f (n) O(nk) then

b

C(n) is O(nk) C(n) is O(nk lg n ) C(n) is O(nlog a)

n Examples: n C(n) = C(n/2) + 1 n a=1, b=2, k=0, a = bk => C(n) is O(lg n)

n C(n) = 2C(n/2) + n n a=2, b=2, k=1, a = bk => C(n) is O(nlg n) n C(n) = 3 C(n/2) + n n a=3, b=2, k=1, a > bk => C(n) is O(nlog 3)
2

Adding all the Numbers in an Integer Array A: Binary Recursive Method

Algorithm BinaryRecSum(A, i, n): Input: An array A and integers i and n Output: The sum of the n integers in A starting at index i BinaryRecSum(A, 0, 5) = 59 if n = 1 then return A[i] return BinaryRecSum(A, i, Ceil(n/ 2)) + BinaryRecSum(A, i + ceil(n/ 2), floor(n/ 2)) C(n) = 2 C(n/2) + 1 a =2, b =2, ; f(n) = 1 ==> k = 0 ;

a > bk ==> C(n) since a > bk then 2 2) = O(n)

C(n) is O(nlog

Computing Fibonacci Numbers

n

Fibonacci numbers are defined recursively:

F0 = 0 F1 = 1 Fi = Fi-1 + Fi-2 for i > 1.

As a recursive algorithm (first attempt):

Algorithm BinaryFib(k): Input: Nonnegative integer k Output: The kth Fibonacci number Fk if k <= 1 then return k else return BinaryFib(k - 1) + BinaryFib(k - 2)

Fibonacci Numbers
n n

The Fibonacci sequence:

0, 1, 1, 2, 3, 5, 8, 13, 21,

Fibonacci recurrence: (use addition as basic operation)

C(n) = C(n-1) + C(n-2) or C(n) - C(n-1) - C(n-2) =0 C(0) = C(1) =0

Another example:
C(n) = C(n-1) 2C(n-2) C(0) = 1 C (1) = 3

Fibonacci Numbers Fibonacci Numbers: Iterative

unsigned int interativeFib (unsigned int n) { if (n<2) return n; else { register int i = 2, tmp, second = 1, first = 0; for (; i<=n, i++){ tmp = first + second; first = second; second = tmp; } return current; } }

Fibonacci Numbers Recursion or Iteration?

Tradeoffs between two options

Sometimes recursive solution is easier, always consistent with the logic of the original definition of the algorithm n Recursive solution is often slower, but not if the stack operation is done in hardware
n

Measuring the execution time sometime may help make a decision, especially when using as part of a real-time system n Recursion should be avoided if some part of the work is unnecessarily repeated to compute an answer, like in the case of Fibonacci
n

Backtracking Algorithm
In solving a problem, you might come a cross-point where you have to choose between different paths to find a solution. n Some/all of the paths might lead to no
n

solution n Backtracking attempts to find solutions to a problem by constructing partial solutions n Backtracking makes sure that any partial solution does not violate the problem requirements n Backtracking tries to extend partial solution towards completion n Recursion serve as a natural implementation of backtracking.

Backtracking Algorithm
n

If it is determined that partial solution would not lead to solution

partial solution would end in dead end n algorithm backs up by removing the most recently added part and then tries other possibilities
n

Solution to 8-Queens Puzzle 4-Queens Puzzle

Towers of Hanoi
n

In the temple of Banares, says he, beneath the

dome which marks the centre of the World, rests a brass plate in which are placed 3 diamond needles, each a cubit high and as thick as the body of a bee. On one of these needles, at the creation, god placed 64 discs of pure gold, the largest disc resting on the brass plate and the others getting smaller and smaller up to the top one. This is the tower of Brahma. Day and night unceasingly the priests transfer the discs from one diamond needle to another according to the fixed and immutable laws of Brahma, which require that the priest on duty must not move more than one disc at a time and that he must place this disc on a needle so that there is no smaller disc below it. When the 64 discs shall have been thus transferred from the needle on which at the creation god placed them to one of the other needles, tower, temple and Brahmans alike will crumble into dust and with a thunder clap the world will vanish.

Towers of Hanoi Relax If the legend were true, and

if the priests were able to move disks at a rate of one disk per second, using the smallest number of moves, it will take them 2641 seconds or roughly 584.542 billion years o move 64 disks.

Towers of Hanoi Problem

Invented by Edouard Lucas, in 1883 n Given a tower of n disks, initially stacked in increasing size on one of three pegs, the objective is to transfer the entire tower to one of the other pegs, moving only one disk at a time and never a larger one onto a smaller.
n

n n

See it at http://www.cut-theknot.org/recurrence/hanoi.shtml

Play it at http://www.mazeworks.com/hanoi/

Towers of Hanoi: Algorithm

Understand problem requirements n Identify base cases
n

Provide direct solution to each base case n Identify general case(s) n Provide solutions to general cases in terms of smaller versions of itself
n

Towers of Hanoi: Recursive Algorithm

void moveDisks(int count, int needle1, int needle3, int needle2) { if(count > 0) { moveDisks(count - 1, needle1, needle2, needle3); cout<<"Move disk "<<count<< from "<<needle1 << to "<<needle3<<"."<<endl; moveDisks(count - 1, needle2, needle3, needle1);

} }

Time Efficiency of Recursive Algorithms

Steps in mathematical analysis of recursive algorithms: n Decide on parameter n indicating input size n Identify algorithms basic operation n Determine worst , average and best case for input of size n
n Set up a recurrence relation and initial condition(s) for

C(n)-the number of times the basic operation will be executed for an input of size n (alternatively count recursive calls).
n Solve the recurrence to obtain a closed form or estimate

the order of magnitude of the solution. You can use by backward substitutions or another method.

Example: Tower of Hanoi

n

Recurrence function
C(n) = C(n-1) + 1 + C(n-1) = 2 C(n-1) + 1 n C(n) = 2 ( 2C(n-2) +1) + 1 = 22C(n-2) + 2 +1 : : n C(n) = = 2iC(n-i) + 2i-1 + 2i-2+1 n We arrive at the base case when i = n-1
n

Remember that : = 2n+1 1

0 i n

2i = 20 + 21 ++ 2n

n n

C(n) = 2iC(n-i) + 2i-1 C(n) = 2n-1+ 2n-1-1 = 2n-1 O(2n)

Questions
Questions
Questions
Questions
Questions
Questions