Substitution Method to solve Recurrence Relations

Recursion is a method to solve a problem where the solution depends on solutions to smaller subproblems of the same problem. Recursive functions (function calling itself) are used to solve problems based on Recursion. The main challenge with recursion is to find the time complexity of the Recursive function. In this article, we will learn about how to find the time complexity of Recursive functions using Substitution Method.

What is a Recurrence Relation?

Whenever any function makes a recursive call to itself, its time can be computed by a Recurrence Relation. Recurrence Relation is simply a mathematical relation/equation that can give the value of any term in terms of some previous smaller terms. For example,

T(n) = T(n-1) + N

It is a recurrence relation because the value of the nth term is given in its previous term i.e (n-1)the term.

Types of Recurrence Relation:

There are different types of recurrence relation that can be possible in the mathematical world. Some of them are-

1. Linear Recurrence Relation: In case of Linear Recurrence Relation every term is dependent linearly on its previous term. Example of Linear Recurrence Relation can be

T(n) = T(n-1) + T(n-2) + T(n-3)

2. Divide and Conquer Recurrence Relation: It the type of Recurrence Relation which is obtained from Divide and Conquer Algorithm. Example of such recurrence relation can be

T(n) = 3T(n/2) + 9n

3. First Order Recurrence Relation: It is the type of recurrence relation in which every term is dependent on just previous term. Example of this type of recurrence relation can be-

T(n) = T(n-1)²

(4) Higher Order Recurrence Relation- It is the type of recurrence relation where one term is not only dependent on just one previous term but on multiple previous terms. If it will be dependent on two previous term then it will be called to be second order. Similarly, for three previous term its will be called to be of third order and so on. Let us see example of an third order Recurrence relation

T(n) = 2T(n-1)² + KT(n-2) + T(n-3)

Till now we have seen different recurrence relations but how to find time taken by any recursive algorithm. So to calculate time we need to solve the recurrence relation. Now for solving recurrence we have three famous methods-

• Substitution Method
• Recursive Tree Method
• Master Theorem

Now in this article we are going to focus on Substitution Method.

Substitution Method:

Substitution Method is very famous method for solving any recurrences. There are two types of substitution methods-

1. Forward Substitution
2. Backward Substitution

1. Forward Substitution:

It is called Forward Substitution because here we substitute recurrence of any term into next terms. It uses following steps to find Time using recurrences-

• Pick Recurrence Relation and the given initial Condition
• Put the value from previous recurrence into the next recurrence
• Observe and Guess the pattern and the time
• Prove that the guessed result is correct using mathematical Induction.

Now we will use these steps to solve a problem. The problem is-

T(n) = T(n-1) + n, n>1

T(n) = 1, n=1

Now we will go step by step-

1. Pick Recurrence and the given initial Condition:

T(n)=T(n-1)+n, n>1T(n)=1, n=1

2. Put the value from previous recurrence into the next recurrence:

T(1) = 1T(2) = T(1) + 2 = 1 + 2 = 3T(3) = T(2) + 3 = 1 + 2 + 3 = 6T(4)= T(3) + 4 = 1 + 2 + 3 + 4 = 10

3. Observe and Guess the pattern and the time:

So guessed pattern will be-T(n) = 1 + 2 + 3 .... + n = (n * (n+1))/2Time Complexity will be O(n²)

4. Prove that the guessed result is correct using mathematical Induction:

• Prove T(1) is true:
  T(1) = 1 * (1+1)/2 = 2/2 = 1 and from definition of recurrence we know T(1) = 1. Hence proved T(1) is true
• Assume T(N-1) to be true:
  Assume T(N-1) = ((N - 1) * (N-1+1))/2 = (N * (N-1))/2 to be true
• Then prove T(N) will be true:T(N) = T(N-1) + N from recurrence definition
  Now, T(N-1) = N * (N-1)/2So, T(N) = T(N-1) + N = (N * (N-1))/2 + N = (N * (N-1) + 2N)/2 =N * (N+1)/2And from our guess also T(N)=N(N+1)/2Hence T(N) is true.Therefore our guess was correct and time will be O(N²)

2. Backward Substitution:

It is called Backward Substitution because here we substitute recurrence of any term into previous terms. It uses following steps to find Time using recurrences-

• Take the main recurrence and try to write recurrences of previous terms
• Take just previous recurrence and substitute into main recurrence
• Again take one more previous recurrence and substitute into main recurrence
• Do this process until you reach to the initial condition
• After this substitute the the value from initial condition and get the solution

Now we will use these steps to solve a problem. The problem is-

T(n) = T(n-1) + n, n>1T(n) = 1, n = 1

Now we will go step by step-

1. Take the main recurrence and try to write recurrences of previous terms:

T(n) = T(n-1) + nT(n-1) = T(n-2) + n - 1T(n-2) = T(n-3) + n - 2

2. Take just previous recurrence and substitute into main recurrence

put T(n-1) into T(n)So, T(n)=T(n-2)+ n-1 + n

3. Again take one more previous recurrence and substitute into main recurrence

put T(n-2) into T(n)So, T(n)=T(n-3)+ n-2 + n-1 + n

4. Do this process until you reach to the initial condition

So similarly we can find T(n-3), T(n-4)......and so on and can insert into T(n). Eventually we will get following: T(n)=T(1) + 2 + 3 + 4 +.........+ n-1 + n

5. After this substitute the the value from initial condition and get the solution

Put T(1)=1, T(n) = 1 +2 +3 + 4 +..............+ n-1 + n = n(n+1)/2. So Time will be O(N²)

Limitations of Substitution method:

The Substitution method is a useful technique to solve recurrence relations, but it also has some limitations. Some of the limitations are:

• It is not guaranteed that we will find the solution as substitution method is based on guesses.
• It doesn't provide guidance on how to make an accurate guess, often relying on intuition or trial and error.
• It may only yield a specific or approximate solution rather than the most general or precise one.
• The substitution method isn't universally applicable to all recurrence relations, especially those with complex or variable forms that do not get simplified using substitution.