UNIT-2

Divide and Conquer Algorithm

 Divide and Conquer is an algorithmic pattern.
 In algorithmic methods, the design is to take a dispute on a huge
input, break the input into minor pieces, decide the problem on each of
the small pieces, and then merge the piecewise solutions into a global
solution.
 This mechanism of solving the problem is called the Divide &
Conquer Strategy

Divide and Conquer algorithm consists of a dispute using

the following three steps.
1. Divide the original problem into a set of subproblems.
2. Conquer: Solve every subproblem individually, recursively.
3. Combine: Put together the solutions of the subproblems to get
the solution to the whole problem.

Let us understand this concept with the help of an example.

Here, we will sort an array using the divide and conquer approach
(ie. merge sort).

Let the given array be:

2 Divide the array into two halves

Again, divide each subpart recursively into two halves until you get
individual elements.

3.Now, combine the individual elements in a sorted manner.

Here, conquer and combine steps go side by side.
Time Complexity

The complexity of the divide and conquer algorithm is calculated using

the master theorem.

T(n) = aT(n/b) + f(n),

where,

n = size of input

a = number of subproblems in the recursion

n/b = size of each subproblem. All subproblems are assumed to have the
same size.

f(n) = cost of the work done outside the recursive call, which includes
the cost of dividing the problem and cost of merging the solutions
Let us take an example to find the time complexity of a recursive problem.

For a merge sort, the equation can be written as:

T(n) = aT(n/b) + f(n)

= 2T(n/2) + O(n)

Where,

a = 2 (each time, a problem is divided into 2 subproblems)

n/b = n/2 (size of each sub problem is half of the input)

f(n) = time taken to divide the problem and merging the subproblems

T(n/2) = O(n log n) (To understand this, please refer to the master
theorem.)

Now, T(n) = 2T(n log n) + O(n)

≈ O(n log n)

Applications of Divide and Conquer Approach:

1 Binary Search: The binary search algorithm is a searching algorithm, which is
also called a half-interval search or logarithmic search.
It works by comparing the target value with the middle element existing in a
sorted array. After making the comparison, if the value differs, then the half that
cannot contain the target will eventually eliminate, followed by continuing the
search on the other half.
We will again consider the middle element and compare it with the target value.

2.Quicksort: It is the most efficient sorting algorithm, which is also known as

partition-exchange sort.
It starts by selecting a pivot value from an array followed by dividing the rest of
the array elements into two sub-arrays.
The partition is made by comparing each of the elements with the pivot value
3.Merge Sort: It is a sorting algorithm that sorts an array by making
comparisons. It starts by dividing an array into sub-array and then recursively
sorts each of them. After the sorting is done, it merges them back.

4.Strassen's Algorithm: It is an algorithm for matrix multiplication,

which is named after Volker Strassen. It has proven to be much faster
than the traditional algorithm when works on large matrices.

Advantages of Divide and Conquer

 This approach is suitable for multiprocessing systems.

 It makes efficient use of memory caches.

 It is more proficient than that of its counterpart Brute Force

technique.

Disadvantages of Divide and Conquer

o Since most of its algorithms are designed by incorporating recursion, so it
necessitates high memory management.
o An explicit stack may overuse the space.
o It may even crash the system if the recursion is performed rigorously
greater than the stack present in the CPU.
Probabilistic analysis and randomized algorithms
Consider the problem of hiring an office assistant.

We interview candidates on a rolling basis, and at any given point we want to hire the best candidate
we’ve seen so far. If a better candidate comes along, we immediately fire the old one and hire the
new one.

Hire-Assistant(n)
1 best=0 // candidate 0 is a least -qualified
2 for i=1 to n
3 Interview candidate i
4 If candidate I is better than other candidate best
5 Best=i
6 Hire candidate i

What is the cost of this strategy?

 If we interview n candidates and hire m of them, cost is Θ(cin + chm)

 We interview all n and ci is small, so we focus on chm.
 chm varies with each run and depends on interview order
 This is a common paradigm: finding the maximum or minimum in a
sequence by examining each element, and changing the winner m times.

Best Case

If each candidate is worse than all who came before, we hire one candidate:
Θ(cin + ch) = Θ(cin)

Worst Case

If each candidate is better than all who came before, we hire all n (m = n):
Θ(cin + chn) = Θ(chn) since ch > ci
But this is pessimistic. What happens in the average case? To figure this out, we
need some tools ...

Probabilistic Analysis

 We must know or make assumptions about the distribution of inputs.

 The expected cost is over this distribution.
 The analysis will give us average case running time.
We don't have this information for the Hiring Problem, but suppose we could
assume that candidates come in random order. Then the analysis can be done by
counting permutations:

 Each ordering of candidates (relative to some reference ordering such as a

ranking of the candidates) is equally likely to be any of the n! permutations
of the candidates.
 In how of these permutations many do we hire once? twice? three
times? ... n−1 times? n times?
 It depends on how many permutations have zero, one two ... n−2 or n−1
candidates that come before a better candidate.
 This is complicated! You would not want to count those up.
 Fortunately, this analysis is much easier with indicator variables (next
section)

Randomized Algorithms
 We might not know the distribution of inputs or be able to model it.
 However, uniform random distributions are easier to analyze than unknown
distributions.
 To obtain such a distribution regardless of input distribution, we
can randomize within the algorithm to impose a distribution on the inputs.
Then it is easier to analzye expected behavior.
 An algorithm is randomized if its behavior is determined in parts by values
provided by a random number generator.

This requires a change in the hiring problem scenario:

 The employment agency sends us a list of n candidates in advance and lets
us choose the interview order.
 We choose randomly.

Thus we take control of the question of whether the input is randomly

ordered: we enforce random order, so the average case becomes the expected
value.
 The maximum-subarray problem Using Divide &
Conquer Method.
The problem of maximum subarray sum is basically finding the part of an
array whose elements has the largest sum.
If all the elements in an array are positive then it is easy, find the sum of all
the elements of the array and it has the largest sum over any other
subarrays you can make out from that array.

Maximum Sum
3+5+1+7+9
=25
But the problem gets more interesting when some of the elements are
negative then the subarray whose sum of the elements is largest over the sum
of the elements of any other subarrays of that element can lie anywhere in the
array.

Maximum Sum
=3-1-1+10-3-2+4
=11
 Maximum Subarray Sum Using
Divide and Conquer
As we know that the divide and conquer solves a problem by breaking into
subproblems, so let's first break an array into two parts. Now, the subarray
with maximum sum can either lie entirely in the left subarray or entirely in
the right subarray or in a subarray consisting both i.e., crossing the middle
element.

 The first two cases where the subarray is entirely on right or on the
left are actually the smaller instances of the original problem
 So, we can solve them recursively by calling the function to calculate
the maximum sum subarray on both the parts.

We are making max_sum_subarray is a function to calculate the maximum

sum of the subarray in an array. Here, we are calling the
function max_sum_subarray for both the left and the right subarrays i.e.,
recursively checking the left and the right subarrays for the maximum sum
subarray.
 max_sum_subarray(array, low, high)
 {
 if (high == low) // only one element in an array
 {
 return array[high]
 }
 mid = (high+low)/2

 max_sum_subarray(array, low, mid)
 max_sum_subarray(array, mid+1, high)
 }

Strassen’s algorithm for matrix multiplication

Matrix A has a size N*M and matrix B has a size A*B. Given two matrices are
multiplied only when the number of columns in the first matrix is equal to the
number of rows in the second matrix

Therefore, we can say that matrix multiplication is possible only when M==A. The given two
matrices are: Matrix A of the size: 3 × 3
Complexity Analysis

Time Complexity: O(N ^3) , where the given matrices are square matrices of size N*N
each.

 For multiplying every column with every element in the row from the given matrices
uses two loops, and adding up the values takes another loop. Therefore the overall
time complexity turns out to be O(N^3^).

Space Complexity: O(N^2), where the given matrices are square matrices of size N*N each.

 The matrix of size N*N has been used to store the result when the two matrices are
multiplied.
Strassen’s Matrix Multiplication Algorithm
 In this context, using Strassen’s Matrix multiplication algorithm, the
time consumption can be improved a little bit.
 It reduces the time complexity for matrix multiplication.
 Strassen’s Matrix multiplication can be performed only on square matrices
where N is a power of 2. And also the order of both of the matrices is N × N.
 The main idea is to use the divide and conquer technique in this algorithm.
 We need to divide matrix A and matrix B into 8 submatrices and then
recursively compute the submatrices of the result.
 Strassen’s Matrix multiplication can be performed only on square
matrices where n is a power of 2. Order of both of the matrices
are n × n
 To optimize the matrix multiplication Strassen's matrix multiplication algorithm can
be used. It reduces the time complexity for matrix multiplication. Strassen’s Matrix
multiplication can be performed only on square matrices where N is a power of 2.
And also the order of both of the matrices is N × N.
 The main idea is to use the divide and conquer technique in this algorithm. We need
to divide matrix A and matrix B into 8 submatrices and then recursively compute the
submatrices of the result.

This can be clearly explained in the below image:

 For each multiplication of size N2×N2, follow the below recursive function
as shown in the figure:

Where the resultant matrix is C and can be obtained in the following way. Now we
need to store the result as follows:
 The recurrence relation obtained is: T(N)=8T(N2) +4 O(N2)
 Here O(N^2) term is for matrix addition. Since, addition is performed 4 times during
the recursion, Therefore, it is 4 times O(N2).
 The recurrence relationship can be simplified using the Master's Theorem.
 And the overall time complexity turns out to be O(N3) which is not better than the
naive method which is discussed earlier.
 To optimize it further, we use Strassen’s Matrix Multiplication where we don't
need 8 recursive calls, as we can solve them using 7 recursive calls and this requires
some manipulation which is achieved using addition and subtraction.
Algorithm: Strasen’s Algorithm.

1. Divide the given matrix A and matrix B into 4 sub-matrices of size each N2xN2as
shown in the above diagram.

A. Now, compute Strassen’s 7 calls recursively.

B. Calculate the submatrices of C.
C. Now, we combine these submatrices into our new resultant matrix .