UNIT 2

Brute Force and Exhaustive Search

• Brute force is a direct problem-solving approach based on the problem statement and
concept definitions.

• It emphasizes computational power rather than intellectual insight.

• Often described as "Just do it!" as it directly follows the problem's definition.

Example: Exponentiation Problem:

• Problem Statement:
• Illustrate brute force with the exponentiation problem: computing an for a
(nonzero) and n (nonnegative integer).

• Brute Force Approach:

• Directly compute an by multiplying a by itself n times.
• Examples encountered earlier include consecutive integer checking for GCD and
definition-based matrix multiplication.

• Encourages readers to identify algorithms they already know following the brute-force
approach.

Importance of Brute Force:

• Practical Value:

• For critical problems like sorting, searching, matrix multiplication, and string
matching, brute force yields practical algorithms, especially for small instances.

• It provides reasonable solutions regardless of the instance size.

• Handling Small Problems:

• Despite general inefficiency, brute force is useful for solving small-size
instances efficiently.

• It can offer practical solutions for specific cases.

 Brute Force Approach for Sorting
Selection Sort
Basic Idea:

• Start with scanning the entire list to find the smallest element and exchange it with the
first element.

• Iterate through the list, finding the smallest among the remaining elements and placing
it in its final position.

• Repeat this process until the entire list is sorted.

ALGORITHM
SelectionSort(A[0..n−1])
for i ← 0 to n−2 do
min ← i
for j ← i+1 to n−1 do
if A[j] < A[min] min ← j
swap A[i] and A[min]

Analysis:

• Input size is n, and the basic operation is key comparison A[j]<A[min].

• The number of key comparisons is given by 𝐶(𝑛) = ∑𝑛−2 𝑛−1

𝑖=0 ∑𝑗=𝑖+1 1

𝑛(𝑛−1)
• Solving the sum: 𝐶(𝑛) = making selection sort an O(n2) algorithm
2

𝑛−2 𝑛−1

𝐶(𝑛) = ∑ ∑ 1
𝑖=0 𝑗=𝑖+1

= ∑𝑛−2
𝑖=0 (𝑛 − 1 − 𝑖)

𝑛(𝑛−1)
= 2
Example:
1. Initial Array: [5, 3, 1, 4, 2]
2. Pass 1:
1. Swap elements at positions 0 and 2. (1 key swap)
2. Array becomes: [1, 3, 5, 4, 2]
3. Pass 2:
1. Swap elements at positions 1 and 4. (1 key swap)
2. Array becomes: [1, 2, 5, 4, 3]
4. Pass 3:
1. Swap elements at positions 2 and 3. (1 key swap)
2. Array becomes: [1, 2, 3, 4, 5]
5. Pass 4:
1. Swap elements at positions 3 and 3. (No key swaps)
2. Array remains: [1, 2, 3, 4, 5]
6. Pass 5:
1. No swaps needed as the array is already sorted.
● Total Key Swaps: In this example, the total number of key swaps is 3. In general, for
an input size 'n', the number of key swaps in the selection sort algorithm is O(n), as the
number of swaps scales linearly with the size of the input array.

● The analysis shows that the number of key comparisons made by the selection sort
algorithm grows quadratically with the size of the input, making it a O(n2) algorithm.

● While the number of key comparisons is O(n2), the number of key swaps is O(n),
making it more efficient in this aspect compared to some other sorting algorithms.

● The space complexity of the selection sort algorithm is O(1), commonly referred to as
constant space. Regardless of the size of the input array, the additional memory used by
the algorithm remains constant.
Bubble Sort
Basic Idea:

• Compare adjacent elements of the list.

• If they are out of order, swap them.

• Repeat this process until the entire list is sorted.

ALGORITHM BubbleSort(A[0..n−1])
// Sorts a given array by bubble sort
// Input: An array A[0..n − 1] of orderable elements
// Output: Array A[0..n − 1] sorted in non-decreasing order
for i ← 0 to n−2 do
for j ← 0 to n−2−i do
if A[j + 1] < A[j] swap A[j] and A[j + 1]

Analysis:

• The number of key comparisons is given by

𝑛−2 𝑛−2−𝑖

𝐶(𝑛) = ∑ ∑ 1
𝑖=0 𝑗=0

= ∑𝑛−2
𝑖=0 [(𝑛 − 2 − 𝑖) − 0 + 1]

= ∑𝑛−2
𝑖=0 (𝑛 − 1 − 𝑖)

𝑛(𝑛−1)
= ∈ 𝑂(𝑛2)
2

• The algorithm grows quadratically with the size of the input.

Number of Key Swaps:

• The number of key swaps (Sworst(n)) is also O(n2), making it inefficient for worst-case
inputs.
 • Although the basic version of bubble sort is not efficient, it can often be
improved with modest effort.

• Specifically, the algorithm can be enhanced by stopping early if a pass makes

no exchanges (sorted list). This optimization reduces the running time on some
inputs.
Number of Key Swaps:
1. Bubble Sort Algorithm:
1. Bubble sort involves swapping adjacent elements if they are out of order.
2. The goal is to move the larger elements towards the end of the list with each
pass.
2. Worst-Case Scenario:
1. Consider the worst-case scenario where the input array is in decreasing order.
2. In this case, every pair of adjacent elements will be out of order in each pass.
3. Inefficiency for Worst-Case Inputs:
1. The O(n2) complexity for the number of key swaps implies that, in the worst
case, the number of swaps grows quadratically with the size of the input array.
2. Quadratic growth is generally considered inefficient for large datasets.
Example:
3. For instance, if the input array is [5, 4, 3, 2, 1], bubble sort would perform
5×4/2=10 swaps in the first pass, 4×3/2=6 swaps in the second pass, and so on.
Let's analyze the number of swaps in each pass of the Bubble Sort algorithm for the
given example array [5, 4, 3, 2, 1].
Pass 1:

• Compare and swap 5 and 4 (1 swap)

• Compare and swap 5 and 3 (2 swaps)

• Compare and swap 5 and 2 (3 swaps)

• Compare and swap 5 and 1 (4 swaps)
Total swaps in Pass 1+2+3+4=10 swaps
Pass 2:

• Compare and swap 4 and 3 (1 swap)

• Compare and swap 4 and 2 (2 swaps)
 • Compare and swap 4 and 1 (3 swaps)
Total swaps in Pass 2: 1+2+3=6 swaps
Pass 3:

• Compare and swap 3 and 2 (1 swap)

• Compare and swap 3 and 1 (2 swaps)

Total swaps in Pass 3:1+2=3 swaps
Pass 4:

• Compare and swap 2 and 1 (1 swap)

Total swaps in Pass 4: 1 swap
Now, let's calculate the total number of swaps: 10+6+3+1=20

● So, for an input array of length 5, Bubble Sort would perform a total of 20 swaps. If we
generalize this for an array of size n, the number of swaps in each pass forms a triangular
number pattern, leading to a quadratic growth concerning the size of the input array.

● This quadratic growth is reflected in the O(n2) time complexity of the Bubble Sort
algorithm.

Enhanced version

• Consider the input list: [1, 2, 3, 4, 5]. In the enhanced version, the algorithm would
detect that the list is already sorted after the first pass (no swaps were made), and it
would terminate early without unnecessary additional passes.
• This enhancement helps to improve the efficiency of bubble sorting in cases where the
list is nearly sorted or already sorted.

Comparison:

• Both selection sort and bubble sort have the same worst-case time complexity O(n2).
• The number of key swaps for bubble sort can be higher than that of selection sort,
making bubble sort less efficient in terms of swaps.
Efficiency Based on Brute Force:
• In terms of key swaps, selection sort is more efficient than bubble sort, especially for
worst-case inputs.

• However, both algorithms are considered inefficient for large datasets compared to
more advanced sorting algorithms.
 Brute Force Approach for Searching

Sequential Search
Sequential search is a straightforward algorithm for finding a target element in a list. It checks
each element in the list one by one until a match is found or the entire list is searched.

ALGORITHM SequentialSearch2(A[0..n], K)
A[n] ← K // Append the search key to the end of the list
i←0
while A[i] ≠ K do
i←i+1
if i < n then
return i
else
return −1

Explanation:
1. Initialization: Append the search key K to the end of the array A to act as a sentinel.
2. Search Loop: Use a while loop to iterate through the array elements until a match is
found (i.e., A[i]=K).
3. Check: If a match is found (i.e., i<n), return the index i where the match occurred.
4. Unsuccessful Search: If the loop completes without finding a match, return -1.
Enhancement for Sorted Lists:

• If the list is sorted, the search can be optimized by stopping as soon as an element
greater than or equal to the search key is encountered.
Efficiency:

• The efficiency of sequential search remains linear (O(n)) in both the worst and average
cases. While it is simple, its efficiency can be inferior, especially for large datasets.
Sentinel Technique:

• Appending the search key as a sentinel simplifies the algorithm by eliminating the need
for an additional check to reach the end of the list.
• This algorithm provides a basic illustration of the brute-force approach, emphasizing
simplicity but not necessarily the most efficient performance.

• A sentinel, in the context of algorithms and data structures, is a special value that is
used to represent the end of a data structure or to simplify algorithm implementation.

• It serves as a signal or marker, often taking a unique value that is unlikely to be confused
with actual data.
Example Array: A = [8, 3, 5, 2, 1, 5]

● Execution:
1. The loop starts with i = 0 and checks if A[0] != 5 (since A[0] = 8). The condition is true,
so it increments i.
2. Now, i = 1, and it checks if A[1] != 5 (since A[1] = 3). The condition is true, so it
increments i.
3. i = 2, and it checks if A[2] != 5 (since A[2] = 5). The condition is false, so the loop exits.
4. Element 5 is found at index 2.

● After finding element 5 at index 2, the algorithm exits the loop. In this case, the loop
doesn't iterate through all the elements, as it terminates once the target element is found.
The sentinel technique allows us to guarantee that the loop will always find the element
without needing an explicit check for the end of the array.

● If the element being searched for is already present at the end of the array, then indeed
the sentinel technique wouldn't work effectively because the loop would traverse the
entire array until it reaches the last element. In such a case, appending a sentinel
wouldn't provide any benefit, and it would essentially result in the algorithm iterating
through the entire array unnecessarily.

Brute-Force String Matching

ALGORITHM BruteForceStringMatch(T[0..n - 1], P[0..m - 1])
// Implements brute-force string matching
// Input: An array T[0..n - 1] of n characters representing a text and
// an array P[0..m - 1] of m characters representing a pattern
// Output: The index of the first character in the text that starts a
// matching substring or -1 if the search is unsuccessful
for i ← 0 to n - m do
j←0
while j < m and P[j] = T[i + j] do
j←j+1
if j = m return i
return -1

Algorithm Overview:
Given a text T of length n and a pattern P of length m, the algorithm aims to find the index of
the first occurrence of P within T.
If a match is found, the algorithm returns the index of the starting character of the matching
substring. If no match is found, it returns -1.
Looping through Text:
The algorithm iterates over each character T[i] of the text T, where i ranges from 0 to n - m.
This ensures that the algorithm doesn't exceed the boundary where the remaining characters in
T are fewer than the characters in P.
● Let's illustrate this with the example of aligning the pattern "DEF" with the text
"ABCDEFGHIJ".

● Text: ABCDEFGHIJ
● Pattern: DEF

● When we subtract the length of the pattern from the length of the text, we get:
● 10−3=7
● This means there are 7 positions in the text where we can potentially place the pattern
without it extending beyond the end of the text. These positions are:

●
Certainly! Let's illustrate this with the example of aligning the pattern "DEF" with the
text "ABCDEFGHIJ".

● Text: ABCDEFGHIJ
● Pattern: DEF

● Here are the 7 possible starting positions for aligning the pattern within the text: n-m =
10-3=7
● Now, let's examine what happens at each position:
 1. Starting position: A Comparison: ABC vs DEF
2. Starting position: B Comparison: BCD vs DEF
3. Starting position: C Comparison: CDE vs DEF
4. Starting position: D Comparison: DEF vs DEF (Match!)
5. Starting position: E Comparison: EFG vs DEF
6. Starting position: F Comparison: FGH vs DEF
7. Starting position: G Comparison: GHI vs DEF

● As you can see, while the starting positions change, the comparisons between the
pattern and the corresponding substring of the text are essentially the same. In position
4, we have a match between the pattern "DEF" and the substring "DEF" of the text.
This is the same comparison that occurs at position 1, just shifted one position to the
right.

● Therefore, even though there are technically 7 different starting positions, they all lead
to the same comparisons between the pattern and the corresponding substring of the
text. This is why we add 1 to (n−m) to account for the possibility of starting the pattern
match at the very last position of the text.

● The addition of 1 to (n−m) accounts for the possibility of starting the pattern match at
the very last position of the text, which in this case is the position where "J" is located.
So, when we say there are (n−m+1) possible starting positions, the additional 1
represents the possibility of starting the pattern match at the end of the text.

Worst-case scenario: In the worst-case scenario, the algorithm may have to perform m
comparisons for each of the (n - m + 1) tries. Here, n is the length of the text, and m is the
length of the pattern.

• For our example:

• n = 10 (length of text)

• m = 3 (length of pattern)
We have 10 - 3 + 1 = 8 possible starting positions for the pattern within the text.
In the worst case, the algorithm may have to perform 3 comparisons for each starting position,
leading to a total of 3 * 8 = 24 comparisons.
This demonstrates the worst-case time complexity of m* (n-m+1) = O(nm).

Average-Case Scenario: In typical scenarios, shifts occur after very few comparisons, leading
to a much better average-case time complexity of O(n).
 ● For example, n this case, the text length (n) is 10, and the pattern length (m) is 3.
● The input size can be expressed as a combination of both the text length and the pattern
length, such as (n, m) or (10, 3).
As the text and pattern lengths increase, the input size increases accordingly.
The number of comparisons made by the algorithm grows linearly with the size of the input.

The best-case scenario for the Brute-Force String Matching algorithm occurs when the pattern
is found at the very beginning of the text.

• In this case, the algorithm only needs to perform a single comparison between each
character of the pattern and the corresponding character in the text.
• Therefore, the best-case time complexity is O(m), where m is the length of the pattern.

Closest-Pair and Convex-Hull Problems by Brute Force

We consider a straightforward approach to two well-known problems dealing with a finite set
of points in the plane. These problems, aside from their theoretical interest, arise in two
important applied areas: computational geometry and operations research

Closest-Pair Problem
ALGORITHM BruteForceClosestPair(P)
// Finds the distance between the two closest points in the plane by brute force
// Input: A list P of n (n≥ 2) points p1(x1, y1), ..., pn(xn, yn)
// Output: The distance between the closest pair of points
d←∞ // Initialize minimum distance to infinity
for i ← 1 to n - 1 do
for j ← i + 1 to n do
d ← min(d, sqrt((xi - xj)^2 + (yi - yj)^2))
// Update minimum distance
return d // Return the minimum distance

1. Outer Loop (for i ← 1 to n - 1):

1. The purpose of the outer loop is to select each point in the list once.
 2. The loop starts from i=1 because the algorithm intends to compare each point
with all subsequent points.
3. If we let the outer loop run until i=n, it would mean selecting each point
including the last one, but there would be no subsequent points to compare it
with, which is unnecessary.
2. Inner Loop (for j ← i + 1 to n):
1. The inner loop starts from j=i+1 to ensure that we don't compare a point with
itself (which would be redundant) and also avoid comparing the same pair of
points more than once.
2. By starting from j=i+1, we ensure that for each i, the inner loop considers only
the subsequent points in the list. This prevents redundant comparisons between
pairs of points, thus optimizing the algorithm's efficiency.

● In summary, by using an outer loop from i=1 to n−1 and an inner loop from j=i+1 to n,
the algorithm efficiently considers each pair of points exactly once, avoiding redundant
comparisons and optimizing the search for the closest pair.
Initialization: Set the initial value of the minimum distance d to infinity. This is because
any actual distance between points will be smaller than infinity.
Nested Loop:

● Iterate over each pair of points in the list P.

● The outer loop (for i ← 1 to n - 1) iterates over each point in the list except for the last
one.

● The inner loop (for j ← i + 1 to n) iterates over the points following the current point
i.
Distance Calculation: For each pair of points (pi, pj), calculate the Euclidean distance
between them using the formula:

● 𝒅𝒊𝒔𝒕𝒂𝒏𝒄𝒆 = √(𝒙𝒊 − 𝒙𝒋)𝟐 + (𝒚𝒊 − 𝒚𝒋)𝟐

● Here, (xi, yi) and (xj, yj) are the coordinates of points pi and pj respectively.
● Update Minimum Distance: Compare the calculated distance with the current
minimum distance d. If the calculated distance is smaller than d, update d to be the
calculated distance.
● Return Minimum Distance: After completing the loop, return the final value of d,
which represents the distance between the two closest points in the list.
Example: P = [(1, 2), (3, 4), (5, 6), (7, 8)]
1. Initialize d to infinity.
2. Iterate over each pair of points:

1. For (1, 2) and (3, 4), calculate the distance: √(𝟏 − 𝟑)𝟐 + (𝟐 − 𝟒 )𝟐 = √8

2. For (1, 2) and (5, 6), calculate the distance: √(𝟏 − 𝟓)𝟐 + (𝟐 − 𝟔 )𝟐 = √32

3. For (1, 2) and (7, 8), calculate the distance: √(𝟏 − 𝟕)𝟐 + (𝟐 − 𝟖 )𝟐 = √72

4. For (3,4) and (5, 6), calculate the distance: √(𝟑 − 𝟓)𝟐 + (𝟒 − 𝟔 )𝟐 = √8

5. For (3,4) and (7, 8), calculate the distance: √(𝟑 − 𝟕)𝟐 + (𝟒 − 𝟖 )𝟐 = √32

6. For (5, 6) and (7, 8), calculate the distance: √(𝟓 − 𝟕)𝟐 + (𝟔 − 𝟖 )𝟐 = √8

3. Return the minimum distance ‘d’, which is √8

4. In this example, the closest points are (1, 2) and (3, 4), (3,4) and (5, 6), (5, 6) and (7,
8), the distance between them is √8 = 2.8284

Time Complexity:

• The algorithm consists of two nested loops iterating over the list of points.

• The outer loop runs from i=1 to n−1, and the inner loop runs from j=i+1 to n.
• The number of iterations for the outer loop is n−1, and for the inner loop, it decreases
by 1 for each increase in i, starting from n and going down to 1.

• Therefore, the total number of iterations is the sum of the first n−1 integers, which is
given by the arithmetic series formula:

∑𝑛−1 𝑛(𝑛−1)
𝑖=1 = 2 = O(n2)

Space Complexity:

• The algorithm uses a constant amount of extra space to store the minimum distance d
and the coordinates of the closest points.

• Hence, the space complexity is constant, denoted as O(1).

Convex-Hull Problem
The convex hull of a set of points in an Euclidean space is the smallest convex shape (such as
a polygon or a polyhedron) that encloses all the points in the set.
Observation:

• The boundary of a convex hull is made up of line segments connecting pairs of points.
• A line segment connecting two points pi and pj is part of the convex hull's boundary if
and only if all other points in the set lie on the same side of the straight line through
these two points.

• For each pair of points, we need to check if all other points lie on the same side of the
line segment connecting them.

• Vertices of the polygon are called as extreme points

Implementation:

• Define the straight line through two points (x1,y1) and (x2,y2) using the equation
ax+by=c, where:
• a=y2−y1
• b=x1−x2
• c=x1y2−y1x2

• Use this line equation to divide the plane into two half-planes: one where ax+by>c and
the other where ax+by<c.

• Check whether all other points lie on the same side of the line segment by evaluating
the expression ax+by−c for each point.
If all points have the same sign when this expression is evaluated, they lie on the same side of
the line segment.
Example:
Suppose we have the following set of points P in the plane: P={(1,1),(2,3),(3,2),(4,5),(5,4)}
We want to find the convex hull of these points.
1. Selecting pairs of points: We start by selecting each pair of distinct points from P. For
example, one pair could be (1,1) and (2,3).
2. Defining the line through the points: For each pair of points (pi,pj), we define the line
passing through them using the equation of a line:
ax+by=c
where:

● a=yj−yi = 3−1=2
● b=xi−xj = 1−2=−1
● c=xiyj−xjyi = 1×3−2×1=1
So, the equation of the line passing through (1,1) and (2,3) is: 2x−y=1
3. Dividing the plane into two half-planes: This line divides the plane into two half-planes:
1. One where 2x−y>1
2. The other where 2x−y<1
4. Checking points on the same side: Now, for each remaining point in P (e.g., (3,2)), we
substitute its coordinates into the equation 2x−y−1 to check which side of the line it lies on.
1. For (3,2), 2×3−2−1=6−2−1=3, which is greater than 1. So, it lies on the side
where 2x−y>1.
2. For (4,5), 2×4−5−1=8−5−1=2, which is greater than 1. So, it lies on the same
side.
3. For (5,4), 2×5−4−1=10−4−1=5, which is greater than 1. So, it also lies on the
same side.
Since all points lie on the same side, the line segment between (1,1) and (2,3) is part of the
convex hull.
5. Checking all points: Repeat step 4 for all remaining points in P. If all points lie on the same
side of the line, then the line segment between (pi,pj) is part of the convex hull.
Now, we repeat the process for the next pair of points, say (1,1) and (3,2), and then for all other
pairs of points.
6. Repeat for other pairs: Repeat steps 2-5 for each pair of points.
By repeating this process for all pairs of points, we can identify the line segments that form the
boundary of the convex hull.
Time complexity: Since there are O(n2) pairs of points, and for each pair, we perform O(n−2)
comparisons, the total number of comparisons becomes O(n2)×O(n−2)=O(n3).
The space complexity of the brute-force algorithm for computing the convex hull is typically
considered to be O(1), which means it uses a constant amount of extra space.

Exhaustive Search

It involves systematically generating and evaluating every possible candidate solution within
the problem domain.
Although conceptually simple, implementing exhaustive search often requires algorithms for
generating specific combinatorial objects, such as permutations, combinations, or subsets.
Traveling Salesman Problem
The Traveling Salesman Problem (TSP) is a classic optimization problem where the objective
is to find the shortest possible tour that visits each city exactly once and returns to the starting
city.
1. Modeling the Problem:
1. The TSP can be represented as a weighted graph, where each vertex represents
a city, and the edges between vertices represent the distances between cities.
2. The objective is to find the shortest Hamiltonian circuit in the graph, which is a
cycle that visits each vertex exactly once.
2. Generating Tours:
1. To find all possible tours, we need to generate all permutations of the
intermediate cities.
2. Assume we have n cities, labeled 1,2,3,...,n, excluding the starting city (which
is also the ending city).
3. Computing Tour Lengths:
1. For each permutation (potential tour), calculate the total length of the tour by
summing the distances between consecutive cities.
2. This involves traversing the edges of the graph according to the permutation
and summing the weights (distances) of the edges.
4. Finding the Shortest Tour:
1. Compare the lengths of all generated tours and select the tour with the shortest
length as the optimal solution.
2. This shortest tour represents the shortest route that visits each city exactly once
and returns to the starting city.

3 ● P-Q-R-S-P = 22
P Q ● P-Q-S-R-P = 15

● P-R-Q-S-P = 27
8 9
4 ● P-R-S-Q-P = 15
● P-S-Q-R-P = 27

● P-S-R-Q-P = 22
R S
2
Efficiency Considerations:

• The number of permutations of n cities (excluding the starting city) is (n−1)!.

• Even for moderate values of n, the number of permutations becomes prohibitively large.

• For example, with just 10 cities, there are 9!=362,8809!=362,880 possible tours to
consider, making exhaustive search impractical for larger instances.

• The time complexity of the exhaustive search approach for solving the Traveling
Salesman Problem (TSP) is O((n−1)!, where n is the number of cities (excluding the
starting city).
Limitations:

• Despite reducing the number of permutations, exhaustive search remains impractical

for large instances due to the exponential growth in the number of possible tours.

Knapsack Problem
The Knapsack Problem involves selecting a subset of items from a given set of items with
known weights and values, such that the total weight of the selected items does not exceed a
given capacity, and the total value of the selected items is maximized.
1. Generate all subsets: The exhaustive search approach starts by generating all possible
subsets of the given set of items. For example, if there are n items, there will be 2n
possible subsets.
2. Compute subset properties: For each generated subset, compute the total weight and
total value of the items in that subset. This involves summing up the weights and values
of the individual items in the subset.
3. Identify feasible subsets: Check if each subset satisfies the constraint of not exceeding
the knapsack capacity. Only consider feasible subsets, i.e., their total weight does not
exceed the knapsack capacity.
4. Find the subset with the maximum value: Among the feasible subsets, find the subset
with the maximum total value. This subset represents the most valuable set of items
that can fit into the knapsack without exceeding its capacity.
Let's consider an example to illustrate this approach:
Given items: 4 items

• Item 1: Weight = 7, Value = $42

• Item 2: Weight = 3, Value = $12

• Item 3: Weight = 4, Value = $40

• Item 4: Weight = 5, Value = $25
● Knapsack capacity: 10
● Generate all subsets:

● Possible subsets include {}, {1}, {2}, {3}, {4}, {1,2}, {1,3}, {1,4}, {2,3},
{2,4}, {3,4}, {1,2,3}, {1,2,4}, {1,3,4}, {2,3,4}, {1,2,3,4}

● Compute subset properties:

● For each subset, calculate the total weight and total value.
● Identify feasible subsets:
● Check if the total weight of each subset does not exceed the knapsack capacity
(10).

● Find the subset with maximum value:

● Among the feasible subsets, find the one with the maximum total value.

● These calculations show the total weight and total value for each subset. Feasible
subsets, i.e., those with a total weight not exceeding the knapsack capacity, are
marked as feasible.

● The subset {3,4} has the maximum total value of $65 and is feasible within the
knapsack capacity.
 ● The time complexity of the brute-force approach to the knapsack problem is
exponential, specifically Ω(2n), where n is the number of items. This is because we
need to generate and evaluate all possible subsets of the items, and there are 2n such
subsets.

● As for space complexity, it also grows exponentially with the number of items since
we need to store information about all possible subsets. Therefore, the space
complexity is also Ω(2n).
In the provided example of the knapsack problem, there are 4 items given. Therefore, n =
4.

● Now, let's calculate 2^n: 2^4 = 16

● So, there are 16 possible subsets of the given items.

Assignment Problem
The assignment problem involves efficiently matching tasks to agents to minimize costs or
maximize benefits, ensuring each task is assigned to exactly one agent and vice versa.
To solve the assignment problem using an exhaustive search, we follow these steps:
1. Generate all permutations: Generate all possible permutations of the integers 1 to n,
where n is the number of people (or jobs).
2. Compute total cost: For each permutation, calculate the total cost by summing up the
corresponding elements from the cost matrix.
3. Select the minimum cost: Choose the permutation that yields the smallest total cost.
This permutation represents the assignment that minimizes the total cost.

Let's illustrate this approach with a small example: Consider the following cost matrix
representing the assignment costs:

Job 1 Job 2 Job 3 Job 4

Person 1 9 2 7 8

Person 2 6 4 3 7

Person 3 5 8 1 8

Person 4 7 6 9 4
Permutation: <1, 2, 3, 4>

• This permutation suggests that Person 1 is assigned to Job 1, Person 2 to Job 2, Person
3 to Job 3, and Person 4 to Job 4.
• Total cost: 9 (from Person 1 assigned to Job 1) + 4 (from Person 2 assigned to Job 2) +
1 (from Person 3 assigned to Job 3) + 4 (from Person 4 assigned to Job 4) = 18
Permutation: <1, 2, 4, 3>

• This permutation suggests that Person 1 is assigned to Job 1, Person 2 to Job 2, Person
3 to Job 4, and Person 4 to Job 3.

• Total cost: 9 (from Person 1 assigned to Job 1) + 4 (from Person 2 assigned to Job 2) +
8 (from Person 3 assigned to Job 4) + 9 (from Person 4 assigned to Job 3) = 30
Permutation: <1, 3, 2, 4>

• This permutation suggests that Person 1 is assigned to Job 1, Person 2 to Job 3,

Person 3 to Job 2, and Person 4 to Job 4.

• Total cost: 9 (from Person 1 assigned to Job 1) + 3 (from Person 2 assigned to

Job 3) + 8 (from Person 3 assigned to Job 2) + 4 (from Person 4 assigned to Job
4) = 24
Permutation: <1, 3, 4, 2>

• This permutation suggests that Person 1 is assigned to Job 1, Person 2 to Job 3,

Person 3 to Job 4, and Person 4 to Job 2.

• Total cost: 9 (from Person 1 assigned to Job 1) + 3 (from Person 2 assigned to

Job 3) + 8 (from Person 3 assigned to Job 4) + 6 (from Person 4 assigned to Job
2) = 26
Permutation: <1, 4, 2, 3>

• This permutation suggests that Person 1 is assigned to Job 1, Person 2 to Job 4,

Person 3 to Job 2, and Person 4 to Job 3.

• Total cost: 9 (from Person 1 assigned to Job 1) + 7 (from Person 2 assigned to

Job 4) + 8 (from Person 3 assigned to Job 2) + 9 (from Person 4 assigned to Job
3) = 33
Permutation: <1, 4, 3, 2>

• This permutation suggests that Person 1 is assigned to Job 1, Person 2 to Job 4,

Person 3 to Job 3, and Person 4 to Job 2.

• Total cost: 9 (from Person 1 assigned to Job 1) + 7 (from Person 2 assigned to

Job 4) + 1 (from Person 3 assigned to Job 3) + 6 (from Person 4 assigned to Job
2) = 23
 • We continue this process for all 24 permutations and select the one with the minimum
total cost as the solution to the assignment problem.

• The time complexity of the exhaustive search approach for the assignment problem is
O(n!), where n is the number of tasks or agents. This is because we need to generate all
possible permutations of assignments.
• In the given example of the assignment problem, there are n=4 people and n=4 jobs.
This means n! = 4! = 24 permutations.

• The space complexity is O(n) to store the cost matrix and O(n) to store each
permutation temporarily during computation. Therefore, the total space complexity is
O(n).

Decrease and Conquer

● Decrease-and-conquer techniques offer a structured way to solve problems by

progressively reducing their size until reaching a base case that can be directly solved.

● Each variation provides a different strategy for reducing the problem size, leading to
different time complexities and efficiency levels.

● There are two main approaches: top-down (recursive) and bottom-up (iterative).

Variations of Decrease-and-Conquer:

• Decrease-by-a-Constant: In this variation, the size of the problem instance is

reduced by the same constant on each iteration of the algorithm. Typically, this
constant is equal to one.

• Decrease-by-a-Constant-Factor: Here, the problem instance is reduced by the same

constant factor on each iteration. Typically, this factor is equal to two.

• Variable-Size Decrease: In this variety, the size-reduction pattern varies from one
iteration of the algorithm to another.
Decrease by one technique
Insertion Sort
Insertion Sort is a simple sorting algorithm that builds the final sorted array one element at a
time.
Here's a step-by-step explanation of how it works:
1. Initialization: Assume we have an array A of size n, containing unsorted elements.
We start with the second element of the array (index 1) since a single element is
already considered sorted.
 2. Insertion:

● For each element starting from the second one, we consider it as the key and compare
it with the elements to its left in the sorted subarray.
● We move elements greater than the key to the right to make space for the key element
in its correct position.
3. Sorting: We repeat this process for each element in the array until all elements are
in their correct sorted positions.
Algorithm InsertionSort(A[0..n-1]):
Input: An array A[0..n-1] of n orderable elements
Output: Array A[0..n-1] sorted in nondecreasing order
for i ← 1 to n - 1 do:
// Select the element to be inserted
key ← A[i]

// Find the correct position for the key within the sorted subarray A[0..i-1]
j←i-1
while j ≥ 0 and A[j] > key do:
// Shift elements of A[0..i-1] that are greater than key to the right
A[j + 1] ← A[j]
j←j-1

// Insert key into its correct position

A[j + 1] ← key
• The worst case, the time complexity is Θ(n2), indicating that the growth rate is exactly
n2.
• In the best-case scenario for insertion sort, where the array is already sorted, the inner
loop only runs once for each element in the array. The time complexity is Θ(n).
• In the case of insertion sort, we observe that the total number of comparisons and shifts,
which contribute to the time complexity, grows quadratically with the input size.
• Therefore, we use the Θ notation to express the precise growth rate of the algorithm's
time complexity as n2.
 Algorithms for Generating Combinatorial Objects
Generating Permutations
Minimal Change algorithm

● The minimal-change requirement in generating permutations ensures that each

permutation in the sequence can be obtained from its immediate predecessor by
exchanging just two adjacent elements.

● This property is advantageous because it simplifies the process of generating

permutations and ensures that we cover all possible permutations without redundancy.
Start: We start with the initial permutation, which is usually the sequence of integers from 1
to n.
Insert 2 into 1 right to left:

• To generate the next permutation, we insert the next element (2) into each possible
position in the current permutation.

• We start from the right and move towards the left.

• This ensures that we maintain the minimal-change requirement.
Insert 3 into 12 right to left: Similarly, we insert the next element (3) into each position in the
permutation [1, 2]. Again, we move from right to left.
Insert 3 into 21 left to right:
• Now, we insert the next element (3) into each position in the permutation [2, 1].

• However, this time we start from the left and move towards the right.
• This alternation in direction ensures that we cover all possible permutations while
satisfying the minimal-change requirement.
Repeat steps 2-4 until we generate all permutations.

Insertion Permutation Action

Insert 1 1

Insert 2 12 21 Insert 2 from Right

to Left

Insert 3 12 123 132 312 Insert 3 from Right

to Left

21 321 231 213 Insert 3 from Left

to Right
Johnson-Trotter algorithm
1. Initialization:
1. Start with the initial permutation, which is the sequence of integers from 1 to n,
arranged in ascending order.
2. Finding Mobile Elements:
1. Identify the mobile elements in the current permutation.
2. An element is mobile if it is greater than the adjacent element to which its arrow
points.
3. Finding the Largest Mobile Element:
1. Among the mobile elements, find the largest one, denoted by k.
4. Swapping Elements:
1. Swap the element k with the adjacent element to which its arrow points.
5. Reversing Direction:
1. Reverse the direction of the arrows for all elements greater than k.
6. Generating Permutations:
1. Add the new permutation to the list of permutations generated.
7. Repeat:
1. Repeat steps 2 to 6 until there are no more mobile elements in the permutation.

Let's illustrate these steps with an example for n = 3:

1. Initial permutation: [1, 2, 3]
 Generating Subsets
Bottom-Up Approach

● Start with an empty set, which represents the subset with no elements.
● Add each element of the original set to the existing subsets one by one.

● For each new element, create new subsets by adding that element to each existing
subset.

● Combine the new subsets with the existing ones to form a larger set of subsets.
● This process is repeated until all subsets are generated.
For example, consider the set {a1, a2, a3}. Here's how the subsets are generated:

• For 0 elements (0 subsets): ∅ (empty set)

• For 1 element (1 subset): ∅, {a1}

• For 2 elements (3 subsets): ∅, {a1}, {a2}, {a1, a2}

• For 3 elements (7 subsets): ∅, {a1}, {a2}, {a1, a2}, {a3}, {a1, a3}, {a2, a3}, {a1, a2,
a3}

● This approach ensures that we cover all possible subsets of the original set. Each subset
can be seen as a combination of elements chosen from the original set.

Bit Manipulation Approach

• For a set with n elements, we aim to generate all 2n subsets.

• Each subset can be represented using a binary string of length n, where each bit
indicates whether the corresponding element is included (1) or excluded (0) from the
subset.
Steps:
1. Initialize: Start with an empty subset (all bits are 0).
2. Iteration: For each bit string representing a subset:
1. Incremental Addition: To generate the next subset, find the rightmost 0 bit and
change it to 1.
2. Repeat this process until all bits are 1 (indicating the last subset).
Example:
• For a set {a, b, c}, the process would generate subsets:
• ∅ (000)
 • {a} (100)
• {b} (010)

• {a, b} (110)
• {c} (001)
• {a, c} (101)

• {b, c} (011)
• {a, b, c} (111)

Binary Reflected Gray Code Approach

Initialization:

• Similar to the bit manipulation approach, start with an empty subset.

Steps:
1. Generate Gray Code: Use a recursive algorithm to generate the binary reflected Gray
code of order n.
1. At each step, construct the next bit string by modifying the previous bit string
in such a way that only one bit changes.
Example:

• For a set {a, b, c}, the Gray code approach would generate subsets:

• ∅ (000)
• {a} (001)

• {a, b} (011)
• {b} (010)
• {b, c} (110)

• {a, b, c} (111)
• {a, c} (101)
• {c} (100)
Key Differences:
1. Transition Between Values:
1. In regular binary strings, adjacent values can differ in multiple bits.
2. In Gray code, adjacent values differ by only one bit.
 2. Transition Complexity:
1. Transitioning between consecutive values in Gray code is smoother and
involves fewer bit changes compared to regular binary strings.

Decrease by Constant Factor (Decrease-by-Half)

Binary Search
1. Initialize pointers: Start with two pointers, left and right, which represent the indices
of the array's boundaries. Initially, the left is set to the first index (0) and the right is set
to the last index (n - 1), where n is the size of the array.
2. Find the midpoint: Calculate the midpoint index mid as (left + right) / 2.
3. Check the midpoint value:
1. If the value at index mid matches the target value, the search is successful, and
the index mid is returned.
2. If the value at index mid is greater than the target value, update right to mid-1,
effectively discarding the right half of the array.
3. If the value at index mid is less than the target value, update left to mid + 1,
effectively discarding the left half of the array.
4. Repeat the process: Continue steps 2 and 3 until either the target value is found or left
becomes greater than right, indicating that the target value is not present in the array.

Now, let's illustrate binary search with an example:

● Suppose we have a sorted array: [1, 3, 5, 7, 9, 11, 13, 15, 17, 19], and we want to find
the index of the value 11.
1. Initialize pointers: Set left = 0 and right = 9.
2. Find the midpoint: Calculate mid = (0 + 9) / 2 = 4. The value at index mid is 9.
3. Check the midpoint value:
1. Since 9 is less than 11, update left to mid + 1, making left = 5.
4. Find the new midpoint: Calculate mid = (5 + 9) / 2 = 7. The value at index mid is 15.
5. Check the midpoint value:
1. Since 15 is greater than 11, update right to mid - 1, making right = 6.
6. Find the new midpoint: Calculate mid = (5 + 6) / 2 = 5. The value at index mid is 11.
 7. Check the midpoint value:
1. Since 11 matches the target value, the search is successful, and we return mid =
5.

● The target value 11 is found at index 5.

For example, let's say we have an array of size n. In the worst-case scenario, binary search will
take log₂(n) steps to find the target element. This is because log₂(n) represents the number of
times we can halve n before reaching 1.
Time Complexity:

• In each step of binary search, the size of the search space is halved.
• Therefore, the time complexity of binary search is O(log n), where n is the number of
elements in the array.

• This is because the algorithm divides the array in half at each step, resulting in a
logarithmic time complexity.

• This makes binary search extremely efficient, especially for large arrays.

Fake Coin Problem

● The fake coin problem involves identifying a single fake coin among a collection of
identical-looking coins, with the assumption that the fake coin is lighter than the
genuine ones.

● The goal is to design an efficient algorithm to detect fake coins using a balance scale
that compares sets of coins.
Initial Approach (Dividing into Two Piles):

• We start by dividing the coins into two piles of approximately equal size: one pile
containing 6 coins and the other containing 6 coins.

• We place each pile on one side of the balance scale.

Scenario 1: Balanced Scale:
• If the scale balances, it means the fake coin is not in either of the piles. In this
case, we take the remaining 6 coins (the ones we didn't weigh) and repeat the
process.

• We divide the remaining 6 coins into two piles of 3 coins each and weigh them
on the balance scale.
Scenario 2: Unbalanced Scale (Left Side Lighter):

• If the scale tips to one side, say the left side, it indicates that the fake coin is
among the 6 coins on the left side of the scale.
• We then take these 6 coins and repeat the process recursively, dividing them into
two piles and weighing them until we find the fake coin.
Solution with Binary Search:

• Using the recurrence relation W(n)=log2n, we can determine the maximum number of
weighing needed in the worst case.

• For 12 coins, log212≈3.58, meaning that at most 3 weighing will be required to find the
fake coin.
To solve the recurrence relation W(n)=W(n/2)+1 for n>1 with the initial condition W(1)=0 for
n=12, we can follow these steps:
1. Substitute n=12 into the recurrence relation.
2. Continue substituting n/2 until n becomes 1, and count the number of times we perform
this operation.
3. Add 1 for each substitution to account for the +1 term in the recurrence relation.
4. Sum up all the added 1s to get the total number of weighings.
Let's do it step by step:
1. W (12) =W (12/2) +1=W (6) +1
2. W (6) =W (6/2) +1=W (3) +1
3. W (3) =W (3/2) +1=W (1) +1
4. Since W (1) = 0, we have W (3) = 0+1=1
5. Substituting back, W (6) = 1+1=2
6. Substituting back, W (12) = 2+1=3
● So, for n=12 coins, the number of weighings needed to identify the fake coin is 3.
Improvement with Three Piles:

• Instead of dividing the coins into two equal piles, we can divide them into three piles
of 4 coins each.

• Weigh two of the piles against each other. If one side is lighter, we know the fake coin
is in that pile, reducing the search space by a larger factor than in the binary approach
W(n)=log3n.
Russian Peasant Application

● The Russian peasant multiplication method is a technique for multiplying two positive
integers using only halving, doubling, and addition operations.
● The algorithm relies on the observation that multiplying by 2 and dividing by 2 are
equivalent operations when dealing with integers.
Here's how the algorithm works step by step:
1. Initialize: Start with the two integers you want to multiply, n and m, and set the product
initially to 0.
2. Repeat Until n=1:
1. If n is even, double m and halve n.
2. If n is odd, add m to the product, double m, and then halve n.
3. Terminate: When n=1, stop. The product is the final value of m.
Let's illustrate this with an example of computing 50×65:
1. Start with n=50 and m=65, and set the product to 0.
2. Since 50 is even, double 65 to get 130, and halve 50 to get 25.
3. 25 is odd, so add 130 to the product (which is currently 0), double 130 to get 260, and
then halve 25 to get 12. product = 0+130= 130
Repeat the process:
1. 12 is even, so double 260 to get 520, and halve 12 to get 6.
2. 6 is even, so double 520 to get 1040, and halve 6 to get 3.
3. 3 is odd, so add 1040 to the product (which is currently 130), double 1040 to
get 2080, and halve 3 to get 1. product = 130+1040 = 1170
1 is the termination condition, so we add 2080 to our running total and stop. The final product
is 3250.

Josephus Problem
Here's a formal statement of the problem:

● Suppose there are n people numbered from 1 to n standing in a circle.

● Starting with person 1, every k-th person is eliminated until only one person remains.
● The problem is to determine the position of the survivor.
 K=2

2
6

3
5

1 K=2
1

K=2
3
5 5

For example, if n=6 and k=2, the elimination process goes as follows:
1. Person 1 remains.
2. Person 2 is eliminated.
3. Person 3 remains.
4. Person 4 is eliminated.
5. Person 5 remains.
6. Person 6 is eliminated.
● Amongst 1 (skipped), 3 (eliminated), 5 (skipped).
 ● 1 (eliminated), 5 (skipped).
● So, the survivor is person 5.
Even n Case (n=2k):
1. In this case, after the first pass, every second person is eliminated, leaving
behind k survivors.
2. The survivor's position among the remaining k people will be denoted by J(k).
3. To find the survivor's position among the original 2k people, we double J(k) and
subtract 1, since the position numbering changes after the first pass.
4. So, for even n, we have the recurrence relation: J(2k) = 2J(k)−1
Odd n Case (n=2k+1):
1. After the first pass, every second person is eliminated, leaving k survivors.
Additionally, the person in the first position is eliminated.
2. The survivor's position among the remaining k people will be denoted by J(k).
3. To find the survivor's position among the original 2k+1 people, we double J(k)
and add 1, since the position numbering changes after the first pass and we
eliminate the person in the first position.
4. So, for odd n, we have the recurrence relation: J(2k+1) = 2J(k)+1
Closed-Form Solution:
• Interestingly, we can represent n in binary form.
• The survivor's position J(n) is obtained by cyclically shifting the binary representation
of n one bit to the left.
• For example, if n=6, represented in binary as 110, the survivor's position is 101, which
is 5.
• Similarly, if n=7, represented in binary as 111, the survivor's position is 111, which is
7.
 Variable-Size-Decrease Algorithms
In the third principal variety of decrease-and-conquer, the size reduction pattern varies from
one iteration of the algorithm to another.

Computing a Median and the Selection Problem

● The selection problem involves finding the kth smallest element in a list of n numbers.
This problem is often referred to as finding the kth order statistic. If k=1 or k=n, it's
trivial to find the smallest or largest element respectively by simply scanning the list.
● For a more interesting case, let's consider k=n/2. This asks us to find an element that is
not larger than half of the list's elements and not smaller than the other half. This middle
value is called the median and is a crucial concept in statistics.
● One naive approach to find the kth smallest element is to sort the entire list and then
select the kth element. However, this is inefficient because we only need to find the kth
smallest element, not to order the entire list.
● Instead, we can use the idea of partitioning the list around a chosen pivot value p.
Lomuto's partitioning algorithm is one way to do this.
● We choose p (usually the first element) and rearrange the list so that all elements smaller
than or equal to p are on the left, p itself is in the middle, and all elements larger than
or equal to p are on the right.
● The variable-size-decrease aspect of Lomuto partitioning comes from the fact that the
size of the array segments being partitioned decreases with each recursive call until they
reach a base case, at which point the algorithm terminates.

Lomuto Partition (A[L…..R])

Pivot = A[L]
current = L
for i = L+1…R
{
If (A[i] < pivot)
{
current ++
Swap (A[i], A[current])
}
}
Swap (A[L], A[current])

return current
 Determining that we need to find any k-th smallest element typically depends on the
problem statement or requirements. In many cases, you may need to find:
• The smallest element: This is when k=1.
• The largest element: This is when k=n (where n is the size of the array).
• A median element: This is when k is approximately n/2.
• Any arbitrary k-th smallest element: For example, finding the 3rd smallest element
in a list.

Let's apply Lomuto's algorithm to find the k-th smallest element for the given array [5,
7, 2, 6, 4, 9]:
1. Select Pivot: For simplicity, let's choose the first element 5 as the pivot.
2. Partitioning: Rearrange the array such that elements less than 5 are on its left, and
elements greater than 5 are on its right.
3. After partitioning, the array becomes [2, 4, 5, 6, 7, 9]. The pivot 5 is now in its correct
position.
4. Recursion: Since the pivot's index is 2 (0-based index) and we're looking for the 3rd
smallest element (k=3), we know that the 3rd smallest element is at index 2.
5. So, the 3rd smallest element is 5.
● Therefore, using Lomuto's algorithm, the 3rd smallest element in the array [5, 7, 2, 6,
4, 9] is 5.

● Lomuto's partitioning algorithm is a specific method for partitioning an array around a

pivot element, which is a crucial step in various algorithms, including Quickselect.
● Quickselect is an algorithm for finding the k-th smallest element in an unsorted list. It
utilizes partitioning (often Lomuto's partitioning) to recursively narrow down the search
space until it finds the desired element.
● In essence, Lomuto's partitioning is a subroutine used within Quickselect to efficiently
partition the array, but Quickselect encompasses the entire process of finding the k-th
smallest element using this partitioning technique.

Check Pivot Index: The pivot's index is 2 (0-based index).

• If k - 1 (for 0-based indexing) is equal to the pivot index, then the pivot element is the
k-th smallest element. In this case, k - 1 = 3 - 1 = 2, which matches the pivot index. So,
the 3rd smallest element is 5.
• If k - 1 is less than the pivot index, then we apply the algorithm to the left subarray
(elements before the pivot).
 • If k - 1 is greater than the pivot index, then we apply the algorithm to the right subarray
(elements after the pivot).

Interpolation Search
● Interpolation search is a searching algorithm used to find a specific value within a sorted
array.
● It differs from binary search by taking into account the value of the search key to
estimate the position of the target element.
● Let's break down the steps of the interpolation search algorithm with an example:
Suppose we have a sorted array ‘arr’ and we want to find the index of a specific value target
within this array.
1. Initialize Variables:
1. Set left = 0 and right = arr.length - 1.
2. Define the target value target.
2. Interpolation Formula:
1. Calculate the index using the interpolation formula:
index = left + ((target - arr[left]) * (right - left)) / (arr[right] - arr[left])
This formula estimates the index where the target might be located based on the assumption of
linearly increasing array values.
3. Search Iteration:
Compare the target value with the value at the calculated index:
1. If arr[index] == target, return index (target found).
2. If arr[index] < target, update left = index + 1 (search in the right
half).
3. If arr[index] > target, update right = index - 1 (search in the left
half).
4. Repeat:
1. Repeat steps 2 and 3 until the target value is found or the search space is
exhausted (left > right).
5. Result:
1. If the target value is found, return the index.
2. If the target value is not found, return -1 (indicating the value is not present in
the array).
Let's illustrate this with an example:
● Suppose we have the sorted array arr = [1, 3, 5, 7, 9, 11, 13, 15, 17, 19] and we want
to find the index of the value target = 13.
1. Initialization:
1. left = 0, right = 9.
2. target = 13.
2. Interpolation Formula:
index = left + ((target - arr[left]) * (right - left)) / (arr[right] - arr[left])
index = 0 + ((13 - 1) * (9 - 0)) / (19 - 1)
= 0 + (12 * 9) / 18
=6
3. Search Iteration:
1. Compare arr[6] with target:
1. arr[6] = 13 == target: Return index = 6.
4. Result:
1. The index of the target is 6.
So, the interpolation search algorithm successfully found the index of the value 13 in the array.

Game of NIM
• The Game of Nim or Nim game is a mathematical combinatorial game in which we
have a pile of some objects, say coins in this case, and two players who will take turns
one after the other.
• In each turn, a player must pick at least one object(coin in this case), and may remove
any number of objects(coins) given that they all belong to the same pile.
• The game is won by the one who picks the last object(coin).
Example 1
● Given piles of coins are: Piles[]=2,5,6
● Let us look at the step-by-step process in which the game will proceed if Player A starts
first.
● As shown in the example, Player A takes the last coin and is the winner.
Example 2
● Given piles of coins are: Piles [] = 4,1,5
● Let us look at the step-by-step process in which the game will proceed if Player A starts
first.
● As shown in the example, Player B takes the last coin and is the winner.
 From the example that we saw above, we can say that the winner of the game surely
depends on the configuration of the piles and the player who starts first.
Let us now introduce a new term called Nim Sum which will play an important role in
finding out the final solution and we'll see how.
Nim-Sum
At any point of time during the duration of the game, we define the Nim sum as
the cumulative Exclusive-OR (XOR) value of all the coins in each pile.
Lemma:
● If both the player’s A and B are playing the game optimally, which means that they
don’t make any moves that may hamper their chances of winning.
● In that case, if the Nim sum of the initial configuration is non-zero, the player making
the first move is guaranteed to win the game and if the Nim sum of the initial
configuration is zero the second player will win the game.

Example 1
Initial configuration is 2 5 6
● 2 0 1 0
● 5 1 0 1
● 6 1 1 0
XOR 0 0 1
Since Num sum is Non-zero player who makes first move wins

Example 2
Initial configuration is 4 1 5
• 0 0 1
• 0 0 1
• 1 0 1
XOR 0 0 0
Since the Num sum is zero second player wins
 Divide and Conquer
Multiplication of Large Integers
Let's break down the process of multiplying two large integers using the divide-and-conquer
technique step by step, using a small example.
Suppose we want to multiply two 2-digit integers, say 23 and 14.
1. Represent the integers: Express each integer as a combination of its digits and powers
of 10. For example:
1. 23=2×101+3×100
2. 14=1×101+4×100
2. Apply the multiplication:
23×14= (2×101+3×100) × (1×101+4×100)
Expand the expression:
= (2×101×1×101) + (3×100×1×101) + (2×101×4×100) + (3×100×4×100)
= 2×102+11×101+12×100
3. Combine the terms:
1. 23×14=2×102+11×101+12×100=322
So, the result of multiplying 23 and 14 is 322.

Strassen’s Matrix Multiplication

Strassen's Matrix Multiplication is a method to multiply two matrices more efficiently
compared to the brute-force approach. Here's a step-by-step explanation of how it works with
an example:
Let's say we want to multiply two 2×2 matrices:

● A=(𝑎00
𝑎10
𝑎01
𝑎11
),

● B=(𝑏00
𝑏10
𝑏01
𝑏11
)

Formulas for Strassen's Algorithm:

Using Strassen's formulas, we can compute the elements of the resulting matrix C=A×B
as follows:
● m1=(a00+a11)×(b00+b11)
● m2=(a10+a11)×b00
● m3=a00×(b01−b11)
● m4=a11×(b10−b00)
● m5=(a00+a01) ×b11
● m6=(a10−a00) ×(b00+b01)
● m7=(a01−a11)×(b10+b11)

Then, we can compute the elements of C using these values:

● c00=m1+m4−m5+m7
● c01=m3+m5
● c10=m2+m4
● c11=m1+m3−m2+m6

Example:Let's say we have two matrices:

2 3
A= ( )
1 4
1 2
B=( )
3 4
Step 1: Compute m1 to m7:
• m1= (2+4) × (1+4) =6×5=30
• m2= (1+4) ×1=5×1=5
• m3=2× (2−4) =2× (−2) =−4
• m4=4× (3−1) =4×2=8
• m5= (2+3) ×4=5×4=20
• m6= (1−2) × (1+2) = (−1) ×3=−3
• m7= (3−4) × (3+4) = (−1) ×7=−7

Step 2: Compute C:
● c00=m1+m4−m5+m7=30+8−20−7=11
● c01=m3+m5=−4+20=16
● c10=m2+m4=5+8=13
● c11=m1+m3−m2+m6=30−4−5−3=18
11 16
Result: C=( )
13 18
Analysis:
• Strassen's algorithm performs seven multiplications (m1 to m7) and 18
additions/subtractions to compute the elements of the resulting matrix.
• This is in contrast to the brute-force approach, which requires eight multiplications and
four additions.
• While Strassen's algorithm appears to have more operations, it offers better efficiency
for large matrices due to its asymptotic superiority.
Asymptotic Efficiency:
• Let M(n) be the number of multiplications made by Strassen's algorithm for multiplying
two n×n matrices.
• The recurrence relation for M(n) is M(n)=7M(n/2) + an2 for n>1, with M (1) =1.
• Solving this recurrence yields M(n)= 𝑛log2 7 , which is approximately n2.807.
Conclusion:
• Strassen's algorithm reduces the number of multiplications required for matrix
multiplication, leading to better efficiency for large matrices.
• While other algorithms with even lower exponents exist, they come with increased
complexity and are not practically useful due to large multiplicative constants.

● Strassen's algorithm can be applied to n x n matrix multiplications where n is not an

exact power of 2 by padding the operands with 0’s.

Note: Let's say we want to multiply a 3x3 matrix A with a 3x3 matrix B:
A = | a11 a12 a13 |
| a21 a22 a23 |
| a31 a32 a33 |
B = | b11 b12 b13 |
| b21 b22 b23 |
| b31 b32 b33 |
Since 3 is not a power of 2, we'll pad both matrices to the next power of 2, which is 4:
A' = | a11 a12 a13 0 |
| a21 a22 a23 0 |
| a31 a32 a33 0 |
| 0 0 0 0
B' = | b11 b12 b13 0 |
| b21 b22 b23 0 |
| b31 b32 b33 0 |
| 0 0 0 0|
● Now, we have two 4x4 matrices A' and B'. We can apply Strassen's algorithm as usual
to multiply these matrices. After obtaining the result, we discard the extra rows and
columns added during padding to get the actual result of the multiplication.
● This approach ensures that we can still use Strassen's algorithm for matrix
multiplication even when the matrix dimensions are not powers of 2. However, padding
with zeros increases the memory usage and may affect the overall efficiency, especially
for large matrices.

The Closest-Pair problem

Input: An array of n points P[]

Output: The smallest distance between two points in the given array.
• Find the closest pair in the LEFT (dl)
• Find the closest pair in the RIGHT (dr)
• Check if there is a pair such that one point is on the LEFT region, other point is on the
RIGHT region, and the distance between them is < d = min(dl,dr)

Approach

1) Find the middle point in the sorted array, we can take P[n/2] as the middle point.

2) Divide the given array into two halves. The first subarray contains points from P[0] to P[n/2].
The second subarray contains points from P[n/2+1] to P[n-1].

3) Recursively find the smallest distances in both subarrays. Let the distances be dl and dr. Find
the minimum of dl and dr. Let the minimum be d.
4) From the above 3 steps, we have an upper bound d of minimum distance.
Now we need to consider the pairs such that one point in pair is from the left half and the other
is from the right half.
Consider the vertical line passing through P[n/2] and find all points whose x coordinate is closer
than d to the middle vertical line. Build an array strip [] of all such points.
5) Sort the array strip [] according to y coordinates. This step is O(nLogn). It can be optimized
to O(n) by recursively sorting and merging.

6) Find the smallest distance in strip []. This is tricky. From the first look, it seems to be a
O(n^2) step, but it is actually O(n).
It can be proved geometrically that for every point in the strip, we only need to check at most
7 points after it (note that strip is sorted according to Y coordinate).
7) Finally return the minimum of d and distance calculated in the above step (step 6)

Time Complexity Let the Time complexity of the above algorithm be T(n). Let us assume that
we use an O(nLogn) sorting algorithm.
• The above algorithm divides all points in two sets and recursively calls for two sets.
• After dividing, it finds the strip in O(n) time, sorts the strip in O(nLogn) time and finally
finds the closest points in strip in O(n) time.
So, T(n) can be expressed as follows

T(n) = 2T(n/2) + O(n) + O(nLogn) + O(n)

T(n) = 2T(n/2) + O(nLogn)
The overall time complexity is O(n log(n)).

Convex-Hull Problem using merge sort

The steps that we’ll follow to solve the problem are:
1. First, we’ll sort the vector containing points in ascending order (according to their x-
coordinates).
2. Next, we’ll divide the points into two halves S1 and S2. The set of points S1 contains
the points to the left of the median, whereas the set S2 contains all the points that are
right to the median.
3. We’ll find the convex hulls for the set S1 and S2 individually. Assuming the convex
hull for S1 is C1, and for S2, it is C2.
4. Now, we’ll merge C1 and C2 such that we get the overall convex hull C.
Here the only tricky part is merging the two convex hulls C1 and C2. The idea here is that
the two tangents of the left and the right convex hulls (C1 and C2) will make the final
convex hull C.
Finding the upper tangent and lower tangent.
● Let L1 be the line that joins the rightmost point of the left convex hull C1 and the
leftmost point of the right convex hull C2.
● As L1 passes through the polygon C2, we take the anti-clockwise next point on C2 and
label the line L2. The line is above the polygon C2, but it crosses the polygon C1, so
we move to the clockwise next point, making the line L3. This again crosses the left
polygon, so we move to line L4. This line is crossing the right polygon, so we move to
line L5. Now, this final line is crossing neither of the points.
● Hence, we get the upper tangent for the given two convex hulls.
● For finding the lower tangent, we need to move inversely through the hulls, i.e., if the
line is crossing the convex hull C2, we move to clockwise next and to anti-clockwise
next if the line is crossing the convex hull C1

● Here, PQRSUVW is the convex hull of the set of given points.

Convex Hull Problem using Quick Hull
One method for finding the convex hull of a point set is the Quickhull algorithm. It makes
use of the divide and conquer paradigm, and builds the convex hull in a recursive manner.

The initialization steps in the first function:

1. Given a set of points S, we'll find the two points A and B, that have a minimum and
maximum x coordinate among all points in the set. These two points are with certainty
part of the convex hull, because they make up the horizontal extremities of the set.
2. The line that connects these two extremities, splits the entire set of points into two
halves. Here it's important to point out (pun not intended) that this line needs to be
treated as a directed line, such that the two halves can be specified as the point sets that
lie to the right of the directed line AB and the point set that lies to the right of the
directed line BA.
3. Computing the point sets in that manner will make things easier in the subsequent steps
of the procedure. These two halves will serve as starting point to get the recursive
procedure going, where they will be split into smaller and smaller point sets.

Starting out with the initial line and these two smaller point sets, the second function beings
the recursive procedure:

1. Given a line (formed by two points that we'll call Q and P) as well as a set of points,
where all the points in the set are positioned on one side of the line, we find the point
that is furthest from this line. We call this point C.
2. This point lies on the extremity of the original set and is also essentially a vertex of the
convex hull polygon.
3. With this new point we form a triangle with the two end points of the line. This triangle
splits the given set of points into three subsets.
4. The set of points that falls inside of the triangle can be disregarded as none of these
points could possibly be part of the convex hull.
5. The other two sets of points need to be processed recursively as they still contain points
that belong to the convex hull. The first set being those points that fall on the right side
of the directed line from point P to C and the other on the right side of the line going
from C to Q.

Here's a visualization of this:

Essentially, starting a line, we find the furthest point towards that line, construct two new lines
that connect that point with the endpoints of the previous line (forming a triangle). Then we
repeat this procedure for the two new lines and the remaining points that fall outside of the
triangle, until no points are left.