Assignment 2:

1. Exercise 15.1-1 from the textbook (5 marks)

 2. Exercise 15.1-5 from the textbook (15 marks)

To compute the nth Fibonacci number using dynamic programming, we can use a bottom-up
approach that builds the solution iteratively by solving smaller subproblems first. The algorithm
will maintain an array to store the Fibonacci numbers from 0 to n. Here's how the algorithm
works:
1. Create an array fib of size n+1 to store Fibonacci numbers.
2. Initialize fib[0] to 0 and fib[1] to 1.
3. For i from 2 to n: a. Calculate fib[i] as fib[i-1] + fib[i-2].
The subproblem graph for this dynamic programming approach will have n+1 vertices
(representing the Fibonacci numbers from 0 to n) and n edges (each edge representing a
subproblem dependency). The graph will be a linear chain.
Here's an illustration of the subproblem graph for n = 5:

1. F(0) -> F(1) -> F(2) -> F(3) -> F(4) -> F(5)
2.

In this graph, each vertex represents a Fibonacci number, and an edge connects a vertex to the
two vertices that are its subproblem dependencies. This chain of dependencies captures the
computation process for the Fibonacci numbers.
Now, let's analyze the number of vertices and edges in the graph:
• Number of vertices = n+1
• Number of edges = n
For example, when n = 5:
• Number of vertices = 6
• Number of edges = 5
Overall, the dynamic programming algorithm has a time complexity of O(n), since each
Fibonacci number is computed once, and each computation takes constant time. The space
complexity is also O(n) due to the array used to store the Fibonacci numbers.
 3. Exercise 15.2-1 from the textbook (5 marks)

Given the sequence of dimensions h5; 10; 3; 12; 5; 50; 6i, we can represent the dimensions as
follows:
• Matrix A1: 5x10
• Matrix A2: 10x3
• Matrix A3: 3x12
• Matrix A4: 12x5
• Matrix A5: 5x50
• Matrix A6: 50x6
Let's denote the number of matrices as n = 6 (since there are 6 matrices).
The matrix-chain multiplication algorithm involves creating a table to store the optimal costs of
multiplying subchains of matrices. The entry m[i][j] in the table represents the optimal cost of
multiplying matrices A_i * A_{i+1} * ... * A_j.
Here's the algorithm to find the optimal parenthesization:
1. Initialize m as an n x n matrix and s as an (n-1) x n matrix.
2. For each chain length l from 2 to n: a. For each i from 1 to n-l+1:
• Set j = i + l - 1
• Set m[i][j] to a large value (e.g., infinity)
• For k from i to j-1:
• Calculate q = m[i][k] + m[k+1][j] + p_{i-1} * p_k * p_j
• If q is smaller than m[i][j], update m[i][j] = q and s[i][j] = k
3. The optimal cost of multiplying all matrices is stored in m[1][n].
4. Use the s table to reconstruct the optimal parenthesization.
Here, p_i represents the number of rows of matrix A_i and p_{i-1} represents the number of
columns of matrix A_{i-1}.
By applying this algorithm to the given matrix dimensions, you can find the optimal
parenthesization of the matrix-chain product. This will tell you how to group the matrices
together to minimize the number of scalar multiplications needed to compute the product.
 4. Exercise 15.2-2 from the textbook (15 marks)

1. def MATRIX_CHAIN_MULTIPLY(A, s, i, j):

2. if i == j:
3. return A[i]
4.
5. parenthesized_A = MATRIX_CHAIN_MULTIPLY(A, s, i, s[i][j]) @ MATRIX_CHAIN_MULTIPLY(A, s,
s[i][j] + 1, j)
6. return parenthesized_A
7.
8. # Example usage
9. n = 6 # Number of matrices
10. A = [None, A1, A2, A3, A4, A5, A6] # Replace A1, A2, ..., A6 with actual matrices
11. s = ... # Replace with the s table computed by MATRIX-CHAIN-ORDER
12. result = MATRIX_CHAIN_MULTIPLY(A, s, 1, n)
13. print(result)
14.

In this algorithm:
• A is an array containing the matrices A1, A2, ..., An (where A[1] corresponds to A1, A[2]
corresponds to A2, and so on).
• s is the table computed by the MATRIX-CHAIN-ORDER algorithm, which specifies the
optimal split points.
• i and j are the indices specifying the subchain of matrices from A[i] to A[j] that need to
be multiplied.
The function first checks if i is equal to j, indicating that there's only one matrix in the subchain.
If this is the case, it simply returns that matrix.
Otherwise, it recursively performs matrix multiplication on the subchains determined by the s
table. It splits the subchain into two parts at index s[i][j] and performs matrix multiplication on
those subchains using the @ operator (assuming Python 3.5+ for matrix multiplication).
The final result is the fully parenthesized matrix-chain product for the given indices i and j.
 5. Exercise 15.3-1 from the textbook (10 marks)

Running the RECURSIVE-MATRIX-CHAIN algorithm is more efficient than enumerating all the
ways of parenthesizing the product and computing the number of multiplications for each.
Here's why:
1. Exponential Complexity vs. Polynomial Complexity:
• Enumerating all possible parenthesizations involves generating and evaluating an
exponential number of possibilities. For n matrices, there are about 2^(n-1)
possible parenthesizations. This leads to an exponential time complexity.
• RECURSIVE-MATRIX-CHAIN, although recursive, avoids the repeated
computation of subproblems by using dynamic programming. It has a time
complexity of O(n^3), which is polynomial in the number of matrices n.
2. Overlapping Subproblems:
• The dynamic programming approach used in RECURSIVE-MATRIX-CHAIN
eliminates redundant computations by solving subproblems only once and
storing their solutions in a table. This greatly reduces the number of
multiplications that need to be performed.
• Enumerating all possible parenthesizations doesn't exploit overlapping
subproblems and often ends up recalculating the same subproblems multiple
times.
3. Efficient Utilization of Information:
• The dynamic programming approach (like RECURSIVE-MATRIX-CHAIN) utilizes
information from previously computed subproblems to efficiently compute the
optimal solution for larger subproblems.
• Enumerating all possible parenthesizations doesn't take advantage of any
previously computed information, leading to inefficient computation.
4. Principle of Optimality:
• Dynamic programming algorithms, like RECURSIVE-MATRIX-CHAIN, are built on
the principle of optimality, where the optimal solution to a problem can be
constructed from optimal solutions to smaller subproblems.
• Enumerating all possible parenthesizations doesn't inherently follow this
principle, as you would need to evaluate all combinations to ensure you're
finding the optimal one.
 6. Exercise 15.4-1 from the textbook (5 marks)

Python code implementation:

1. def find_lcs(seq_a, seq_b):

2. m = len(seq_a)
3. n = len(seq_b)
4. dp = [[0] * (n+1) for _ in range(m+1)]
5.
6. for i in range(1, m+1):
7. for j in range(1, n+1):
8. if seq_a[i-1] == seq_b[j-1]:
9. dp[i][j] = dp[i-1][j-1] + 1
10. else:
11. dp[i][j] = max(dp[i-1][j], dp[i][j-1])
12.
13. lcs = []
14. i, j = m, n
15. while i > 0 and j > 0:
16. if seq_a[i-1] == seq_b[j-1]:
17. lcs.append(seq_a[i-1])
18. i -= 1
19. j -= 1
20. elif dp[i-1][j] > dp[i][j-1]:
21. i -= 1
22. else:
23. j -= 1
24.
25. lcs.reverse()
26. return lcs
27.
28. sequence_a = [1, 0, 0, 1, 0, 1, 0, 1]
29. sequence_b = [0, 1, 0, 1, 1, 0, 1, 1, 0]
30. lcs = find_lcs(sequence_a, sequence_b)
31. print(lcs)
32.

Output:

1. [1, 0, 1, 0, 1]
2.
 7. Exercise 15.4-2 from the textbook (15 marks)

1. RECONSTRUCT-LCS(c, X, Y, i, j):
2. Let lcs be an empty list
3.
4. while i > 0 and j > 0:
5. if X[i] == Y[j]:
6. Add X[i] to the beginning of lcs
7. Decrement i and j by 1
8. else if c[i-1][j] >= c[i][j-1]:
9. Decrement i by 1
10. else:
11. Decrement j by 1
12.
13. return lcs
14.

In this pseudocode:
• c is the completed dynamic programming table where c[i][j] represents the length of the
LCS for the prefixes X[1:i] and Y[1:j].
• X and Y are the original sequences.
• i and j are the current indices in the c table that we're starting from.
The algorithm works by traversing the c table in a way that mimics backtracking in dynamic
programming. It compares the characters at positions i and j in sequences X and Y. If they
match, it adds the character to the LCS and decrements both i and j. If not, it moves towards the
direction that leads to a higher value in the c table (which is achieved by comparing c[i-1][j] and
c[i][j-1]).
This algorithm takes O(m + n) time, where m is the length of sequence X and n is the length of
sequence Y. This is because each step in the while loop decreases either i or j, ensuring that the
total number of iterations is bounded by m + n.
 8. Exercise 16.1-2 from the textbook (15 marks)

The approach of selecting the last activity to start that is compatible with all previously selected
activities is known as the "Activity Selection Problem" solved using a greedy algorithm. This
approach is indeed a greedy algorithm because at each step, it makes a locally optimal choice
(choosing the last compatible activity) with the hope that this choice will lead to a globally
optimal solution.
Greedy Choice Property: The greedy choice in this case is to select the last compatible activity.
This choice minimizes the potential conflicts with future activities since it leaves more time
available for other activities.
Optimal Substructure: The key to proving the optimality of this greedy approach lies in showing
that the problem exhibits the optimal substructure property. This means that an optimal
solution to the original problem can be constructed from optimal solutions to its subproblems.
Proof of Optimality: Let's consider an arbitrary solution and prove that we can transform it into
the greedy solution without making it worse. Assume that the activities are sorted in increasing
order of their finishing times.
Suppose we have a solution S in which the last activity is not the last one to finish. Let's denote
this last activity as A_last. Now, let's find the activity B with the smallest finishing time after
A_last but still compatible with A_last. If we replace A_last with B in the solution S, we end up
with a solution that still covers the same time interval as S but potentially includes more
activities. Thus, this solution is at least as good as S, if not better.
This process can be repeated until the last activity in the solution S is indeed the last to finish.
Since the greedy choice is to select the last compatible activity at each step, and any deviation
from this choice leads to a solution that is at least as good, if not better, it implies that the
greedy solution is optimal.
 9. Exercise 16.2-2 from the textbook (15 marks)

Python Implementation:

1. def knapsack_01(values, weights, W):

2. n = len(values)
3. dp = [[0] * (W + 1) for _ in range(n + 1)]
4.
5. for i in range(1, n + 1):
6. for w in range(W + 1):
7. if weights[i - 1] <= w:
8. dp[i][w] = max(dp[i - 1][w], dp[i - 1][w - weights[i - 1]] + values[i - 1])
9. else:
10. dp[i][w] = dp[i - 1][w]
11.
12. return dp[n][W]
13.
14. # Example usage
15. values = [60, 100, 120]
16. weights = [10, 20, 30]
17. W = 50
18. max_value = knapsack_01(values, weights, W)
19. print("Maximum value:", max_value)
20.

In this code:
• values is a list containing the values of the items.
• weights is a list containing the weights of the items.
• W is the maximum weight capacity of the knapsack.
The dp table has dimensions (n + 1) x (W + 1) where dp[i][w] represents the maximum value
that can be obtained using the first i items with a weight capacity of w.
The loops iterate through the items and weight capacities, and at each step, the algorithm
considers two options: including the current item or excluding it. If the weight of the current
item is less than or equal to the current capacity, the algorithm chooses the maximum value
between not including the current item (dp[i - 1][w]) and including it (dp[i - 1][w - weights[i -
1]] + values[i - 1]). If the weight of the current item exceeds the capacity, the algorithm simply
carries forward the value from the previous row (dp[i - 1][w]).
Finally, dp[n][W] contains the maximum value that can be obtained using all items within the
given weight capacity W.
The time complexity of this solution is O(nW), where n is the number of items and W is the
maximum weight capacity.