Dynamic Programming

• Matrix-chain multiplication
• 0-1 Knapsack
 Matrix Multiplication

• In particular for 1  i  p and 1  j  r,

C[i, j] = k = 1 to q A[i, k] B[k, j]
• Observe that there are pr total entries in C and
each takes O(q) time to compute, thus the total
time to multiply 2 matrices is pqr.
 Matrix Multiplication

for i=1 to p
for j=1 to r
C[i,j]=0
for i=1 to p
for j=1 to r
for k=1 to q
C[i,j] = C[i,j]+ A[i,k]B[k,j]
 Matrix- Chain Multiplication
• Given a sequence of matrices A1 A2…An , and
dimensions p0 p1…pn where Ai is of
dimension pi-1 x pi , determine multiplication
sequence that minimizes the number of
operations.
• This algorithm does not perform the
multiplication, it just figures out the best
order in which to perform the multiplication.
 Example 1: MCM

• Consider 3 matrices: A1 be 5 x 4, A2 be 4 x 6,
and A3 be 6 x 2.

Mult[((A1 A2)A3)] = (5x4x6) + (5x6x2) = 180

Mult[(A1 (A2A3 ))] = (4x6x2) + (5x4x2) = 88

Even for this small example, considerable savings

can be achieved by reordering the evaluation
sequence.
 Example 2: MCM

• Consider 4 matrices:
A1: 3  5, A2: 5  4, A3: 4  2, A4: 2  5
((A1  A2)  A3)  A4, # of scalar multiplications:
3 * 5 * 4 + 3 * 4 * 2 + 3 * 2 * 5 = 114
(A1  (A2  A3))  A4, # of scalar multiplications:
5 * 4 * 2 + 3 * 5 * 2 + 3 * 2 * 5 = 100
(A1  A2)  (A3  A4), # of scalar multiplications:
3 * 5 * 4 + 4 * 2 * 5 + 3 * 4 * 5 = 160
 Cost of Naive Algorithm
• Is brute-force search efficient for this problem?
• Let P(n) be the number of different ways of
parenthesizing of n matrices. Then

• Solving this, we obtain P(n) = C(n – 1) - Catalan number

• Thus, the brute-force search is too inefficient.

 DP Solution (I)
• Structure of an optimal parenthesization
– Let Ai..j (ij) denote the matrix resulting from AiAi+1…Aj
– Any parenthesization of AiAi+1…Aj must split the product
between Ak and Ak+1 for some k, (ik<j). The cost = # of
computing Ai..k + # of computing Ak+1..j + # Ai..k  Ak+1..j.
– If k is the position for an optimal parenthesization, the
parenthesization of “prefix” subchain AiAi+1…Ak within
this optimal parenthesization of AiAi+1…Aj must be an
optimal parenthesization of AiAi+1…Ak (optimal
substructure)
– AiAi+1…Ak  Ak+1…Aj

8
 DP Solution (II)
• For 1  i  j  n, let m[i, j] denote the minimum number of
multiplications needed to compute Ai…j .
• Example: Minimum number of multiplies for A3…7

A1 A2 A3 A4 A5 A6 A7 A8 A9

m[ 3, 7 ]

 In terms of pi , the product A3…7 has

dimensions p2 x p7.
 DP Solution (III)
• The optimal cost can be described be as follows:
– i = j  the sequence contains only 1 matrix, so m[i, j] = 0.
– i < j  This can be split by considering each k, i  k < j,
as Ai…k (pi-1 x pk ) times Ak+1…j (pk x pj).

• This suggests the following recursive rule for computing

m[i, j]:

m[i, i] = 0
m[i, j] = mini  k < j (m[i, k] + m[k+1, j] + pi-1pkpj ) for i < j
 Computing m[i, j]
• For a specific k,
(Ai …Ak)( Ak+1 … Aj)
=

m[i, j] = mini  k < j (m[i, k] + m[k+1, j] + pi-1pkpj )

 Computing m[i, j]
• For a specific k,
(Ai …Ak)( Ak+1 … Aj)
= Ai…k( Ak+1 … Aj) (m[i, k] mults)

m[i, j] = mini  k < j (m[i, k] + m[k+1, j] + pi-1pkpj )

 Computing m[i, j]
• For a specific k,
(Ai …Ak)( Ak+1 … Aj)
= Ai…k( Ak+1 … Aj) (m[i, k] mults)
= Ai…k Ak+1…j (m[k+1, j] mults)

m[i, j] = mini  k < j (m[i, k] + m[k+1, j] + pi-1pkpj )

 Computing m[i, j]
• For a specific k,
(Ai …Ak)( Ak+1 … Aj)
= Ai…k( Ak+1 … Aj) (m[i, k] mults)
= Ai…k Ak+1…j (m[k+1, j] mults)
= Ai…j (pi-1 pk pj mults)

m[i, j] = mini  k < j (m[i, k] + m[k+1, j] + pi-1pkpj )

 Computing m[i, j]
• For a specific k,
(Ai …Ak)( Ak+1 … Aj)
= Ai…k( Ak+1 … Aj) (m[i, k] mults)
= Ai…k Ak+1…j (m[k+1, j] mults)
= Ai…j (pi-1 pk pj mults)
• For solution, evaluate for all k and take minimum.

m[i, j] = mini  k < j (m[i, k] + m[k+1, j] + pi-1pkpj )

 DP Solution (IV)

• Algorithm,
– array m[1..n,1..n], with m[i,j] records the optimal
cost for AiAi+1…Aj .
– array s[1..n,1..n], s[i,j] records index k which
achieved the optimal cost when computing m[i,j].
– Suppose the input to the algorithm is
p=< p0 , p1 ,…, pn >.
 
0 if i  j
m(i, j)  
 min m(i, k)  m(k  1, j)  p i 1p k p j  if i  j
 i  k  j-1

• Computation sequence :
m(1,4)

m(1,3) m(2,4)

m(1,2) m(2,3) m(3,4)

 Order of matrix computations
– To get a DP solution, order the problems:
m[1,1] m[2,2] ... m[n-1,n-1] m[n,n]
m[1,2] m[2,3] ... m[n-1,n]
•••
m[1,n-1] m[2,n]
m[1,n]
1. for i = 1 to n, set m[i; i] = 0.
2. for ℓ = 2 to n,
compute s- and m-values
for all length ℓ subchains
(Ai,, Aj).
MCM DP Example
DP Solution (IV)
running time is (n3)
 DP Solution (V)

• Constructing a parenthesization order for

the optimal solution.
– Since s[1..n,1..n] is computed, and s[i,j] is the
split position for AiAi+1…Aj , i.e, Ai…As[i,j] and
As[i,j] +1…Aj , thus, the parenthesization order
can be obtained from s[1..n,1..n] recursively,
beginning from s[1,n].
– s[1,n]tells us what multiplication to perform
last.
DP Solution (V)
 Mult (A, i, j)

1. if (j > i)
2. then k = s[i, j]
3. X = Mult(A, i, k) // X = A[i]...A[k]
4. Y = Mult(A, k+1, j) // Y = A[k+1]...A[j]
5. return X*Y // Multiply X*Y
6. else return A[i] // Return ith matrix
 Example: DP
• The initial set of dimensions are <5, 4, 6, 2, 7>: we are
multiplying A1 (5x4) times A2 (4x6) times A3 (6x2) times A4
(2x7). Optimal sequence is (A1 (A2A3 )) A4.
 Characteristics
1. optimal substructure: If s[1, n] = k, then an optimal full
parenthesization contains those of (A1 Ak) and
(Ak+1 An).
2. a small number of subproblems: The number of
subproblems is the number of (i; j) with 1 i  j  n,
which is n(n+1)/2 .
3. overlapping subproblems: m[i, j’] and m[i’, j] are referred
to during the computation of m[i, j], for every
i < i’  j and i  j’ < j.
• Because of the last property, computing an m-entry by
recursive calls takes exponentially many steps.
 The 0/1 Knapsack Problem
• Given: A set S of n items, with each item i
having
– bi - a positive benefit value
– wi - a positive weight
• Goal: Choose items with maximum total benefit
but with weight at most W.

There are two versions of the problem:

1. “0-1 knapsack problem” - Items are
indivisible; you either take an item or not.
2. “Fractional knapsack problem” - Items are
divisible: you can take any fraction of an
item.
 The 0/1 Knapsack Problem

• Let T denote the set of items we take

– Objective: maximize  bi
iT

– Constraint: w i W
iT

• Problem, in other words, is to find

n
max  xi bi subject to  xi wi  W
i 1 i 1

where 𝑥𝑖 𝜖 0,1
 Example
• Given: A set S of n items, with each item i having
– bi - a positive benefit
– wi - a positive weight
• Goal: Choose items with maximum total benefit but with
weight at most W.
“knapsack”

Items: Solution:
• 5 (2 in)
1 2 3 4 5 • 3 (2 in)
• 1 (4 in)
Weight: 4 in 2 in 2 in 6 in 2 in
9 in
Benefit: $20 $3 $6 $25 $80
 0-1 Knapsack problem:
brute-force approach
Let’s first solve this problem with a
straightforward algorithm
• Since there are n items, there are 2n
possible combinations of items.
• We go through all combinations and find
the one with maximum value and with total
weight less or equal to W
• Running time will be O(2n)
 A 0/1 Knapsack Algorithm,
First Attempt
• Sk: Set of items numbered 1 to k.
• Define V[k] = best selection from Sk.
• The question is: can we describe the final solution (Sn ) in
terms of subproblems (Sk)?
– Consider S={(4,20),(2,3),(2,6),(6,25),(2,80)} , W=9

Best for S4: 6 25

Best for S5: 20 6 80

Solution for S4 is not part of the solution for S5!!!

– This definition of subproblem does not have optimal substructure

 A 0/1 Knapsack Algorithm,
Second Attempt
• Let’s add another parameter: w, which will represent
the maximum weight for each subset of items
• The subproblem then will be to compute V[k,w]:
to find an optimal solution for Sk = {items labeled 1,
2, .. k} in a knapsack of size w
• Good news: this does have optimal substructure
 Recursive Formula for subproblems

Recursive formula for subproblems:

 V [ k  1, w ] if wk  w
V [k , w]  
 max{ V [ k  1, w ],V [ k  1, w  wk ]  bk } else

It means, that the best subset of Sk that has total weight w is:
1) the best subset of Sk-1 that has total weight  w, or
2) the best subset of Sk-1 that has total weight  w-wk plus
the item k
 Recursive Formula

 V [ k  1, w ] if wk  w
V [k , w]  
 max{ V [ k  1, w ],V [ k  1, w  wk ]  bk } else

• The best subset of Sk that has the total weight  w,

either contains item k or not.
• First case: wk>w. Item k can’t be part of the solution,
since if it was, the total weight would be > w, which is
unacceptable.
• Second case: wk  w. Then the item k can be in the
solution, and we choose the case with greater value.
 0-1 Knapsack Algorithm
for w = 0 to W
V[0,w] = 0
for i = 1 to n
V[i,0] = 0
for i = 1 to n
for w = 1 to W
if wi <= w // item i can be part of the solution
if bi + V[i-1,w-wi] > V[i-1,w]
V[i,w] = bi + V[i-1,w- wi]
else
V[i,w] = V[i-1,w]
else V[i,w] = V[i-1,w] // wi > w
 Running time
for w = 0 to W
V[0,w] = 0 O(W)
for i = 1 to n
V[i,0] = 0
for i = 1 to n
Repeat n times
for w = 1 to W
< the rest of the code > O(W)

What is the running time of this algorithm? O(n*W)

Not a polynomial-time algorithm if W is large
This is a pseudo-polynomial time algorithm
Remember: the brute-force algorithm takes O(2n)
 Example

Let’s run our algorithm on the

following data:

n = 4 (# of elements)
W = 5 (max weight)
Elements (weight, benefit):
(2,3), (3,4), (4,5), (5,6)
 Example (2)
i\W 0 1 2 3 4 5
0 0 0 0 0 0 0
1
2
3
4

for w = 0 to W
V[0,w] = 0
 Example (3)
i\W 0 1 2 3 4 5
0 0 0 0 0 0 0
1 0
2 0
3 0
4 0

for i = 1 to n
V[i,0] = 0
 Items:
Example (4) 1: (2,3)
2: (3,4)
3: (4,5)
i\W 0 1 2 3 4 5 i=1 4: (5,6)
0 0 0 0 0 0 0 bi=3
1 0 0
wi=2
2 0
w=1
3 0
w-wi =-1
4 0
if wi <= w // item i can be part of the solution
if bi + V[i-1,w-wi] > V[i-1,w]
V[i,w] = bi + V[i-1,w- wi]
else
V[i,w] = V[i-1,w]
else V[i,w] = V[i-1,w] // wi > w
 Items:
Example (5) 1: (2,3)
2: (3,4)
3: (4,5)
i\W 0 1 2 3 4 5 i=1 4: (5,6)
0 0 0 0 0 0 0 bi=3
1 0 0 3
wi=2
2 0
w=2
3 0
w-wi =0
4 0
if wi <= w // item i can be part of the solution
if bi + V[i-1,w-wi] > V[i-1,w]
V[i,w] = bi + V[i-1,w- wi]
else
V[i,w] = V[i-1,w]
else V[i,w] = V[i-1,w] // wi > w
 Items:
Example (6) 1: (2,3)
2: (3,4)
3: (4,5)
i\W 0 1 2 3 4 5 i=1 4: (5,6)
0 0 0 0 0 0 0 bi=3
1 0 0 3 3
wi=2
2 0
w=3
3 0
w-wi =1
4 0
if wi <= w // item i can be part of the solution
if bi + V[i-1,w-wi] > V[i-1,w]
V[i,w] = bi + V[i-1,w- wi]
else
V[i,w] = V[i-1,w]
else V[i,w] = V[i-1,w] // wi > w
 Items:
Example (7) 1: (2,3)
2: (3,4)
3: (4,5)
i\W 0 1 2 3 4 5 i=1 4: (5,6)
0 0 0 0 0 0 0 bi=3
1 0 0 3 3 3
wi=2
2 0
w=4
3 0
w-wi =2
4 0
if wi <= w // item i can be part of the solution
if bi + V[i-1,w-wi] > V[i-1,w]
V[i,w] = bi + V[i-1,w- wi]
else
V[i,w] = V[i-1,w]
else V[i,w] = V[i-1,w] // wi > w
 Items:
Example (8) 1: (2,3)
2: (3,4)
3: (4,5)
i\W 0 1 2 3 4 5 i=1 4: (5,6)
0 0 0 0 0 0 0 bi=3
1 0 0 3 3 3 3
wi=2
2 0
w=5
3 0
w-wi =3
4 0
if wi <= w // item i can be part of the solution
if bi + V[i-1,w-wi] > V[i-1,w]
V[i,w] = bi + V[i-1,w- wi]
else
V[i,w] = V[i-1,w]
else V[i,w] = V[i-1,w] // wi > w
 Items:
Example (9) 1: (2,3)
2: (3,4)
3: (4,5)
i\W 0 1 2 3 4 5 i=2 4: (5,6)
0 0 0 0 0 0 0 bi=4
1 0 0 3 3 3 3
wi=3
2 0 0
w=1
3 0
w-wi =-2
4 0
if wi <= w // item i can be part of the solution
if bi + V[i-1,w-wi] > V[i-1,w]
V[i,w] = bi + V[i-1,w- wi]
else
V[i,w] = V[i-1,w]
else V[i,w] = V[i-1,w] // wi > w
 Items:
Example (10) 1: (2,3)
2: (3,4)
3: (4,5)
i\W 0 1 2 3 4 5 i=2 4: (5,6)
0 0 0 0 0 0 0 bi=4
1 0 0 3 3 3 3
wi=3
2 0 0 3
w=2
3 0
w-wi =-
4 0
1
if wi <= w // item i can be part of the solution
if bi + V[i-1,w-wi] > V[i-1,w]
V[i,w] = bi + V[i-1,w- wi]
else
V[i,w] = V[i-1,w]
else V[i,w] = V[i-1,w] // wi > w
 Items:
Example (11) 1: (2,3)
2: (3,4)
3: (4,5)
i\W 0 1 2 3 4 5 i=2 4: (5,6)
0 0 0 0 0 0 0 bi=4
1 0 0 3 3 3 3
wi=3
2 0 0 3 4
w=3
3 0
w-wi =0
4 0
if wi <= w // item i can be part of the solution
if bi + V[i-1,w-wi] > V[i-1,w]
V[i,w] = bi + V[i-1,w- wi]
else
V[i,w] = V[i-1,w]
else V[i,w] = V[i-1,w] // wi > w
 Items:
Example (12) 1: (2,3)
2: (3,4)
3: (4,5)
i\W 0 1 2 3 4 5 i=2 4: (5,6)
0 0 0 0 0 0 0 bi=4
1 0 0 3 3 3 3
wi=3
2 0 0 3 4 4
w=4
3 0
w-wi =1
4 0
if wi <= w // item i can be part of the solution
if bi + V[i-1,w-wi] > V[i-1,w]
V[i,w] = bi + V[i-1,w- wi]
else
V[i,w] = V[i-1,w]
else V[i,w] = V[i-1,w] // wi > w
 Items:
Example (13) 1: (2,3)
2: (3,4)
3: (4,5)
i\W 0 1 2 3 4 5 i=2 4: (5,6)
0 0 0 0 0 0 0 bi=4
1 0 0 3 3 3 3
wi=3
2 0 0 3 4 4 7
w=5
3 0
w-wi =2
4 0
if wi <= w // item i can be part of the solution
if bi + V[i-1,w-wi] > V[i-1,w]
V[i,w] = bi + V[i-1,w- wi]
else
V[i,w] = V[i-1,w]
else V[i,w] = V[i-1,w] // wi > w
 Items:
Example (14) 1: (2,3)
2: (3,4)
3: (4,5)
i\W 0 1 2 3 4 5 i=3 4: (5,6)
0 0 0 0 0 0 0 bi=5
1 0 0 3 3 3 3
wi=4
2 0 0 3 4 4 7
w= 1..3
3 0 0 3 4
4 0
if wi <= w // item i can be part of the solution
if bi + V[i-1,w-wi] > V[i-1,w]
V[i,w] = bi + V[i-1,w- wi]
else
V[i,w] = V[i-1,w]
else V[i,w] = V[i-1,w] // wi > w
 Items:
Example (15) 1: (2,3)
2: (3,4)
3: (4,5)
i\W 0 1 2 3 4 5 i=3 4: (5,6)
0 0 0 0 0 0 0 bi=5
1 0 0 3 3 3 3
wi=4
2 0 0 3 4 4 7
w= 4
3 0 0 3 4 5
w- wi=0
4 0
if wi <= w // item i can be part of the solution
if bi + V[i-1,w-wi] > V[i-1,w]
V[i,w] = bi + V[i-1,w- wi]
else
V[i,w] = V[i-1,w]
else V[i,w] = V[i-1,w] // wi > w
 Items:
Example (16) 1: (2,3)
2: (3,4)
3: (4,5)
i\W 0 1 2 3 4 5 i=3 4: (5,6)
0 0 0 0 0 0 0 bi=5
1 0 0 3 3 3 3
wi=4
2 0 0 3 4 4 7
w= 5
3 0 0 3 4 5 7
w- wi=1
4 0
if wi <= w // item i can be part of the solution
if bi + V[i-1,w-wi] > V[i-1,w]
V[i,w] = bi + V[i-1,w- wi]
else
V[i,w] = V[i-1,w]
else V[i,w] = V[i-1,w] // wi > w
 Items:
Example (17) 1: (2,3)
2: (3,4)
3: (4,5)
i\W 0 1 2 3 4 5 i=4 4: (5,6)
0 0 0 0 0 0 0 bi=6
1 0 0 3 3 3 3
wi=5
2 0 0 3 4 4 7
w= 1..4
3 0 0 3 4 5 7
4 0 0 3 4 5
if wi <= w // item i can be part of the solution
if bi + V[i-1,w-wi] > V[i-1,w]
V[i,w] = bi + V[i-1,w- wi]
else
V[i,w] = V[i-1,w]
else V[i,w] = V[i-1,w] // wi > w
 Items:
Example (18) 1: (2,3)
2: (3,4)
3: (4,5)
i\W 0 1 2 3 4 5 i=4 4: (5,6)
0 0 0 0 0 0 0 bi=6
1 0 0 3 3 3 3
wi=5
2 0 0 3 4 4 7
w= 5
3 0 0 3 4 5 7
w- wi=0
4 0 0 3 4 5 7
if wi <= w // item i can be part of the solution
if bi + V[i-1,w-wi] > V[i-1,w]
V[i,w] = bi + V[i-1,w- wi]
else
V[i,w] = V[i-1,w]
else V[i,w] = V[i-1,w] // wi > w
 How to find actual Knapsack Items

• All of the information we need is in the table.

• V[n,W] is the maximal value of items that
can be placed in the Knapsack.
• Let i=n and k=W
if V[i,k]  V[i1,k] then
mark the ith item as in the knapsack
i = i1, k = k-wi
else
i = i1 // Assume the ith item is not in the
knapsack
 Items:
Finding the Items 1: (2,3)
2: (3,4)
3: (4,5)
i\W 0 1 2 3 4 5 i=4 4: (5,6)
0 0 0 0 0 0 0 k= 5
1 0 0 3 3 3 3 bi=6
2 0 0 3 4 4 7 wi=5
3 0 0 3 4 5 7 V[i,k] = 7
V[i1,k] =7
4 0 0 3 4 5 7
i=n, k=W
while i,k > 0
if V[i,k]  V[i1,k] then
mark the ith item as in the knapsack
i = i1, k = k-wi
else
i = i1
 Items:
Finding the Items (2) 1: (2,3)
2: (3,4)
3: (4,5)
i\W 0 1 2 3 4 5 i=4 4: (5,6)
0 0 0 0 0 0 0 k= 5
1 0 0 3 3 3 3 bi=6
2 0 0 3 4 4 7 wi=5
3 0 0 3 4 5 7 V[i,k] = 7
V[i1,k] =7
4 0 0 3 4 5 7
i=n, k=W
while i,k > 0
if V[i,k]  V[i1,k] then
mark the ith item as in the knapsack
i = i1, k = k-wi
else
i = i 1
 Items:
Finding the Items (3) 1: (2,3)
2: (3,4)
3: (4,5)
i\W 0 1 2 3 4 5 i=3 4: (5,6)
0 0 0 0 0 0 0 k= 5
1 0 0 3 3 3 3 bi=5
2 0 0 3 4 4 7 wi=4
3 0 0 3 4 5 7 V[i,k] = 7
V[i1,k] =7
4 0 0 3 4 5 7
i=n, k=W
while i,k > 0
if V[i,k]  V[i1,k] then
mark the ith item as in the knapsack
i = i1, k = k-wi
else
i = i 1
 Items:
Finding the Items (4) 1: (2,3)
2: (3,4)
3: (4,5)
i\W 0 1 2 3 4 5 i=2 4: (5,6)
0 0 0 0 0 0 0 k= 5
1 0 0 3 3 3 3 bi=4
2 0 0 3 4 4 7 wi=3
3 0 0 3 4 5 7 V[i,k] = 7
V[i1,k] =3
4 0 0 3 4 5 7
k  wi=2
i=n, k=W
while i,k > 0
if V[i,k]  V[i1,k] then
mark the ith item as in the knapsack
i = i1, k = k-wi
else
i = i 1
 Items:
Finding the Items (5) 1: (2,3)
2: (3,4)
3: (4,5)
i\W 0 1 2 3 4 5 i=1 4: (5,6)
0 0 0 0 0 0 0 k= 2
1 0 0 3 3 3 3 bi=3
2 0 0 3 4 4 7 wi=2
3 0 0 3 4 5 7 V[i,k] = 3
V[i1,k] =0
4 0 0 3 4 5 7
k  wi=0
i=n, k=W
while i,k > 0
if V[i,k]  V[i1,k] then
mark the ith item as in the knapsack
i = i1, k = k-wi
else
i = i 1
 Items:
Finding the Items (6) 1: (2,3)
2: (3,4)
3: (4,5)
i\W 0 1 2 3 4 5 i=0 4: (5,6)
0 0 0 0 0 0 0 k= 0
1 0 0 3 3 3 3
2 0 0 3 4 4 7
3 0 0 3 4 5 7 The optimal
knapsack
4 0 0 3 4 5 7
should contain
i=n, k=W {1, 2}
while i,k > 0
if V[i,k]  V[i1,k] then
mark the nth item as in the knapsack
i = i1, k = k-wi
else
i = i1
 The Fractional Knapsack Problem
• Fractional knapsack problem: you can take
any fraction of an item.

 Problem, in other words, is to find

n
max  f i bi subject to  f i wi  W
i 1 i 1

where 0  fi  1.
 Example
• Given: A set S of n items, with each item i having
– bi - a positive benefit
– wi - a positive weight
• Goal: Choose items with maximum total benefit but with
weight at most W.
“knapsack”

Solution:
• 1 ml of 5
Items:
1 2 3 4 5 • 2 ml of 3
• 6 ml of 4
Weight: 4 ml 8 ml 2 ml 6 ml 1 ml • 1 ml of 2
Benefit: $12 $32 $40 $30 $50 10 ml
Value: 3 4 20 5 50
($ per ml)
 The Fractional Knapsack
Algorithm
• Greedy choice: Keep taking Algorithm fractionalKnapsack(S, W)
item with highest value Input: set S of items : benefit bi
(benefit to weight ratio) and weight wi; max. weight W
Output: amount xi of each item i
– Run time: O(n log n). Why? to maximize benefit with
weight at most W
for each item i in S
xi  0
vi  bi / wi {value}
w0 {total weight}
while w < W
remove item i with highest vi
xi  min{wi , W - w}
w  w + min{wi , W - w}
 The Knapsack Problem:
Greedy Vs. Dynamic
• The optimal solution to the fractional knapsack
problem can be found with a greedy algorithm
– Greedy strategy: take in order of dollars/pound
• The optimal solution to the 0-1 problem cannot
be found with the same greedy strategy
– Example: 3 items weighing 10, 20, and 30 pounds,
with values 80, 100, and 90 dollars, knapsack can
hold 50 pounds
bi/wi 80/10=8, 100/20=5, 90/30=3
• For the 0-1 problem, some special instances (i.e.,
where all wi and W are integer values) can be
solved with a dynamic programming approach