Dynamic Programming

Overview of Serial Dynamic Programming

• Dynamic programming (DP) is used to solve a wide

variety of discrete optimization problems such as
scheduling, string-editing, packaging, and inventory
management.
• Break problems into subproblems and combine their
solutions into solutions to larger problems.
• In contrast to divide-and-conquer, there may be
relationships across subproblems.
 Dynamic Programming: Example

• Consider the problem of finding a shortest path between

a pair of vertices in an acyclic graph.
• An edge connecting node i to node j has cost c(i,j).
• The graph contains n nodes numbered 0,1,…, n-1, and
has an edge from node i to node j only if i < j. Node 0 is
source and node n-1 is the destination.
• Let f(x) be the cost of the shortest path from node 0 to
node x.
 Dynamic Programming: Example

• A graph for which the shortest path between nodes 0

and 4 is to be computed.
 Dynamic Programming

• The solution to a DP problem is typically expressed as a

minimum (or maximum) of possible alternate solutions.
• If r represents the cost of a solution composed of
subproblems x1, x2,…, xl, then r can be written as

Here, g is the composition function.

• If the optimal solution to each problem is determined by
composing optimal solutions to the subproblems and
selecting the minimum (or maximum), the formulation is
said to be a DP formulation.
 Dynamic Programming: Example

The computation and composition of subproblem solutions

to solve problem f(x8).
 Dynamic Programming

• The recursive DP equation is also called the functional

equation or optimization equation.
• In the equation for the shortest path problem the
composition function is f(j) + c(j,x). This contains a single
recursive term (f(j)). Such a formulation is called
monadic.
• If the RHS has multiple recursive terms, the DP
formulation is called polyadic.
 Dynamic Programming

• The dependencies between subproblems can be

expressed as a graph.
• If the graph can be levelized (i.e., solutions to problems
at a level depend only on solutions to problems at the
previous level), the formulation is called serial, else it is
called non-serial.
• Based on these two criteria, we can classify DP
formulations into four categories - serial-monadic, serial-
polyadic, non-serial-monadic, non-serial-polyadic.
• This classification is useful since it identifies concurrency
and dependencies that guide parallel formulations.
 0/1 Knapsack Problem

• We are given a knapsack of capacity c and a set of n objects

numbered 1,2,…,n. Each object i has weight wi and profit pi.
• Let v = [v1, v2,…, vn] be a solution vector in which vi = 0 if object i is
not in the knapsack, and vi = 1 if it is in the knapsack.
• The goal is to find a subset of objects to put into the knapsack so
that

(that is, the objects fit into the knapsack) and

is maximized (that is, the profit is maximized).

 0/1 Knapsack Problem

• The naive method is to consider all 2n possible subsets

of the n objects and choose the one that fits into the
knapsack and maximizes the profit.
• Let F[i,x] be the maximum profit for a knapsack of
capacity x using only objects {1,2,…,i}. The DP
formulation is:
 0/1 Knapsack Problem

• Construct a table F of size n x c in row-major order.

• Filling an entry in a row requires two entries from the
previous row: one from the same column and one from
the column offset by the weight of the object
corresponding to the row.
• Computing each entry takes constant time; the
sequential run time of this algorithm is Θ(nc).
• The formulation is serial-monadic.
 0/1 Knapsack Problem

Computing entries of table F for the 0/1 knapsack problem. The computation of
entry F[i,j] requires communication with processing elements containing
entries F[i-1,j] and F[i-1,j-wi].
 0/1 Knapsack Problem

• Using c processors in a PRAM, we can derive a simple

parallel algorithm that runs in O(n) time by partitioning
the columns across processors.
• In a distributed memory machine, in the jth iteration, for
computing F[j,r] at processing element Pr-1, F[j-1,r] is
available locally but F[j-1,r-wj] must be fetched.
• The communication operation is a circular shift and the
time is given by (ts + tw) log c. The total time is therefore
tc + (ts + tw) log c.
• Across all n iterations (rows), the parallel time is O(n log
c). Note that this is not cost optimal.
 0/1 Knapsack Problem

• Using p-processing elements, each processing element

computes c/p elements of the table in each iteration.
• The corresponding shift operation takes time (2ts + twc/p),
since the data block may be partitioned across two
processors, but the total volume of data is c/p.
• The corresponding parallel time is n(tcc/p + 2ts + twc/p),
or O(nc/p) (which is cost-optimal).
• Note that there is an upper bound on the efficiency of
this formulation.
 Warshall Floyd's All-Pairs Shortest Path

• A PRAM formulation of this algorithm uses n2 processors

in a logical 2D mesh. Processor Pi,j computes the value
of dik,j for k=1,2,…,n in constant time.
• The parallel runtime is Θ(n) and it is cost-optimal.
• The algorithm can easily be adapted to practical
architectures, as discussed in our treatment of Graph
Algorithms.
 Warshall Floyd's All-Pairs Shortest Path

• When multiplying a sequence of matrices, the order of

multiplication significantly impacts operation count.
• Let C[i,j] be the optimal cost of multiplying the matrices
Ai,…Aj.
• The chain of matrices can be expressed as a product of
two smaller chains, Ai,Ai+1,…,Ak and Ak+1,…,Aj.
• The chain Ai,Ai+1,…,Ak results in a matrix of dimensions
ri-1 x rk, and the chain Ak+1,…,Aj results in a matrix of
dimensions rk x rj.
• The cost of multiplying these two matrices is ri-1rkrj.
 Optimal Matrix-chain multiplication Problem

• We have:
Optimal Matrix-chain multiplication Problem

A nonserial polyadic DP formulation for finding an optimal matrix

parenthesization for a chain of four matrices. A square node
represents the optimal cost of multiplying a matrix chain. A circle
node represents a possible parenthesization.
 Optimal Matrix-chain multiplication Problem

• The goal of finding C[1,n] is accomplished in a bottom-up

fashion.
• Visualize this by thinking of filling in the C table
diagonally. Entries in diagonal l corresponds to the cost
of multiplying matrix chains of length l+1.
• The value of C[i,j] is computed as min{C[i,k] + C[k+1,j] +
ri-1rkrj}, where k can take values from i to j-1.
• Computing C[i,j] requires that we evaluate (j-i) terms and
select their minimum.
• The computation of each term takes time tc, and the
computation of C[i,j] takes time (j-i)tc. Each entry in
diagonal l can be computed in time ltc.
 Optimal Matrix-chain multiplication Problem

• The algorithm computes (n-1) chains of length two. This

takes time (n-1)tc; computing n-2 chains of length three
takes time (n-2)tc. In the final step, the algorithm
computes one chain of length n in time (n-1)tc.
• It follows that the serial time is Θ(n3).
Optimal Matrix-chain multiplication Problem

The diagonal order of computation for the optimal matrix-

parenthesization problem.
 Parallel Optimal Matrix-Parenthesization
Problem
• Consider a logical ring of processors. In step l, each processor computes a
single element belonging to the lth diagonal.
• On computing the assigned value of the element in table C, each processor
sends its value to all other processors using an all-to-all broadcast.
• The next value can then be computed locally.
• The total time required to compute the entries along diagonal l is ltc+tslog
n+tw(n-1).
• The corresponding parallel time is given by:
 Parallel Optimal Matrix-Parenthesization
Problem
• When using p (<n) processors, each processor stores n/p nodes.
• The time taken for all-to-all broadcast of n/p words is

and the time to compute n/p entries of the table in the lth diagonal is
ltcn/p.

• This formulation can be improved to use up to n(n+1)/2 processors

using pipelining.
 Discussion of Parallel Dynamic Programming
Algorithms

• By representing computation as a graph, we identify

three sources of parallelism: parallelism within nodes,
parallelism across nodes at a level, and pipelining nodes
across multiple levels. The first two are available in serial
formulations and the third one in non-serial formulations.

• Data locality is critical for performance. Different DP

formulations, by the very nature of the problem instance,
have different degrees of locality.
Optimal Binary Search Tree
 1.Preface

• OBST is one special kind of advanced tree.

• It focus on how to reduce the cost of the search

of the BST.

• It may not have the lowest height !

• It needs 3 tables to record probabilities, cost,

and root.
 2.Premise
• It has n keys (representation k1,k2,…,kn) in sorted order
(so that k1<k2<…<kn), and we wish to build a binary
search tree from these keys. For each ki ,we have a
probability pi that a search will be for ki.
• In contrast of, some searches may be for values not in ki,
and so we also have n+1 “dummy keys” d0,d1,…,dn
representating not in ki.
• In particular, d0 represents all values less than k1, and dn
represents all values greater than kn, and for i=1,2,…,n-
1, the dummy key di represents all values between ki and
ki+1.
＊The dummy keys are leaves (external nodes), and the
data keys mean internal nodes.
 3.Formula & Prove

• The case of search are two situations, one is success,

and the other, without saying, is failure.

• We can get the first statement :

(i=1~n) ∑ pi + (i=0~n) ∑ qi = 1

Success Failure
• Because we have probabilities of searches for each key
and each dummy key, we can determine the expected
cost of a search in a given binary search tree T. Let us
assume that the actual cost of a search is the number of
nodes examined, i.e., the depth of the node found by the
search in T,plus1. Then the expected cost of a search in
T is : (The second statement)

• E[ search cost in T]
= (i=1~n) ∑ pi ．(depthT(ki)+1)
+ (i=0~n) ∑ qi ．(depthT(di)+1)
=1 + (i=1~n) ∑ pi ．depthT(ki)
+ (i=0~n) ∑ qi ．depthT(di)
Where depthT denotes a node’s depth in the tree T.
 k2 k2

k1 k4 k1 k5

d0 d1
d0 d1 d5
k3 k5 k4

d2 d3 d4 d5 d4
k3
Figure (a)

i 0 1 2 3 4 5
d2 d3

pi 0.15 0.10 0.05 0.10 0.20

Figure (b)

qi 0.05 0.10 0.05 0.05 0.05 0.10

• By Figure (a), we can calculate the expected search cost node by node:

Cost=
Node# Depth probability cost
Probability *
k1 1 0.15 0.30 (Depth+1)

k2 0 0.10 0.10
k3 2 0.05 0.15
k4 1 0.10 0.20
K5 2 0.20 0.60
d0 2 0.05 0.15
d1 3 0.10 0.30
d2 3 0.05 0.20
d3 3 0.05 0.20
d4 3 0.05 0.20
d5 3 0.10 0.40
• And the total cost = (0.30 + 0.10 + 0.15 + 0.20 + 0.60 +
0.15 + 0.30 + 0.20 + 0.20 + 0.20 + 0.40 ) = 2.80
• So Figure (a) costs 2.80 ,on another, the Figure (b) costs
2.75, and that tree is really optimal.
• We can see the height of (b) is more than (a) , and the
key k5 has the greatest search probability of any key, yet
the root of the OBST shown is k2.(The lowest expected
cost of any BST with k5 at the root is 2.85)
 Step1:The structure of an OBST

• To characterize the optimal substructure of OBST, we

start with an observation about subtrees. Consider any
subtree of a BST. It must contain keys in a contiguous
range ki,…,kj, for some 1≦i ≦j ≦n. In addition, a subtree
that contains keys ki,…,kj must also have as its leaves
the dummy keys di-1 ,…,dj.
• We need to use the optimal substructure to
show that we can construct an optimal solution
to the problem from optimal solutions to
subproblems. Given keys ki ,…, kj, one of these
keys, say kr (I ≦r ≦j), will be the root of an
optimal subtree containing these keys. The left
subtree of the root kr will contain the keys (ki ,…,
kr-1) and the dummy keys( di-1 ,…, dr-1), and the
right subtree will contain the keys (kr+1 ,…, kj)
and the dummy keys( dr ,…, dj). As long as we
examine all candidate roots kr, where I ≦r ≦j,
and we determine all optimal binary search trees
containing ki ,…, kr-1 and those containing kr+1
,…, kj , we are guaranteed that we will find an
OBST.
 Step2: A recursive solution

• We are ready to define the value of an optimal

solution recursively. We pick our subproblem
domain as finding an OBST containing the keys
ki,…,kj, where i≧1, j ≦n, and j ≧ i-1. (It is when
j=i-1 that ther are no actual keys; we have just
the dummy key di-1.)
• Let us define e[i,j] as the expected cost of
searching an OBST containing the keys ki,…, kj.
Ultimately, we wish to compute e[1,n].
• The easy case occurs when j=i-1. Then we have just the
dummy key di-1. The expected search cost is e[i,i-1]= qi-1.
• When j≧1, we need to select a root krfrom among ki,…,kj
and then make an OBST with keys ki,…,kr-1 its left
subtree and an OBST with keys kr+1,…,kj its right
subtree. By the time, what happens to the expected
search cost of a subtree when it becomes a subtree of a
node? The answer is that the depth of each node in the
subtree increases by 1.
 OPTIMAL—BST(p,q,n)
• For i 1 to n+1
do e[i,i-1] qi-1
do w[i,i-1] qi-1
For l 1 to n
do for i 1 to n-l +1
do j i+l-1
e[i,j] ∞
w[i,j] w[i,j-1]+pj+qj
For r i to j
do t e[i,r-1]+e[r+1,j]+w[i,j]
if t<e[i,j]
then e[i,j] t
root [i,j] r
Return e and root
 e w
5 1
5 1
4 2.75 2
4 1.00 2
3 1.75 2.00 3 0.70 0.80
1.25 3 3
2 1.20 1.30 4 0.55
0.90 5 2 0.50 0.60 4
1 0.70 0.60 0.90
0.45 0.35
0.25 0.50 1 0.30 0.50 5
0 0.45 0.40 0.30 6
0 0.30 0.25 0.15 0.20 0.35 6
0.05 0.10 0.05 0.05 0.05 0.10
0.05 0.10 0.05 0.05 0.05 0.10

root
5 1
4 2 2
3 2 4 3
2 2 2 5 4
1 2 4 5 5
1
1 2 3 4 5

The tables e[i,j], w[i,j], and root [i,j]computed by Optimal-BST