Introduction to Algorithms

Class 18: Dynamic programming 4

Jianxi Gao

Department of Computer Science

Rensselaer Polytechnic Institute
www.gaojianxi.com
Subsequence

Definitions
A sequence of numbers with n elements: 𝑆 = {𝑎1 , 𝑎2 , … , 𝑎𝑛 }
A subsequence of 𝑆 with m elements: Ss = {𝑎𝑗1 , 𝑎𝑗2 , … , 𝑎𝑗𝑚 }, where 𝑗1 < 𝑗2 < ⋯ < 𝑗𝑚
An increasing subsequence of 𝑆: Ss = {𝑎𝑗1 , 𝑎𝑗2 , … , 𝑎𝑗𝑚 }, where 𝑎𝑗1 < 𝑎𝑗2 < ⋯ < 𝑎𝑗𝑚 .
Longest common subsequence: Given two sequences, find the length of the
longest subsequence presented in both of them.
Examples
A sequence: 𝑆 = 5, 2, 8, 6, 3, 6, 9, 7 .
subsequences of S: 5, 2, 8 , 5, 8, 6,6 , 2, 6, 9 , 2
Increasing subsequences of S: 5,8 , 2,3,6,9 , 2 5,6,7
Longest Increasing subsequences of S: 2,3,6,9 .
For example: “attaaaggtt” and “cacgcagtat”
The longest common sequences = ”at”, “ag”, “ta”, “gt”
Longest increasing subsequence algorithm

for i = 6 i from 1 to length(S)-1

for j = 5 j from 0 to i-1

Iterator j i
S 5 2 8 6 3 6 9 7
LIS 1 1 2 2 2 3 3, 4 1

if 𝑆 𝑖 > 𝑆 𝑗 && 𝑇𝑎𝑏𝑙𝑒 𝑖 < 𝑇𝑎𝑏𝑙𝑒 𝑗 + 1

Table[i] = Table[j] + 1;
Dynamic programming (Bottom-Up)

LCS Empty A T A G C
Initialize:

Zero common Empty 0 0 0 0 0 0

subsequence
G 0 0 (U) 0 (U) 0 (U) 1(M)

T 0 L(i-1,j-1) L(i,j-1)
If L1(i) and L2(j) match,
L(i,j) = L(i-1,j-1) +1
G 0 L(i-1,j) L(i,j)
If L1(i) and L2(j) do not match,
L(i,j) = max(L(i-1,j), L(I,j-1)).
A 0

C 0
Introduction
I love her.
I loeve har.

What is the distance between “loeve” and “love”, and what is the distance between
“loeve” and “leave”? How to give the suggestions to the writers?
Introduction

The natural measure of the distance between two strings is the extent to which
they can be aligned, or matched.

String 1 L O E V E
String 2 L O V E

Approach 1 Cost: 3
String 1 L O E, V V, E E
String 2 L O V E

Approach 2 Cost: 1
String 1 L O E V E
String 2 L O V E
Edit distance

Definitions
Given two strings String 1 and string 2 and below operations that can be
performed on string 1. Find the minimum number of operations or edits
required to convert string 1 to string 2.
Edits are:
1. Insert String 1 H A R
2. Delete String 2 H E R Cost: 1
3. Modify Edits H A, E R
Examples
String 1 L O E V E String 1 L O E V E
String 2 L E V E E String 2 L E V E E
Edits L O,E E,V V,E E Edits L O E V E E

Cost: 3 Cost: 2
The Approach

Given two strings of length m and n.

1. If the last letters of the two strings match, we do not change anything
and recur for length m-1 and n-1.

2. Else we compute minimum cost of all three operations (insert, delete,

and modify) and take minimum of these three values.
a. Insert.: Recur for m and n-1
b. Delete: Recur for m-1 and n
c. Modify: Recur for m-1 and n-1
Recursive solution
D(LOEVE,LOVE)
Minimum number of edits from
“LOEVE to “LOVE””
Recursive solution
“L” Matches “L”, we need D(LOEVE,LOVE)
0 edit.
D(OEVE,OVE)
Recursive solution
“O” Matches “O”, we D(LOEVE,LOVE)
need 0 edit.
D(OEVE,OVE)
It means D(LOEVE,LOVE)
= D(EVE,VE) + 0.
D(EVE,VE)
Recursive solution
“E” does not match “V”, D(LOEVE,LOVE)
we have three choices .
But we are not sure D(OEVE,OVE)
which one is the best.
D(EVE,VE)
Recursive solution
D(LOEVE,LOVE)
Choice 1,
Insert V in string 1 to
match the V in string D(OEVE,OVE)
2. The cost is 1.
D(EVE,VE)

D(VEVE,VE)
Recursive solution
D(LOEVE,LOVE) Choice 2,
Choice 1,
Insert V in string 1 to Delete E in string 1.
match the V in string D(OEVE,OVE)
2. The cost is 1.
D(EVE,VE)

D(VEVE,VE) D(VE,VE)
Recursive solution
D(LOEVE,LOVE) Choice 2, Choice 3,
Choice 1,
Delete E in string 1. Modify E to be V.
Insert V in string 1 to
The cost is 1. The cost is 1.
match the V in string D(OEVE,OVE)
2. The cost is 1.
D(EVE,VE)

D(VEVE,VE) D(VE,VE) D(VVE,VE)

Recursive solution
D(LOEVE,LOVE)
V matches V.
Cost 0.
D(OEVE,OVE)

D(EVE,VE)

D(VEVE,VE) D(VE,VE) D(VVE,VE)

D(EVE,E)
Recursive solution
D(LOEVE,LOVE)
E matches E.
Cost 0.
D(OEVE,OVE)

D(EVE,VE)

D(VEVE,VE) D(VE,VE) D(VVE,VE)

D(EVE,E)

D(VE,[])
Recursive solution
D(LOEVE,LOVE)
Delete V
Cost 1.
D(OEVE,OVE)

D(EVE,VE)

D(VEVE,VE) D(VE,VE) D(VVE,VE)

D(EVE,E)

D(VE,[])

D(E,[])
Recursive solution
D(LOEVE,LOVE)
Delete E
Cost 1.
D(OEVE,OVE)

D(EVE,VE)

D(VEVE,VE) D(VE,VE) D(VVE,VE)

D(EVE,E)

D(VE,[])

D(E,[])

D([],[])
Recursive solution
D(LOEVE,LOVE) x Means number of edits.

D(OEVE,OVE)

D(EVE,VE)

D(VEVE,VE) D(VE,VE) D(VVE,VE)

D(EVE,E)

D(VE,[])

D(E,[])

0 D([],[])
Recursive solution
D(LOEVE,LOVE) x Means number of edits.

D(OEVE,OVE)

D(EVE,VE)

D(VEVE,VE) D(VE,VE) D(VVE,VE)

D(EVE,E)

D(VE,[])

1 D(E,[])

0 D([],[])
Recursive solution
D(LOEVE,LOVE) x Means number of edits.

D(OEVE,OVE)

D(EVE,VE)

D(VEVE,VE) D(VE,VE) D(VVE,VE)

D(EVE,E)

2 D(VE,[])

1 D(E,[])

0 D([],[])
Recursive solution
D(LOEVE,LOVE) x Means number of edits.

D(OEVE,OVE)

D(EVE,VE)

D(VEVE,VE) D(VE,VE) D(VVE,VE)

2 D(EVE,E)

2 D(VE,[])

1 D(E,[])

0 D([],[])
Recursive solution
D(LOEVE,LOVE) x Means number of edits.

D(OEVE,OVE)

D(EVE,VE)

2 D(VEVE,VE) D(VE,VE) D(VVE,VE)

2 D(EVE,E)

2 D(VE,[])

1 D(E,[])

0 D([],[])
Recursive solution
D(LOEVE,LOVE)
V matches V.
Cost 0.
D(OEVE,OVE)

D(EVE,VE)

2 D(VEVE,VE) D(VE,VE) D(VVE,VE)

2 D(EVE,E) D(E,E)

2 D(VE,[])

1 D(E,[])

0 D([],[])
Recursive solution
D(LOEVE,LOVE)
E matches E.
Cost 0.
D(OEVE,OVE)

D(EVE,VE)

2 D(VEVE,VE) D(VE,VE) D(VVE,VE)

2 D(EVE,E) D(E,E)

2 D(VE,[]) D([],[])

1 D(E,[])

0 D([],[])
Recursive solution
D(LOEVE,LOVE)
E matches E.
Cost 0.
D(OEVE,OVE)

D(EVE,VE)

2 D(VEVE,VE) D(VE,VE) D(VVE,VE)

2 D(EVE,E) D(E,E)

2 D(VE,[]) 0 D([],[])

1 D(E,[])

0 D([],[])
Recursive solution
D(LOEVE,LOVE)
E matches E.
Cost 0.
D(OEVE,OVE)

D(EVE,VE)

2 D(VEVE,VE) 0 D(VE,VE) D(VVE,VE)

2 D(EVE,E) 0 D(E,E)

2 D(VE,[]) 0 D([],[])

1 D(E,[])

0 D([],[])
Recursive solution
D(LOEVE,LOVE)
V matches V.
Cost 0.
D(OEVE,OVE)

D(EVE,VE)

2 D(VEVE,VE) 0 D(VE,VE) D(VVE,VE)

2 D(EVE,E) 0 D(E,E) D(VE,E)

2 D(VE,[]) 0 D([],[])

1 D(E,[])

0 D([],[])
Recursive solution
“V” does not match “E”, D(LOEVE,LOVE)
we have three choices .
But we are not sure D(OEVE,OVE)
which one is the best.
D(EVE,VE)

2 D(VEVE,VE) 0 D(VE,VE) D(VVE,VE)

2 D(EVE,E) 0 D(E,E) D(VE,E)

2 D(VE,[]) 0 D([],[])

1 D(E,[])

0 D([],[])
Recursive solution
D(LOEVE,LOVE)
Choice 1,
Insert E in string 1 to
match the E in string D(OEVE,OVE)
2. The cost is 1.
D(EVE,VE)

2 D(VEVE,VE) 0 D(VE,VE) D(VVE,VE)

2 D(EVE,E) 0 D(E,E) D(VE,E)

2 D(VE,[]) 0 D([],[]) D(EVE,E)

1 D(E,[])

0 D([],[])
Recursive solution
D(LOEVE,LOVE) Choice 2,
Choice 1,
Insert E in string 1 to Delete V in string 1.
match the E in string D(OEVE,OVE) The cost is 1.
2. The cost is 1.
D(EVE,VE)

2 D(VEVE,VE) 0 D(VE,VE) D(VVE,VE)

2 D(EVE,E) 0 D(E,E) D(VE,E)

2 D(VE,[]) 0 D([],[]) D(EVE,E) D(E,E)

1 D(E,[])

0 D([],[])
Recursive solution
D(LOEVE,LOVE) Choice 2, Choice 3,
Choice 1,
Delete V in string 1. Modify V to be E.
Insert E in string 1 to
The cost is 1 The cost is 1.
match the E in string D(OEVE,OVE)
2. The cost is 1.
D(EVE,VE)

2 D(VEVE,VE) 0 D(VE,VE) D(VVE,VE)

2 D(EVE,E) 0 D(E,E) D(VE,E)

2 D(VE,[]) 0 D([],[]) D(EVE,E) D(E,E) D(EE,E)

1 D(E,[])

0 D([],[])
Recursive solution
D(LOEVE,LOVE)
E matches E.
Cost 0.
D(OEVE,OVE)

D(EVE,VE)

2 D(VEVE,VE) 0 D(VE,VE) D(VVE,VE)

2 D(EVE,E) 0 D(E,E) D(VE,E)

2 D(VE,[]) 0 D([],[]) D(EVE,E) D(E,E) D(EE,E)

1 D(E,[]) L(VE,[])

0 D([],[])
Recursive solution
D(LOEVE,LOVE)
Delete V.
The cost is 1.
D(OEVE,OVE)

D(EVE,VE)

2 D(VEVE,VE) 0 D(VE,VE) D(VVE,VE)

2 D(EVE,E) 0 D(E,E) D(VE,E)

2 D(VE,[]) 0 D([],[]) D(EVE,E) D(E,E) D(EE,E)

1 D(E,[]) L(VE,[])

0 D([],[]) D(E,[])
Recursive solution
D(LOEVE,LOVE)
Delete E.
The cost is 1.
D(OEVE,OVE)

D(EVE,VE)

2 D(VEVE,VE) 0 D(VE,VE) D(VVE,VE)

2 D(EVE,E) 0 D(E,E) D(VE,E)

2 D(VE,[]) 0 D([],[]) D(EVE,E) D(E,E) D(EE,E)

1 D(E,[]) L(VE,[])

0 D([],[]) D([],[]) D(E,[])

Recursive solution
D(LOEVE,LOVE)

D(OEVE,OVE)

D(EVE,VE)

2 D(VEVE,VE) 0 D(VE,VE) D(VVE,VE)

2 D(EVE,E) 0 D(E,E) D(VE,E)

2 D(VE,[]) 0 D([],[]) D(EVE,E) D(E,E) D(EE,E)

1 D(E,[]) L(VE,[])

0 D([],[]) 0 D([],[]) D(E,[])

Recursive solution
D(LOEVE,LOVE)

D(OEVE,OVE)

D(EVE,VE)

2 D(VEVE,VE) 0 D(VE,VE) D(VVE,VE)

2 D(EVE,E) 0 D(E,E) D(VE,E)

2 D(VE,[]) 0 D([],[]) D(EVE,E) D(E,E) D(EE,E)

1 D(E,[]) L(VE,[])

0 D([],[]) 0 D([],[]) 1 D(E,[])

Recursive solution
D(LOEVE,LOVE)

D(OEVE,OVE)

D(EVE,VE)

2 D(VEVE,VE) 0 D(VE,VE) D(VVE,VE)

2 D(EVE,E) 0 D(E,E) D(VE,E)

2 D(VE,[]) 0 D([],[]) D(EVE,E) D(E,E) D(EE,E)

1 D(E,[]) 2 L(VE,[])

0 D([],[]) 0 D([],[]) 1 D(E,[])

Recursive solution
D(LOEVE,LOVE)

D(OEVE,OVE)

D(EVE,VE)

2 D(VEVE,VE) 0 D(VE,VE) D(VVE,VE)

2 D(EVE,E) 0 D(E,E) D(VE,E)

2 D(VE,[]) 0 D([],[]) 2 D(EVE,E) D(E,E) D(EE,E)

1 D(E,[]) 2 L(VE,[])

0 D([],[]) 0 D([],[]) 1 D(E,[])

Recursive solution
D(LOEVE,LOVE)
E matches E.
Cost 0.
D(OEVE,OVE)

D(EVE,VE)

2 D(VEVE,VE) 0 D(VE,VE) D(VVE,VE)

2 D(EVE,E) 0 D(E,E) D(VE,E)

2 D(VE,[]) 0 D([],[]) 2 D(EVE,E) D(E,E) D(EE,E)

1 D(E,[]) 2 L(VE,[]) D([],[])

0 D([],[]) 0 D([],[]) 1 D(E,[])

Recursive solution
D(LOEVE,LOVE)

D(OEVE,OVE)

D(EVE,VE)

2 D(VEVE,VE) 0 D(VE,VE) D(VVE,VE)

2 D(EVE,E) 0 D(E,E) D(VE,E)

2 D(VE,[]) 0 D([],[]) 2 D(EVE,E) 0 D(E,E) D(EE,E)

1 D(E,[]) 2 L(VE,[]) 0 D([],[])

0 D([],[]) 0 D([],[]) 1 D(E,[])

Recursive solution
D(LOEVE,LOVE)
E matches E.
Cost 0.
D(OEVE,OVE)

D(EVE,VE)

2 D(VEVE,VE) 0 D(VE,VE) D(VVE,VE)

2 D(EVE,E) 0 D(E,E) D(VE,E)

2 D(VE,[]) 0 D([],[]) 2 D(EVE,E) 0 D(E,E) D(EE,E)

1 D(E,[]) 2 L(VE,[]) 0 D([],[]) D(E,[])

0 D([],[]) 0 D([],[]) 1 D(E,[])

Recursive solution
D(LOEVE,LOVE)
Delete E.
Cost 1.
D(OEVE,OVE)

D(EVE,VE)

2 D(VEVE,VE) 0 D(VE,VE) D(VVE,VE)

2 D(EVE,E) 0 D(E,E) D(VE,E)

2 D(VE,[]) 0 D([],[]) 2 D(EVE,E) 0 D(E,E) D(EE,E)

1 D(E,[]) 2 L(VE,[]) 0 D([],[]) D(E,[])

0 D([],[]) 0 D([],[]) 1 D(E,[]) D([],[])

Recursive solution
D(LOEVE,LOVE)

D(OEVE,OVE)

D(EVE,VE)

2 D(VEVE,VE) 0 D(VE,VE) D(VVE,VE)

2 D(EVE,E) 0 D(E,E) D(VE,E)

2 D(VE,[]) 0 D([],[]) 2 D(EVE,E) 0 D(E,E) D(EE,E)

1 D(E,[]) 2 L(VE,[]) 0 D([],[]) D(E,[])

0 D([],[]) 0 D([],[]) 1 D(E,[]) 0 D([],[])

Recursive solution
D(LOEVE,LOVE)

D(OEVE,OVE)

D(EVE,VE)

2 D(VEVE,VE) 0 D(VE,VE) D(VVE,VE)

2 D(EVE,E) 0 D(E,E) D(VE,E)

2 D(VE,[]) 0 D([],[]) 2 D(EVE,E) 0 D(E,E) D(EE,E)

1 D(E,[]) 2 L(VE,[]) 0 D([],[]) 1 D(E,[])

0 D([],[]) 0 D([],[]) 1 D(E,[]) 0 D([],[])

Recursive solution
D(LOEVE,LOVE)

D(OEVE,OVE)

D(EVE,VE)

2 D(VEVE,VE) 0 D(VE,VE) D(VVE,VE)

2 D(EVE,E) 0 D(E,E) D(VE,E)

2 D(VE,[]) 0 D([],[]) 2 D(EVE,E) 0 D(E,E) 1 D(EE,E)

1 D(E,[]) 2 L(VE,[]) 0 D([],[]) 1 D(E,[])

0 D([],[]) 0 D([],[]) 1 D(E,[]) 0 D([],[])

Recursive solution
D(LOEVE,LOVE)
D(VE,E) = 1 +
min{D(EVE,E),D(E,VE),D(EE,E)}
D(OEVE,OVE)

D(EVE,VE)

2 D(VEVE,VE) 0 D(VE,VE) D(VVE,VE)

2 D(EVE,E) 0 D(E,E) 1 D(VE,E)

2 D(VE,[]) 0 D([],[]) 2 D(EVE,E) 0 D(E,E) 1 D(EE,E)

1 D(E,[]) 2 L(VE,[]) 0 D([],[]) 1 D(E,[])

0 D([],[]) 0 D([],[]) 1 D(E,[]) 0 D([],[])

Recursive solution
D(LOEVE,LOVE)

D(OEVE,OVE)

D(EVE,VE)

2 D(VEVE,VE) 0 D(VE,VE) 1 D(VVE,VE)

2 D(EVE,E) 0 D(E,E) 1 D(VE,E)

2 D(VE,[]) 0 D([],[]) 2 D(EVE,E) 0 D(E,E) 1 D(EE,E)

1 D(E,[]) 2 L(VE,[]) 0 D([],[]) 1 D(E,[])

0 D([],[]) 0 D([],[]) 1 D(E,[]) 0 D([],[])

Recursive solution
D(LOEVE,LOVE)
D(EVE,VE) = 1 +
min{D(VEVE,VE),D(VE,VE),D(VVE,VE)}
D(OEVE,OVE)

1 D(EVE,VE)

2 D(VEVE,VE) 0 D(VE,VE) 1 D(VVE,VE)

2 D(EVE,E) 0 D(E,E) 1 D(VE,E)

2 D(VE,[]) 0 D([],[]) 2 D(EVE,E) 0 D(E,E) 1 D(EE,E)

1 D(E,[]) 2 L(VE,[]) 0 D([],[]) 1 D(E,[])

0 D([],[]) 0 D([],[]) 1 D(E,[]) 0 D([],[])

Recursive solution
D(LOEVE,LOVE)

1 D(OEVE,OVE)

1 D(EVE,VE)

2 D(VEVE,VE) 0 D(VE,VE) 1 D(VVE,VE)

2 D(EVE,E) 0 D(E,E) 1 D(VE,E)

2 D(VE,[]) 0 D([],[]) 2 D(EVE,E) 0 D(E,E) 1 D(EE,E)

1 D(E,[]) 2 L(VE,[]) 0 D([],[]) 1 D(E,[])

0 D([],[]) 0 D([],[]) 1 D(E,[]) 0 D([],[])

Recursive solution
1 D(LOEVE,LOVE)

1 D(OEVE,OVE)

1 D(EVE,VE)

2 D(VEVE,VE) 0 D(VE,VE) 1 D(VVE,VE)

2 D(EVE,E) 0 D(E,E) 1 D(VE,E)

2 D(VE,[]) 0 D([],[]) 2 D(EVE,E) 0 D(E,E) 1 D(EE,E)

1 D(E,[]) 2 L(VE,[]) 0 D([],[]) 1 D(E,[])

0 D([],[]) 0 D([],[]) 1 D(E,[]) 0 D([],[])

Recursive solution
D(LOEVE,LOVE)

We can use dynamic programming.

D(OEVE,OVE)
Overlapping
subproblems
D(EVE,VE)

D(VEVE,VE) D(VE,VE) D(VVE,VE)

D(EVE,E) D(E,E) D(VE,E)

D(VE,[]) D([],[]) D(EVE,E) D(E,E) D(EE,E)

D(E,[]) L(VE,[]) D([],[]) D(E,[])

D([],[]) D([],[]) D(E,[]) D([],[])

Dynamic programming (Bottom-Up)

EDist Empty L O V E String 2

Initialize:

Empty 0

O
String 1

E
Dynamic programming (Bottom-Up)

EDist Empty L O V E String 2

Initialize:

Empty 0 1 2 3 4

O
String 1

E
Dynamic programming (Bottom-Up)

EDist Empty L O V E String 2

Initialize:

Empty 0 1 2 3 4

L 1

O 2
String 1

E 3

V 4

E 5
 Dynamic programming (Bottom-Up)

String 2
EDist Empty L O V E
If String2(i) matches String1(j)
Empty 0 1 2 3 4 D(i,j) = D(i-1,j-1).
Because the cost is 0 when match.
L 1 If String2(i) DOES NOT match
String1(j)
String 1

O 2 D(i-1,j-1) D(i,j-1) D(i,j) = 1+ min{D(i-1,j-1),D(i,j-

1),D(i-1,j)}

E 3 D(i-1,j) D(i,j) D(i-1,j-1) --> Modify

D(i,j-1) --> Delete
V 4 D(i-1,j) → Insert

When NOT match, we need to select

E 5 the best from the three choices.
Dynamic programming (Bottom-Up)

EDist Empty L O V E String 1 = “L”

Initialize:

Empty 0 1 2 3 4 String 2 = “L”

L 1 0 Cost = 0

O 2

E 3

V 4

E 5
Dynamic programming (Bottom-Up)

EDist Empty L O V E String 1 = “L”

Initialize:

Empty 0 1 2 3 4 String 2 = “LO”

L 1 0 1 Cost = 1

O 2 D(i,j) = D(i-1,j) +1
Insert “O”
E 3

V 4

E 5
Dynamic programming (Bottom-Up)

EDist Empty L O V E String 1 = “L”

Initialize:

Empty 0 1 2 3 4 String 2 = “LOV”

L 1 0 1 2 Cost = 2

O 2 D(i,j) = D(i-1,j) +1
Insert “O” and “V”
E 3

V 4

E 5
Dynamic programming (Bottom-Up)

EDist Empty L O V E String 1 = “L”

Initialize:

Empty 0 1 2 3 4 String 2 = “LOVE”

L 1 0 1 2 3 Cost = 3

O 2 D(i,j) = D(i-1,j) +1
Insert “O”, “V”, and
E 3 “E”.

V 4

E 5
Dynamic programming (Bottom-Up)

EDist Empty L O V E String 1 = “LO”

Initialize:

Empty 0 1 2 3 4 String 2 = “L”

L 1 0 1 2 3 Cost = 1

O 2 1 D(i,j) = D(i,j-1) +1

E 3 Delete “O”

V 4

E 5
Dynamic programming (Bottom-Up)

EDist Empty L O V E String 1 = “LO”

Initialize:

Empty 0 1 2 3 4 String 2 = “LO”

L 1 0 1 2 3 Cost = 0

O 2 1 0 D(i,j) = D(i-1,j-1)

E 3 Match

V 4

E 5
Dynamic programming (Bottom-Up)

EDist Empty L O V E String 1 = “LO”

Initialize:

Empty 0 1 2 3 4 String 2 = “LOV”

L 1 0 1 2 3 Cost = 1

O 2 1 0 1 D(i,j) = D(i-1,j) +1

E 3 Insert V.

V 4

E 5
Dynamic programming (Bottom-Up)

EDist Empty L O V E String 1 = “LO”

Initialize:

Empty 0 1 2 3 4 String 2 = “LOVE”

L 1 0 1 2 3 Cost = 2

O 2 1 0 1 2 D(i,j) = D(i-1,j) +1

E 3 Insert V and E.

V 4

E 5
Dynamic programming (Bottom-Up)

EDist Empty L O V E String 1 = “LOE”

Initialize:

Empty 0 1 2 3 4 String 2 = “L”

L 1 0 1 2 3 Cost = 2

O 2 1 0 1 2 D(i,j) = D(i,j-1) +1

E 3 2 Delete O and E.

V 4

E 5
Dynamic programming (Bottom-Up)

EDist Empty L O V E String 1 = “LOE”

Initialize:

Empty 0 1 2 3 4 String 2 = “LO”

L 1 0 1 2 3 Cost = 1

O 2 1 0 1 2 D(i,j) = D(i,j-1) +1

E 3 2 1 Delete E.

V 4

E 5
Dynamic programming (Bottom-Up)

EDist Empty L O V E String 1 = “LOE”

Initialize:

Empty 0 1 2 3 4 String 2 = “LOV”

L 1 0 1 2 3 Cost = 1

O 2 1 0 1 2 D(i,j) = D(i-1,j-1) +1

E 3 2 1 1 Modify E to V.

V 4

E 5
Dynamic programming (Bottom-Up)

EDist Empty L O V E String 1 = “LOE”

Initialize:

Empty 0 1 2 3 4 String 2 = “LOVE”

L 1 0 1 2 3 Cost = 1

O 2 1 0 1 2 D(i,j) = D(i-1,j-1)

E 3 2 1 1 1

V 4

E 5
Dynamic programming (Bottom-Up)

EDist Empty L O V E String 1 = “LOEV”

Initialize:

Empty 0 1 2 3 4 String 2 = “L”

L 1 0 1 2 3 Cost = 3

O 2 1 0 1 2 D(i,j) = D(i,j-1) +1

E 3 2 1 1 1 Delete O E V.

V 4 3

E 5
Dynamic programming (Bottom-Up)

EDist Empty L O V E String 1 = “LOEV”

Initialize:

Empty 0 1 2 3 4 String 2 = “LO”

L 1 0 1 2 3 Cost = 2

O 2 1 0 1 2 D(i,j) = D(i,j-1) +1

E 3 2 1 1 1 Delete E V.

V 4 3 2

E 5
Dynamic programming (Bottom-Up)

EDist Empty L O V E String 1 = “LOEV”

Initialize:

Empty 0 1 2 3 4 String 2 = “LOV”

L 1 0 1 2 3 Cost = 1

O 2 1 0 1 2 D(i,j) = D(i,j-1) +1

E 3 2 1 1 1 Delete E.

V 4 3 2 1

E 5
Dynamic programming (Bottom-Up)

EDist Empty L O V E String 1 = “LOEV”

Initialize:

Empty 0 1 2 3 4 String 2 = “LOVE”

L 1 0 1 2 3 Cost = 2

Two ways
O 2 1 0 1 2
D(i,j) = D(i,j-1) +1

E 3 2 1 1 1 1. Delete E before V
and insert E after V.

V 4 3 2 1 2 2. Modify E to V
and Modify V to E.
E 5
Dynamic programming (Bottom-Up)

EDist Empty L O V E String 1 = “LOEVE”

Initialize:

Empty 0 1 2 3 4 String 2 = “L”

L 1 0 1 2 3 Cost = 4

Delete “OEVE”
O 2 1 0 1 2

E 3 2 1 1 1

V 4 3 2 1 2

E 5 4
Dynamic programming (Bottom-Up)

EDist Empty L O V E String 1 = “LOEVE”

Initialize:

Empty 0 1 2 3 4 String 2 = “LO”

L 1 0 1 2 3 Cost = 3

Delete “EVE”
O 2 1 0 1 2

E 3 2 1 1 1

V 4 3 2 1 2

E 5 4 3
Dynamic programming (Bottom-Up)

EDist Empty L O V E String 1 = “LOEVE”

Initialize:

Empty 0 1 2 3 4 String 2 = “LOV”

L 1 0 1 2 3 Cost = 2

Delete “E” and then

O 2 1 0 1 2
delete the other “E”

E 3 2 1 1 1

V 4 3 2 1 2

E 5 4 3 2
Dynamic programming (Bottom-Up)

EDist Empty L O V E String 1 = “LOEVE”

Initialize:

Empty 0 1 2 3 4 String 2 = “LOVE”

L 1 0 1 2 3 Cost = 1

Delete “E”
O 2 1 0 1 2

E 3 2 1 1 1

V 4 3 2 1 2

E 5 4 3 2 1
Dynamic programming (Bottom-Up)

EDist Empty L O V E
Initialize:

Empty 0 1 2 3 4

L 1 0 1 2 3

O 2 1 0 1 2

E 3 2 1 1 1

V 4 3 2 1 2

E 5 4 3 2 1
Edit distance algorithm (Bottom-Up store dynamic programming)

Procedure EDist(S1, m, S2, n) // S1 and S2 are strings, m and n are sizes.

Define a Table[m+1,n+1]
For i for 0 to m // recursive function
for j from 0 to n
if 𝑖 == 0 // if S1 is empty, insert letters in S2 to S1.
m+1
Table[ i ][ j ] = j;
else if 𝑗 == 0 // if S2 is empty, delete all letters in S1
Table[ i ][ j ] = i; n+1
else if S1 𝑖 == 𝑆2 (𝑗)
Table[i][j] = Table[i-1][j-1]; O((m+1)*(n+1))
else. // Insert, Delete , Modify
Table[i][j] = 1+ min(Table[i][j-1], Table[i-1][j], Table[i-1][j-1]);
Return Table[ m ][ n ] Overall O(m*n)
Recursive solution
D(LOEVE,LOVE)

We can use dynamic programming.

D(OEVE,OVE)
Overlapping
problems
D(EVE,VE)

D(VEVE,VE) D(VE,VE) D(VVE,VE)

D(EVE,E) D(E,E) D(VE,E)

D(VE,[]) D([],[]) D(EVE,E) D(E,E) D(EE,E)

D(E,[]) L(VE,[]) D([],[]) D(E,[])

D([],[]) D([],[]) D(E,[]) D([],[])

Practice
What is the edit distance between EXPONENTIAL and POLYNOMIAL?
P O L Y N O M I A L
0 1 2 3 4 5 6 7 8 9 10
15 mins E 1
X 2
P 3
O 4
N 5
E 6
N 7
T 8
I 9
A 10
L 11
Practice
What is the edit distance between EXPONENTIAL and POLYNOMIAL?
P O L Y N O M I A L
0 1 2 3 4 5 6 7 8 9 10
E 1 1 2 3 4 5 6 7 8 9 10
X 2 2 2 3 4 5 6 7 8 9 10
P 3 2 3 3 4 5 6 7 8 9 10
O 4 3 2 3 4 5 5 6 7 8 9
N 5 4 3 3 4 4 5 6 7 8 9
E 6 5 4 4 4 5 5 6 7 8 9
N 7 6 5 5 5 4 5 6 7 8 9
T 8 7 6 6 6 5 5 6 7 8 9
I 9 8 7 7 7 6 6 6 6 7 8
A 10 9 8 8 8 7 7 7 7 6 7
L 11 10 9 8 9 8 8 8 8 7 6
Dynamic programming (Top-Down)
D(LOEVE,LOVE) Table size m*n rows EiditDist #
D(LOEVE,LOVE) ?
Dynamic programming (Top-Down)
D(LOEVE,LOVE) Table size m*n rows EiditDist #
D(LOEVE,LOVE) ?
D(OEVE,OVE) D(OEVE,OVE) ?
Dynamic programming (Top-Down)
D(LOEVE,LOVE) Table size m*n rows EiditDist #
D(LOEVE,LOVE) ?
D(OEVE,OVE) D(OEVE,OVE) ?
D(EVE,VE) ?
D(EVE,VE)
Dynamic programming (Top-Down)
D(LOEVE,LOVE) Table size m*n rows EiditDist #
D(LOEVE,LOVE) ?
D(OEVE,OVE) D(OEVE,OVE) ?
D(EVE,VE) ?
D(EVE,VE)
D(VEVE,VE) ?

D(VEVE,VE)
Dynamic programming (Top-Down)
D(LOEVE,LOVE) Table size m*n rows EiditDist #
D(LOEVE,LOVE) ?
D(OEVE,OVE) D(OEVE,OVE) ?
D(EVE,VE) ?
D(EVE,VE)
D(VEVE,VE) ?
D(EVE,E) ?
D(VEVE,VE)

D(EVE,E)
Dynamic programming (Top-Down)
D(LOEVE,LOVE) Table size m*n rows EiditDist #
D(LOEVE,LOVE) ?
D(OEVE,OVE) D(OEVE,OVE) ?
D(EVE,VE) ?
D(EVE,VE)
D(VEVE,VE) ?
D(EVE,E) ?
D(VEVE,VE)
D(VE,[]) ?

D(EVE,E)

D(VE,[])
Dynamic programming (Top-Down)
D(LOEVE,LOVE) Table size m*n rows EiditDist #
D(LOEVE,LOVE) ?
D(OEVE,OVE) D(OEVE,OVE) ?
D(EVE,VE) ?
D(EVE,VE)
D(VEVE,VE) ?
D(EVE,E) ?
D(VEVE,VE)
D(VE,[]) ?

D(EVE,E)
D(E,[]) 1

D(VE,[])

D(E,[])
Dynamic programming (Top-Down)
D(LOEVE,LOVE) Table size m*n rows EiditDist #
D(LOEVE,LOVE) ?
D(OEVE,OVE) D(OEVE,OVE) ?
D(EVE,VE) ?
D(EVE,VE)
D(VEVE,VE) ?
D(EVE,E) ?
D(VEVE,VE)
D(VE,[]) 2

D(EVE,E)

D(VE,[])
Dynamic programming (Top-Down)
D(LOEVE,LOVE) Table size m*n rows EiditDist #
D(LOEVE,LOVE) ?
D(OEVE,OVE) D(OEVE,OVE) ?
D(EVE,VE) ?
D(EVE,VE)
D(VEVE,VE) ?
D(EVE,E) 2
D(VEVE,VE)
D(VE,[]) 2

D(EVE,E)

D(VE,[])
Dynamic programming (Top-Down)
D(LOEVE,LOVE) Table size m*n rows EiditDist #
D(LOEVE,LOVE) ?
D(OEVE,OVE) D(OEVE,OVE) ?
D(EVE,VE) ?
D(EVE,VE)
D(VEVE,VE) 2
D(EVE,E) 2
D(VEVE,VE)
D(VE,[]) 2

D(EVE,E)

D(VE,[])
Dynamic programming (Top-Down)
D(LOEVE,LOVE) Table size m*n rows EiditDist #
D(LOEVE,LOVE) ?
D(OEVE,OVE) D(OEVE,OVE) ?
D(EVE,VE) ?
D(EVE,VE)
D(VEVE,VE) 2
D(EVE,E) 2
D(VEVE,VE) D(VE,VE)
D(VE,[]) 2

D(EVE,E)
D(VE,VE) ?

D(VE,[])
Dynamic programming (Top-Down)
D(LOEVE,LOVE) Table size m*n rows EiditDist #
D(LOEVE,LOVE) ?
D(OEVE,OVE) D(OEVE,OVE) ?
D(EVE,VE) ?
D(EVE,VE)
D(VEVE,VE) 2
D(EVE,E) 2
D(VEVE,VE) D(VE,VE)
D(VE,[]) 2

D(EVE,E) D(E,E)
D(VE,VE) ?
D(E,E) 0
D(VE,[])
Dynamic programming (Top-Down)
D(LOEVE,LOVE) Table size m*n rows EiditDist #
D(LOEVE,LOVE) ?
D(OEVE,OVE) D(OEVE,OVE) ?
D(EVE,VE) ?
D(EVE,VE)
D(VEVE,VE) 2
D(EVE,E) 2
D(VEVE,VE) D(VE,VE)
D(VE,[]) 2

D(EVE,E) D(E,E)
D(VE,VE) 0
D(E,E) 0
D(VE,[])
Dynamic programming (Top-Down)
D(LOEVE,LOVE) Table size m*n rows EiditDist #
D(LOEVE,LOVE) ?
D(OEVE,OVE) D(OEVE,OVE) ?
D(EVE,VE) ?
D(EVE,VE)
D(VEVE,VE) 2
D(EVE,E) 2
D(VEVE,VE) D(VE,VE) D(VVE,VE)
D(VE,[]) 2

D(EVE,E) D(E,E)
D(VE,VE) 0
D(E,E) 0
D(VE,[]) D(VVE,VE) ?
Dynamic programming (Top-Down)
D(LOEVE,LOVE) Table size m*n rows EiditDist #
D(LOEVE,LOVE) ?
D(OEVE,OVE) D(OEVE,OVE) ?
D(EVE,VE) ?
D(EVE,VE)
D(VEVE,VE) 2
D(EVE,E) 2
D(VEVE,VE) D(VE,VE) D(VVE,VE)
D(VE,[]) 2

D(EVE,E) D(E,E) D(VE,E)

D(VE,VE) 0
D(E,E) 0
D(VE,[]) D(VVE,VE) ?
D(VE,E) ?
Dynamic programming (Top-Down)
D(LOEVE,LOVE) Table size m*n rows EiditDist #
D(LOEVE,LOVE) ?
D(OEVE,OVE) D(OEVE,OVE) ?
D(EVE,VE) ?
D(EVE,VE)
D(VEVE,VE) 2
D(EVE,E) 2
D(VEVE,VE) D(VE,VE) D(VVE,VE)
D(VE,[]) 2

D(EVE,E) D(E,E) D(VE,E)

D(VE,VE) 0
D(E,E) 0
D(VE,[]) D(EVE,E)
D(VVE,VE) ?
D(VE,E) ?
Dynamic programming (Top-Down)
D(LOEVE,LOVE) Table size m*n rows EiditDist #
D(LOEVE,LOVE) ?
D(OEVE,OVE) D(OEVE,OVE) ?
D(EVE,VE) ?
D(EVE,VE)
D(VEVE,VE) 2
D(EVE,E) 2
D(VEVE,VE) D(VE,VE) D(VVE,VE)
D(VE,[]) 2

D(EVE,E) D(E,E) D(VE,E)

D(VE,VE) 0
D(E,E) 0
D(VE,[]) D(EVE,E) D(E,E)
D(VVE,VE) ?
D(VE,E) ?
Dynamic programming (Top-Down)
D(LOEVE,LOVE) Table size m*n rows EiditDist #
D(LOEVE,LOVE) ?
D(OEVE,OVE) D(OEVE,OVE) ?
D(EVE,VE) ?
D(EVE,VE)
D(VEVE,VE) 2
D(EVE,E) 2
D(VEVE,VE) D(VE,VE) D(VVE,VE)
D(VE,[]) 2

D(EVE,E) D(E,E) D(VE,E)

D(VE,VE) 0
D(E,E) 0
D(VE,[]) D(EVE,E) D(E,E) D(EE,E)
D(VVE,VE) ?
D(VE,E) ?
D(EE,E) ?
Dynamic programming (Top-Down)
D(LOEVE,LOVE) Table size m*n rows EiditDist #
D(LOEVE,LOVE) ?
D(OEVE,OVE) D(OEVE,OVE) ?
D(EVE,VE) ?
D(EVE,VE)
D(VEVE,VE) 2
D(EVE,E) 2
D(VEVE,VE) D(VE,VE) D(VVE,VE)
D(VE,[]) 2

D(EVE,E) D(E,E) D(VE,E)

D(VE,VE) 0
D(E,E) 0
D(VE,[]) D(EVE,E) D(E,E) D(EE,E)
D(VVE,VE) ?
D(VE,E) ?
D(E,[]) D(EE,E) ?
D(E,[]) 1
Dynamic programming (Top-Down)
D(LOEVE,LOVE) Table size m*n rows EiditDist #
D(LOEVE,LOVE) ?
D(OEVE,OVE) D(OEVE,OVE) ?
D(EVE,VE) ?
D(EVE,VE)
D(VEVE,VE) 2
D(EVE,E) 2
D(VEVE,VE) D(VE,VE) D(VVE,VE)
D(VE,[]) 2

D(EVE,E) D(E,E) D(VE,E)

D(VE,VE) 0
D(E,E) 0
D(VE,[]) D(EVE,E) D(E,E) D(EE,E)
D(VVE,VE) ?
D(VE,E) ?
D(E,[]) D(EE,E) 1
D(E,[]) 1
Dynamic programming (Top-Down)
D(LOEVE,LOVE) Table size m*n rows EiditDist #
D(LOEVE,LOVE) ?
D(OEVE,OVE) D(OEVE,OVE) ?
D(EVE,VE) ?
D(EVE,VE)
D(VEVE,VE) 2
D(EVE,E) 2
D(VEVE,VE) D(VE,VE) D(VVE,VE)
D(VE,[]) 2

D(EVE,E) D(E,E) D(VE,E)

D(VE,VE) 0
D(E,E) 0
D(VE,[]) D(EVE,E) D(E,E) D(EE,E)
D(VVE,VE) ?
D(VE,E) 1
D(E,[]) D(EE,E) 1
D(E,[]) 1
Dynamic programming (Top-Down)
D(LOEVE,LOVE) Table size m*n rows EiditDist #
D(LOEVE,LOVE) ?
D(OEVE,OVE) D(OEVE,OVE) ?
D(EVE,VE) ?
D(EVE,VE)
D(VEVE,VE) 2
D(EVE,E) 2
D(VEVE,VE) D(VE,VE) D(VVE,VE)
D(VE,[]) 2

D(EVE,E) D(E,E) D(VE,E)

D(VE,VE) 0
D(E,E) 0
D(VE,[]) D(EVE,E) D(E,E) D(EE,E)
D(VVE,VE) 1
D(VE,E) 1
D(E,[]) D(EE,E) 1
D(E,[]) 1
Dynamic programming (Top-Down)
D(LOEVE,LOVE) Table size m*n rows EiditDist #
D(LOEVE,LOVE) ?
D(OEVE,OVE) D(OEVE,OVE) ?
D(EVE,VE) 1
D(EVE,VE)
D(VEVE,VE) 2
D(EVE,E) 2
D(VEVE,VE) D(VE,VE) D(VVE,VE)
D(VE,[]) 2

D(EVE,E) D(E,E) D(VE,E)

D(VE,VE) 0
D(E,E) 0
D(VE,[]) D(EVE,E) D(E,E) D(EE,E)
D(VVE,VE) 1
D(VE,E) 1
D(E,[]) D(EE,E) 1
D(E,[]) 1
Dynamic programming (Top-Down)
D(LOEVE,LOVE) Table size m*n rows EiditDist #
D(LOEVE,LOVE) 1
D(OEVE,OVE) D(OEVE,OVE) 1
D(EVE,VE) 1
D(EVE,VE)
D(VEVE,VE) 2
D(EVE,E) 2
D(VEVE,VE) D(VE,VE) D(VVE,VE)
D(VE,[]) 2

D(EVE,E) D(E,E) D(VE,E)

D(VE,VE) 0
D(E,E) 0
D(VE,[]) D(EVE,E) D(E,E) D(EE,E)
D(VVE,VE) 1
D(VE,E) 1
D(E,[]) D(EE,E) 1
D(E,[]) 1
Recursive solution
D(LOEVE,LOVE)

We can use dynamic programming.

D(OEVE,OVE)
Overlapping
subproblems
D(EVE,VE)

D(VEVE,VE) D(VE,VE) D(VVE,VE)

D(EVE,E) D(E,E) D(VE,E)

D(VE,[]) D([],[]) D(EVE,E) D(E,E) D(EE,E)

D(E,[]) L(VE,[]) D([],[]) D(E,[])

D([],[]) D([],[]) D(E,[]) D([],[])