Design & Analysis of

Algorithms EID305

Module-2: Greedy Method

Agenda of Today’s Session
1. The General Method
2. Knapsack problem
3. Job sequencing with deadlines
1
4. Minimum cost spanning trees
5. Optimal storage on tapes
6. Single source shortest paths

Greedy Method
The greedy method is perhaps the most straight forward design technique
 The problems have n inputs and require to obtain a subset that satisfies
some constraints.
 A solution that satisfy the constraints of a problem is known as feasible
solution
 A feasible solution that either maximize or minimize a given objective
function is known as optimal solution.

General idea:

2
 Given a problem with n inputs, we are required to obtain a
subset that maximizes or minimizes a given objective function
subject to some constraints.

Feasible solution — any subset that satisfies some

constraints

Optimal solution — a feasible solution that maximizes

or minimizes the objective
function
procedure Greedy (A, n)
begin solution  Ø;
for i  1 to n do
3
 x  Select (A); // based on the objective
// function if Feasible
(solution, x),
then solution  Union (solution, x);
end;
Select: A greedy procedure, based on a given objective function,
which selects input from A, removes it and assigns its value to x.

Feasible: A boolean function to decide if x can be included into

solution vector (without violating any given constraint).

4
General Algorithm

5
6
The function ‘Select’ selects an input
from a[ ] and removes it.
The selected input's value is assigned
to x.
Feasible is a Boolean-valued function
that determines whether x can be
included into the solution vector.

7
The function Union combines x with
the solution and updates the objective
function.
KNAPSACKPROBLEM

 There are given n objects and a knapsack or bag. Object i has

a weight wi and the Knapsack has a capacity m.
 If a fraction xi, 0 <xi < 1,of object i is placed into the
knapsack, then a profit of pixi is earned.
 The objective is to obtain a filling of the knapsack that
maximizes the total profit earned.

8
  Since the knapsack capacity is m, we require the total weight
of all chosen objects to be at most m.

An optimal solution is a feasible solution

for which is maximized.
Algorithm Greedy Knapsack(m,n)
9
10
 Example
Let n = 3, M = 20,
(p1 p2 p3) = (25, 24, 15)
(w1 w2 w3) = (18, 15, 10)
Feasible Solutions

(x1, x2, x3) ∑n wixi i=1 ∑n pixi i=1

1 (1/2, 1/3,1/4) 16.5 24.25

2 (1, 2/15,0) 20 28.2
3 (0, 1,1/2) 20 31.5
Strategy 1 : maximise objective function

11
Put the object with the greatest profit in the knapsack.

Then use a fraction of the last object to fill the knapsack to capacity.
n
𝑛

෍ pixi = 25 × 1 + 24 × 2/15 + 0 = 28.5

𝑖=1

෍ wixi = 18 × 1 + 15 × 2/15 + 0 = 20
𝑖=1

Strategy does not yield an optimal solution.

Capacity was quickly exhausted which con- strained the profit attained.
Strategy 2 : maximise capacity
12
Choose objects according to least weight.
The idea is that we will get more objects into the knapsack and potentially
more profit !
𝑛

෍ 0 + 24 × 2/3 + 15 × 1 = 31
𝑖=1

෍ 0+15 2=3+10 1 = 20
𝑖=1

 Solution is still not optimal

 Rate of increase of profit was not high enough
Strategy 3: balancing profit and capacity
Find the object to include by maximum profit per unit of capacity, i.e.,
compute pi/wi Then choose objects starting with the largest and working
to smallest ratio.
𝑛
13
 ෍ 0+24 1+15 1=2 = 31.5
𝑖=1
𝑛

෍ 0+15 1+10 1=2 = 20

𝑖=1

Solution is optimal !
Achieves a balance between rate at which profit increases with the rate at
which the capacity is used.

JOBSEQUENCINGWITH DEADLINES
 a set of n job are given.
 Associated with job i is an integer deadline di >0 and a profit pi >0.
 For any job i the profit pi is earned iff the job is completed by its deadline.
 To complete a job, one has to process the job on a machine for one unit of
time.

14
 A feasible solution for this problem is a subset J of jobs such that each job
in this subset can be completed by its deadline.

15
16
17
High level description of Job sequencing
Algorithm.

18
19
20
Let n=5, {p1,p2,p3,p4,p5} = {60,100,20,40,20} and (d1,d2,d3,d4,d5} =
{2,1,3,2,1}. Find the Optimal solution.

Jobs J2 J1 J4 J3 J5
Profits 100 60 40 20 20
Deadlines 1 2 2 3 1
0 𝑗2 1 𝑗1 2 𝑗4 3
15 20 5

J Assigned Slots Jobs Considered Action Profit

Φ none 2 assign to[0,1] 0
{2} [0,1] 1 assign to[1,2] 100
{2, 1} [0,1][1,2] 4 assign fit, reject
100+60
{2, 1} [0,1][1,2] 3 assign to[2,3]
100+60
{2, 1,3} [0,1][1,2][2,3] 5 Reject 100+60+20
The optimal solution is J={2,1,3} with a profit of 180.

21
 TREE

A
Connected acyclic graph
B C
Tree with n nodes contains
D E F exactly n-1 edges.

GRAPH
Graph with n nodes contains less than or equal
to n(n-1)/2 edges.
22
 SPANNING TREE...
Suppose you have a connected undirected graph
Connected: every node is reachable from every
other node
Undirected: edges do not have an associated direction
...then a spanning tree of the graph is a connected
subgraph in which there are no cycles

A connected, Four of the spanning trees of the graph

undirected
23
graph

24
EXAMPLE..

25
 MINIMUM SPANNING TREE

 LetG = (N, A) be a connected, undirected graph where N is

the set of nodes and A is the set of edges. Each edge has a
given nonnegative length. The problem is to find a subset T
of the edges of G such that all the nodes remain connected
when only the edges in T are used, and the sum
of the lengths of the edges in T is as small as possible
possible. Since G is connected, at least one solution
must exist.

Finding Spanning Trees

Applications
• They are helpful in routing applications.
• Communication Networks.
26
• They can be used to obtain circuit equations for an electric
network.

• There are two basic algorithms for finding minimum-cost

spanning trees, and both are greedy algorithms

• Kruskal’s algorithm:
Created in 1957 by Joseph Kruskal

• Prim’s algorithm
Created by Robert C. Prim

27
Kruskal’s Algorithm
List the edges in
order of size:

E
28
 B 5
C
ED 2
3 AB 3
6
4 AE 4
8
CD 4
8 BC 5
A EF 5
D
7 F CF 6
AF 7
5 BF 8
4
2 CF 8

E
29
 Kruskal’s Algorithm
B 5
C Select the shortest

E
30
 edge in the network
3
4
8 6 ED

8
A D
7 F

5
4
2

E
31
 Kruskal’s Algorithm
B 5
C
Select the next
3 shortest
4 edge which does not
6
8 create a cycle
8
ED 2
A
7 F D AB 3

5
4
2

E
32
Kruskal’s Algorithm
B 5
C
Select the next
3 shortest
6
4 edge which does not
8
create a cycle
8
A ED 2
D
7 F AB 3
CD 4 (or AE 4)
5
4
2

E
33
Kruskal’s Algorithm
B 5
C Select the next
shortest edge which does not
create a cycle

34
 E

A
3
4
8 6

8
A
F D C
7

5
4
2

E
35
 Kruskal’s Algorithm
B 5
C
Select the next shortest
3
edge which does not
4 createa cycle
8 6
ED 2
8
AB 3
A D CD
7 F 4
AE 4
5 BC 5 – forms acycle
4 EF 5
2

E
36
Kruskal’s Algorithm
B 5 All vertices have been

37
 C
connected.
3
4 The solution is
8 6
ED 2
8
AB 3
A
7 F D CD 4
AE 4
5 EF 5
4
2
Total weight of tree:
18
E

38
39
Example

40
41
Prim’s Algorithm
B 5

E
42
 3
4
8 6

8
A
F D that vertex
7

5
4
2

C Select any vertex A

E
43
 Select the shortest
edge connected to

AB 3

Prim’s Algorithm
B 5
C
Select the shortest
3 edge connected to
6
4 any vertex already
8
connected.
8
A D
7 E F
44

5
 AE 4

Prim’s Algorithm
B 5
C

3 Select the shortest

4
8 6 edge connected to
any vertex already
8 connected.
A D
7 F
ED 2
5
4
E 2
45
 Prim’s Algorithm
B 5
C
Select the shortest
3 edge connected to
6
4 any vertex already
8
connected.
8
A DC 4
F D
7

5
4
2

E
46
Prim’s Algorithm

E
47
 B 5
C

3
4
8 6

8
A D EF 5
7 F

5
4
2

Select the shortest

edge connected to
any vertex already
connected.
E
48
 Prim’s Algorithm
B 5 All vertices have been

E
49
 C
connected.
3
4 The solution is
8 6
AB 3
8
AE 4
A
7 F D ED 2
DC 4
5 EF 5
4
2
Total weight of tree:
18

E
50
51
Example

52
53
 Example

8 7
b c d 9
4
2
a 11 14 e
i 4
7 6
8 10
h g f
1 2

Solution

54
 8 7
b c d 9
4
2
a 11 14 e
i 4
7 6
8 10
h g f
1 2

Minimum Connector Algorithms

55
 Prim’s algorithm 1. Kruskal’s algorithm

1. Select any vertex Select the shortest edge in

2. Select the shortest edge a network
2.
connected to that vertex
Select the next shortest
3. Select the shortest edge edge which does not create
connected to any vertex a cycle
3.
already connected
4. Repeat step 3 until all Repeat step 2 until all
vertices have been vertices have been
connected connected

56
OPTIMAL STORAGE ON TAPES
 There are n programs that are to be stored on a computer tape of length I
 Associated with each program i is a length li, 1<= i < =n.
 All programs can be stored on the tape if and only if the sum of the lengths
of the programs is at most I.
 In tape, each program is retrieved sequentially.
 Let Tj be the time required to retrieve program.
 If all programs are retrieved equally often, then the expected or mea
retrieval time (MRT) is given by 𝑛1
𝑛
𝑇𝑗which should be minimum.
σ𝑗−1

57
 Optimal Storage on Tapes

The problem:

Given n programs to be stored on tape, the lengths

of these n programs are l1, l2 , . . . , ln respectively.
Suppose the programs are stored in the order of i1, i2
, . . . , in
Let tj be the time to retrieve program ij.
Assume that the tape is initially positioned at the beginning.

tj is proportional to the sum of all lengths of programs stored

in front of the program ij.
58
 1
The goal is to minimize MRT (Mean Retrieval Time), n jn=1t
j
n j

i.e. want to minimize  l

ik
j=1 k=1

Ex: n = 3, (l1,l2,l3) = (5,10,3)

There are n! = 6 possible orderings for storing them.
order total retrieval time MRT

1 123 5+(5+10)+(5+10+3)=38 38/3

2 132 5+(5+3)+(5+3+10)=31 31/3
 3 213 10+(10+5)+(10+5+3)=43 43/3 4 231
10+(10+3)+(10+3+5)=41 41/3
5 3+(3+5)+(3+5+10)=29 29/3 Smallest
6 3+(3+10)+(3+10+5)=34 34/3

Note: The problem can be solved using greedy strategy,

just always let the shortest program goes first.
( Can simply get the right order by using any sorting 50

algorithm) Analysis:
Try all combination: O( n! )
Shortest-length-First Greedy method: O (nlogn)
Shortest-length-First Greedy method:
Sort the programs s.t. l1 l2 . . . ln and
call this ordering L.
60
 Next is to show that the ordering L is the best
Proof by contradiction:
Suppose Greedy ordering L is not optimal, then there
exists some other permutation I that is optimal.
I = (i1, i2, … in)  a < b, s.t. li  li (otherwise I = L)
a b

Interchange ia and ib in and call the new list I :

I x

61
 … ia ia+1 ia+2 … ib …
I SWAP
… ib ia+1 ia+2 … ia …

x
In I, Program ia+1 will take less (lia- lib) time than in I to be retrieved
In fact, each program ia+1 , …, ib-1 will take less (lia- lib) time
For ib, the retrieval time decreases x + lia
For ia, the retrieval time increases x + lib

totalRT(I)−totalRT(I) = (b−a −1)(li −li )+(x +li )−(x+li )

a b a b

62
 = (b−a)(li −li )  0 (→)
a b

Contradiction!!
Therefore, greedy ordering L is optimal

63
 When they are stored on the tape in this order the MRT is minimized.
64
The time taken to build the required permutation is O(n!).

65
 Assigning Programs to tapes
Find an optimal placement for 13 programs on three tapes To,T1,and T2,where
the program are of lengths 12,5,8,32,7,5,18,26,4,3,11,10 and 6.
Step1: Arrange all the files in ascending order according to their length.
If ascending order is:
3,4,5,5,6,7,8,10,11,12,18,26,32.
By using the algorithm store the following programs are assigned to the tapes.

66
Tape-0 : 3,5,8,12,32 = 3+(3+5)+(3+5+8)+(3+5+8+12)+(3+5+8+12+32)
Tape-1: 4,6,10,18 = 4+(4+6)+(4+6+10)+(4+6+10+13) Tape-
2: 5,7,11,26 = 5+(5+7)+(5+7+11)+(5+7+11+26)

67
SINGLE SOURCE SHORTEST PATHS

Graph is used to represent the distance between two cities.

In single source shortest path problem the shortest distance from a single
vertex called source is obtained.
Let G(V,E) be a graph, the in single source shortest path, the shortest paths
from vertex V0 to all remaining vertex is determined.
The vertex V0 is then called as source and the last vertex is called
destination

Single-Source Shortest Paths in DAGs

In a dag:
68
• Every path is a subsequence of the topologically sorted
vertex order
• If we do topological sort and process vertices in that order •
We will process each path in forward order
➢ Never relax edges out of a vertex until have processed all
edges into the vertex
• Thus, just 1 pass is sufficient

69
Example

70
Example

71
Example

72
Example

73
Example

74
Example

Single-Source Shortest Paths in DAGs

75
DAG-SHORTEST PATHS(G, s)
TOPOLOGICALLY-SORT the vertices of G
INIT(G, s)
for each vertex u taken in topologically sorted order do
for each v  Adj[u] do
RELAX(u, v)

76
77
Greedy Algorithm to generate Shortest paths

78
79
80
81
82
Thank you!

83