Scribd
Upload
56 views

Design & Analysis of Algorithms: DR Anwar Ghani

The document discusses insertion sort, including: 1) How insertion sort works by iterating through an array and inserting each element into the sorted portion; 2) The loop invariant that the subarray from index 1 to j-1 is sorted at each iteration; 3) Analyses of best, worst, and average cases, with worst case quadratic time complexity.

Uploaded by

Mahmood Syed
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
56 views

Design & Analysis of Algorithms: DR Anwar Ghani

The document discusses insertion sort, including: 1) How insertion sort works by iterating through an array and inserting each element into the sorted portion; 2) The loop invariant that the subarray from index 1 to j-1 is sorted at each iteration; 3) Analyses of best, worst, and average cases, with worst case quadratic time complexity.

Uploaded by

Mahmood Syed
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 21

Design & Analysis of Algorithms

Dr Anwar Ghani

Semester: Fall 2017

Department of Computer Science & Software Engineering,


International Islamic University, Islamabad.
Insertion Sort
The Sorting Problem
• Input:

– A sequence of n numbers a1, a2, . . . , an

• Output:

• A permutation (reordering) a1’, a2’, . . . , an’ of the

input sequence such that a1’ ≤ a2’ ≤ · · · ≤ an’


Insertion Sort
• like sorting a hand of playing cards
– Start with an empty left hand and the cards facing
down on the table.
– Remove one card at a time from the table, and insert
it into the correct position in the left hand
• compare it with each of the cards already in the hand, from
right to left
– The cards held in the left hand are sorted
• these cards were originally the top cards of the pile on the
table
Insertion Sort
To insert 9, we need to
make room for it by moving
first 22 and then 15.
Insertion Sort
Insertion Sort
Insertion Sort
input array

5 2 4 6 1 3

at each iteration, the array is divided in two sub-arrays:

left sub-array right sub-array

sorted unsorted
Insertion Sort
INSERTION-SORT
INSERTION-SORT(A) 1 2 3 4 5 6 7 8

for j ← 2 to n a1 a2 a3 a4 a5 a6 a7 a8
do key ← A[ j ] key
Insert A[ j ] into the sorted sequence A[1 . . j -1]

i←j-1
while i > 0 and A[i] > key
do A[i + 1] ← A[i]
i←i–1
A[i + 1] ← key
Loop Invariant for Insertion Sort
INSERTION-SORT(A)
for j ← 2 to n
do key ← A[ j ]
Insert A[ j ] into the sorted sequence A[1 . . j -1]
i←j-1
while i > 0 and A[i] > key
do A[i + 1] ← A[i]
i←i–1
A[i + 1] ← key
Invariant: at the start of the for loop the elements in A[1 . . j-1]
are sorted
Proof
• Proving loop invariants works like induction
• Initialization (base case):
– It is true prior to the first iteration of the loop
• Maintenance (inductive step):
– If it is true before an iteration of the loop, it remains true before
the next iteration
• Termination:
– When the loop terminates, the invariant gives us a useful
property that helps show that the algorithm is correct
– Stop the induction when the loop terminates
Loop Invariant for Insertion Sort
• Initialization:
– Just before the first iteration,
j = 2:
The sub array A[1 . . j-1] = A[1],
The element originally in A[1] is sorted
Loop Invariant for Insertion Sort
• Maintenance:
– the while inner loop moves A[j -1], A[j -2], A[j -
3], and so on, by one position to the right until
the proper position for key is found
– At that point, the value of key is placed into this
position.

So, the sub array A[1…..j] is sorted after jth iteration


Loop Invariant for Insertion Sort
• Termination:
– The outer for loop ends when j = n + 1 ⇒ j-1 = n
– Replace n with j-1 in the loop invariant:
• the sub array A[1 . . n] consists of the elements originally in
A[1 . . n], but in sorted order
j-1 j

• The entire array is sorted


Invariant: at the start of the for loop the elements in A[1 . . j-1]
are in sorted order
Analysis
INSERTION-SORT(A) cost times
for j ← 2 to n c1 n
do key ← A[ j ] c2 n-1
Insert A[ j ] into the sorted sequence A[1 . . j -1] 0 n-1
i←j-1 c4 n-1
n
while i > 0 and A[i] > key c5 ∑ j =2 j
t
n
do A[i + 1] ← A[i] c6 ∑ j =2
(t j − 1)
n
i←i–1 c7 ∑ j =2
(t j − 1)
A[i + 1] ← key c8 n-1

T (n) = c1n + c2 (n − 1) + c4 (n − 1) + c5 ∑ t j + c6 ∑ (t j − 1) + c7 ∑ (t j − 1) + c8 ( n − 1)
n n n

j =2 j =2 j =2
Best Case Analysis
• Pre-Sorted Array “while i > 0 and A[i] > key”
– A[i] ≤ key upon the first time the while loop test is run
(when i = j -1)

– tj = 1

• T(n) = c1n + c2(n -1) + c4(n -1) + c5(n -1) + c8(n-1)


= (c1 + c2 + c4 + c5 + c8)n + (-c2 - c4 - c5 - c8)

= An + B
T (n) = c1n + c2 (n − 1) + c4 (n − 1) + c5 ∑ t j + c6 ∑ (t j − 1) + c7 ∑ (t j − 1) + c8 (n − 1)
n n n

j =2 j =2 j =2
Worst Case Analysis
• Reverse sorted
– Always A[i] > key in while loop test
– Have to compare key with all elements to the left of the jth
position ⇒ compare with j-1 elements ⇒ tj = j

using n
n ( n + 1) n
n ( n + 1) n
n ( n − 1) we have:
∑ j= => ∑ j= −1 => ∑ ( j − 1) =
j =1 2 j=2 2 j=2 2

 n(n + 1)  n(n − 1) n(n − 1)


T (n) = c1n + c2 (n − 1) + c4 (n − 1) + c5  − 1 + c6 + c7 + c8 (n − 1)
 2  2 2
= An 2 + Bn + C a quadratic function of n
Where A = (C5 + C 6 + C 7 ) / 2
B = C1 + C 2 + C 4 + C8 + (C 5 − C 6 − C 7 ) / 2

C = − (C 2 + C 4 + C 5 + C 8 )
Average Case Analysis
INSERTION-SORT(A) cost times
for j ← 2 to n c1 n

do key ← A[ j ] c2 n-1
Insert A[ j ] into the sorted sequence A[1 . . j -1] 0 n-1

i←j-1 ≈ j/2 comparisons c4 n-1


n
while i > 0 and A[i] > key c5 ∑ j =2
j/2

do A[i + 1] ← A[i] c6 ∑
n
j =2
( j / 2 − 1)

i ← i – 1 ≈ j/2-1 exchanges c7 ∑
n
j =2
( j / 2 − 1)

A[i + 1] ← key c8 n-1


Average Case Analysis (Cont..)

• In average case scenario tj = j/2

n n n n
∑ j =2 j
t = ∑ j =2 j / 2 and ∑ t − 1 = ∑ j =2 j / 2 − 1
j =2 j

 n(n + 1)   n(n − 1)   n(n − 1) 


T (n) = c1n + c2 (n − 1) + c4 (n − 1) + c5  − 1 / 2 + c6   / 2 + c7   / 2 + c8 (n − 1)
 2   2   2 

= An 2 + Bn + C a quadratic function of n

Where A = (C5 + C 6 + C 7 ) / 4
B = C1 + C 2 + C 4 + C8 + (C 5 − C 6 − C 7 ) / 4

C = − (C 2 + C 4 + C 8 + C 5 / 2 )
Insertion Sort
• Advantages
– Good running time for “Pre-sorted” arrays “n”
• Disadvantages
– “n2” running time in worst and average case

You might also like