Chapter 2

Michelle Bodnar, Andrew Lohr

January 3, 2018

Exercise 2.1-1

31 41 59 26 41 58
31 41 59 26 41 58
31 41 59 26 41 58
26 31 41 59 41 58
26 31 41 41 59 58
26 31 41 41 58 59
Exercise 2.1-2

Algorithm 1 Non-increasing Insertion-Sort(A)

1: for j = 2 to A.length do
2: key = A[j]
3: // Insert A[j] into the sorted sequence A[1..j − 1].
4: i=j−1
5: while i > 0 and A[i] < key do
6: A[i + 1] = A[i]
7: i=i−1
8: end while
9: A[i + 1] = key
10: end for

Exercise 2.1-3

On each iteration of the loop body, the invariant upon entering is that there
is no index k < j so that A[k] = v. In order to proceed to the next iteration of
the loop, we need that for the current value of j, we do not have A[j] = v. If
the loop is exited by line 5, then we have just placed an acceptable value in i
on the previous line. If the loop is exited by exhausting all possible values of j,
then we know that there is no index that has value j, and so leaving N IL in i
is correct.

Exercise 2.1-4

1
Algorithm 2 Linear-Search(A,v)
1: i = N IL
2: for j = 1 to A.length do
3: if A[j] = v then
4: i=j
5: return i
6: end if
7: end for
8: return i

Input: two n-element arrays A and B containing the binary digits of two
numbers a and b.
Output: an (n + 1)-element array C containing the binary digits of a + b.

Algorithm 3 Adding n-bit Binary Integers

1: carry = 0
2: for i=n to 1 do
3: C[i + 1] = (A[i] + B[i] + carry) (mod 2)
4: if A[i] + B[i] + carry ≥ 2 then
5: carry = 1
6: else
7: carry = 0
8: end if
9: end for
10: C[1] = carry

Exercise 2.2-1

n3 /1000 − 100n2 − 100n + 3 ∈ Θ(n3 )

Exercise 2.2-2

Input: An n-element array A.

Output: The array A with its elements rearranged into increasing order.
The loop invariant of selection sort is as follows: At each iteration of the for
loop of lines 1 through 10, the subarray A[1..i − 1] contains the i − 1 smallest
elements of A in increasing order. After n − 1 iterations of the loop, the n − 1
smallest elements of A are in the first n − 1 positions of A in increasing order,
so the nth element is necessarily the largest element. Therefore we do not need
to run the loop a final time. The best-case and worst-case running times of
selection sort are Θ(n2 ). This is because regardless of how the elements are
initially arranged, on the ith iteration of the main for loop the algorithm always
inspects each of the remaining n − i elements to find the smallest one remaining.

2
Algorithm 4 Selection Sort
1: for i = 1 to n − 1 do
2: min = i
3: for j = i + 1 to n do
4: // Find the index of the ith smallest element
5: if A[j] < A[min] then
6: min = j
7: end if
8: end for
9: Swap A[min] and A[i]
10: end for

This yields a running time of

n−1 n−1
X X n2 − n n2 − n
n − i = n(n − 1) − i = n2 − n − = = Θ(n2 ).
i=1 i=1
2 2

Exercise 2.2-3

Suppose that every entry has a fixed probability p of being the element
looked for. A different interpretation of the question is given at the end of this
solution. Then, we will only check k elements if the previous k − 1 positions
were not the element being looked for, and the kth position is the desired value.
This means that the probabilty that the number of steps taken is k is p(1 − p)k .
The last possibility is that none of the elements in the array match what we
are looking for, in which case we look at all A.length many positions, and it
happens with probability (1 − p).
By multiplying the number of steps in each case by the probability that that
case happens, we get the expected value of:
A.length
X
E(steps) = A.length(1 − p)A.length + k(1 − p)k−1 p
k=1

The worst case is obviously if you have to check all of the possible positions,
in which case, it will take exactly A.length steps, so it is Θ(A.length).
Now, we analyze the asyptotic behavior of the average-case. Consider the
following manipulations, where first, we rewrite the single summation as a dou-
ble summation, and then use the geometric sum formula twice.

3
 A.length
X
A.length
E(steps) = A.length(1 − p) + k(1 − p)k−1 p
k=1
A.length
X A.length
X
= A.length(1 − p)A.length + (1 − p)k−1 p
s=1 k=s
A.length
X A.length
X
= A.length(1 − p)A.length + p (1 − p)k−1
s=1 k=s
A.length
X (1 − p)s−1 − (1 − p)A.length
= A.length(1 − p)A.length + p
s=1
1 − (1 − p)
A.length
X
= A.length(1 − p)A.length + (1 − p)s−1 − (1 − p)A.length
s=1
A.length
X A.length
X
= A.length(1 − p)A.length + (1 − p)s−1 − (1 − p)A.length
s=1 s=1
A.length
X
= (1 − p)s−1
s=1
1 − (1 − p)A.length
=
1 − (1 − p)
1 (1 − p)A.length
= −
p p

(1−p)A.length
Since p > 0, we have that E(steps) < p1 . Also, since A.length ≥ 1,
we have
1 1−p
E(steps) > − =1
p p
Therefore, since we have bounded above and below by a constant, we get
the somewhat unintuitive result that the expected runtime as a function of
A.length, where p is held constant is Θ(1).
A different way of interpreting this statement of this question is that there
is exactly one element in the array that is the one you are looking for, and
then each position is equally likely to be the one containing that element. In
this case, the worst case behavior is unchanged, and the expected runtime is
PA.length i A.length+1
i=1 A.length = 2 . This makes the asymptotics for the expected
case Θ(A.length).

Exercise 2.2-4

4
 For a good best-case running time, modify an algorithm to first randomly
produce output and then check whether or not it satisfies the goal of the al-
gorithm. If so, produce this output and halt. Otherwise, run the algorithm as
usual. It is unlikely that this will be successful, but in the best-case the running
time will only be as long as it takes to check a solution. For example, we could
modify selection sort to first randomly permute the elements of A, then check if
they are in sorted order. If they are, output A. Otherwise run selection sort as
usual. In the best case, this modified algorithm will have running time Θ(n).

Exercise 2.3-1

If we start with reading across the bottom of the tree and then go up level
3 41 52 26 38 57 9 49
3 41 26 52 38 57 9 49
by level.
3 26 41 52 9 38 49 57
3 9 26 38 41 49 52 57
Exercise 2.3-2

The following is a rewrite of MERGE which avoids the use of sentinels. Much
like MERGE, it begins by copying the subarrays of A to be merged into arrays
L and R. At each iteration of the while loop starting on line 13 it selects the
next smallest element from either L or R to place into A. It stops if either L
or R runs out of elements, at which point it copies the remainder of the other
subarray into the remaining spots of A.

Exercise 2.3-3

Since n is a power of two, we may write n = 2k . If k = 1, T (2) = 2 = 2 lg(2).

Suppose it is true for k, we will show it is true for k + 1.
 k+1 
2
T (2k+1 ) = 2T + 2k+1 = 2T 2k + 2k+1 = 2(2k lg(2k )) + 2k+1


2

= k2k+1 + 2k+1 = (k + 1)2k+1 = 2k+1 lg(2k+1 ) = n lg(n)

Exercise 2.3-4

Let T (n) denote the running time for insertion sort called on an array of size
n. We can express T (n) recursively as

Θ(1) if n ≤ c
T (n) =
T (n − 1) + I(n) otherwise

where I(n) denotes the amount of time it takes to insert A[n] into the sorted
array A[1..n − 1]. Since we may have to shift as many as n − 1 elements once
we find the correct place to insert A[n], we have I(n) = θ(n).

5
Algorithm 5 M erge(A, p, q, r)
1: n1 = q − p + 1
2: n2 = r − q
3: let L[1, ..n1 ] and R[1..n2 ] be new arrays
4: for i = 1 to n1 do
5: L[i] = A[p + i − 1]
6: end for
7: for j = 1 to n2 do
8: R[j] = A[q + j]
9: end for
10: i = 1
11: j = 1
12: k = p
13: while i 6= n1 + 1 and j 6= n2 + 1 do
14: if L[i] ≤ R[j] then
15: A[k] = L[i]
16: i=i+1
17: else A[k] = R[j]
18: j =j+1
19: end if
20: k =k+1
21: end while
22: if i == n1 + 1 then
23: for m = j to n2 do
24: A[k] = R[m]
25: k =k+1
26: end for
27: end if
28: if j == n2 + 1 then
29: for m = i to n1 do
30: A[k] = L[m]
31: k =k+1
32: end for
33: end if

6
Exercise 2.3-5

The following recursive algorithm gives the desired result when called with
a = 1 and b = n.

1: BinSearch(a,b,v)
2: if thena > b
3: return NIL
4: end if
5: m = b a+b
2 c
6: if thenm = v
7: return m
8: end if
9: if thenm < v
10: return BinSearch(a,m,v)
11: end if
12: return BinSearch(m+1,b,v)

Note that the initial call should be BinSearch(1, n, v). Each call results in
a constant number of operations plus a call to a problem instance where the
quantity b − a falls by at least a factor of two. So, the runtime satisfies the
recurrence T (n) = T (n/2) + c. So, T (n) ∈ Θ(lg(n))

Exercise 2.3-6

A binary search wouldn’t improve the worst-case running time. Insertion

sort has to copy each element greater than key into its neighboring spot in the
array. Doing a binary search would tell us how many how many elements need
to be copied over, but wouldn’t rid us of the copying needed to be done.

Exercise 2.3-7

We can see that the while loop gets run at most O(n) times, as the quantity
j −i starts at n−1 and decreases at each step. Also, since the body only consists
of a constant amount of work, all of lines 2-15 takes only O(n) time. So, the
runtime is dominated by the time to perform the sort, which is Θ(n lg(n)).
We will prove correctness by a mutual induction. Let mi,j be the proposition
A[i] + A[j] < S and Mi,j be the proposition A[i] + A[j] > S. Note that because
the array is sorted, mi,j ⇒ ∀k < j, mi,k , and Mi,j ⇒ ∀k > i, Mk,j .
Our program will obviously only output true in the case that there is a valid
i and j. Now, suppose that our program output false, even though there were
some i, j that was not considered for which A[i] + A[j] = S. If we have i > j,
then swap the two, and the sum will not change, so, assume i ≤ j. we now have
two cases:
Case 1 ∃k, (i, k) was considered and j < k. In this case, we take the smallest

7
 1: Use Merge Sort to sort the array A in time Θ(n lg(n))
2: i=1
3: j=n
4: while i < j do
5: if A[j] + A[j] = S then
6: return true
7: end if
8: if A[i] + A[j] < S then
9: i=i+1
10: end if
11: if A[i] + A[j] > S then
12: j =j−1
13: end if
14: end while
15: return false

such k. The fact that this is nonzero meant that immediately after considering
it, we considered (i+1,k) which means mi,k this means mi,j
Case 2 ∃k, (k, j) was considered and k < i. In this case, we take the largest
such k. The fact that this is nonzero meant that immediately after considering
it, we considered (k,j-1) which means Mk,j this means Mi,j
Note that one of these two cases must be true since the set of considered
points separates {(m, m0 ) : m ≤ m0 < n} into at most two regions. If you are
in the region that contains (1, 1)(if nonempty) then you are in Case 1. If you
are in the region that contains (n, n) (if non-empty) then you are in case 2.

Problem 2-1

a. The time for insertion sort to sort a single list of length k is Θ(k 2 ), so, n/k
of them will take time Θ( nk k 2 ) = Θ(nk).
b. Suppose we have coarseness k. This meas we can just start using the usual
merging procedure, except starting it at the level in which each array has size
at most k. This means that the depth of the merge tree is lg(n) − lg(k) =
lg(n/k). Each level of merging is still time cn, so putting it together, the
merging takes time Θ(n lg(n/k)).
c. Viewing k as a function of n, as long as k(n) ∈ O(lg(n)), it has the same
asymptotics. In particular, for any constant choice of k, the asymptotics are
the same.
d. If we optimize the previous expression using our calculus 1 skills to get k, we
have that c1 n− nck2 = 0 where c1 and c2 are the coeffients of nk and n lg(n/k)
hidden by the asymptotics notation. In particular, a constant choice of k is
optimal. In practice we could find the best choice of this k by just trying
and timing for various values for sufficiently large n.

8
Problem 2-2

1. We need to prove that A0 contains the same elements as A, which is easily

seen to be true because the only modification we make to A is swapping
its elements, so the resulting array must contain a rearrangement of the
elements in the original array.

2. The for loop in lines 2 through 4 maintains the following loop invari-
ant: At the start of each iteration, the position of the smallest element of
A[i..n] is at most j. This is clearly true prior to the first iteration because
the position of any element is at most A.length. To see that each iter-
ation maintains the loop invariant, suppose that j = k and the position
of the smallest element of A[i..n] is at most k. Then we compare A[k] to
A[k − 1]. If A[k] < A[k − 1] then A[k − 1] is not the smallest element
of A[i..n], so when we swap A[k] and A[k − 1] we know that the smallest
element of A[i..n] must occur in the first k − 1 positions of the subarray,
the maintaining the invariant. On the other hand, if A[k] ≥ A[k − 1] then
the smallest element can’t be A[k]. Since we do nothing, we conclude that
the smallest element has position at most k − 1. Upon termination, the
smallest element of A[i..n] is in position i.

3. The for loop in lines 1 through 4 maintain the following loop invariant:
At the start of each iteration the subarray A[1..i − 1] contains the i − 1
smallest elements of A in sorted order. Prior to the first iteration i = 1,
and the first 0 elements of A are trivially sorted. To see that each iteration
maintains the loop invariant, fix i and suppose that A[1..i − 1] contains
the i − 1 smallest elements of A in sorted order. Then we run the loop in
lines 2 through 4. We showed in part b that when this loop terminates,
the smallest element of A[i..n] is in position i. Since the i − 1 smallest
elements of A are already in A[1..i − 1], A[i] must be the ith smallest
element of A. Therefore A[1..i] contains the i smallest elements of A in
sorted order, maintaining the loop invariant. Upon termination, A[1..n]
contains the n elements of A in sorted order as desired.

4. The ith iteration of the for loop of lines 1 through 4 will cause n − i
iterations of the for loop of lines 2 through 4, each with constant time
execution, so the worst-case running time is Θ(n2 ). This is the same as
that of insertion sort; however, bubble sort also has best-case running time
Θ(n2 ) whereas insertion sort has best-case running time Θ(n).

Problem 2-3

9
 a. If we assume that the arithmetic can all be done in constant time, then since
the loop is being executed n times, it has runtime Θ(n).

b. 1: y=0
2: for i=0 to n do
3: yi = x
4: for j=1 to n do
5: yi = yi x
6: end for
7: y = y + ai yi
8: end for

This code has runtime Θ(n2 ) because it has to compute each of the powers
of x. This is slower than Horner’s rule.
c. Initially, i = n, so, the upper bound of the summation is −1, so the sum
evaluates to 0, which is the value of y. For preservation, suppose it is true
for an i, then,
n−(i+1) n−i n−i
X X X
y = ai + x ak+i+1 xk = ai + x ak+i xk−1 = ak+i xk
k=0 k=1 k=0

At termination, i = 0, so is summing up to n − 1, so executing the body of

the loop a last time gets us the desired final result.
d. We just showed that the algorithm evaluated Σnk=0 ak xk . This is the value of
the polynomial evaluated at x.

Problem 2-4

a. The five inversions are (2, 1), (3, 1), (8, 6), (8, 1), and (6, 1).

b. The n-element array with the most inversions is hn, n − 1, . . . , 2, 1i. It has
n − 1 + n − 2 + . . . + 2 + 1 = n(n−1)
2 inversions.

c. The running time of insertion sort is a constant times the number of inver-
sions.
Pn Let I(i) denote the number of j < i such that A[j] > A[i]. Then
i=1 I(i) equals the number of inversions in A. Now consider the while
loop on lines 5-7 of the insertion sort algorithm. The loop will execute once
for each element of A which has index less than j is larger than A[j]. Thus,
it will execute I(j) times. We reach this while loop once for each iteration
of the for loop, so the number of constant time steps of insertion sort is
P n
j=1 I(j) which is exactly the inversion number of A.

10
d. We’ll call our algorithm M.Merge-Sort for Modified Merge Sort. In addition
to sorting A, it will also keep track of the number of inversions. The algorithm
works as follows. When we call M.Merge-Sort(A,p,q) it sorts A[p..q] and
returns the number of inversions in the elements of A[p..q], so lef t and right
track the number of inversions of the form (i, j) where i and j are both in
the same half of A. When M.Merge(A,p,q,r) is called, it returns the number
of inversions of the form (i, j) where i is in the first half of the array and j
is in the second half. Summing these up gives the total number of inversions
in A. The runtime is the same as that of Merge-Sort because we only add an
additional constant-time operation to some of the iterations of some of the
loops. Since Merge is Θ(n log n), so is this algorithm.

Algorithm 6 M.Merge-Sort(A, p, r)
if p < r then
q = b(p + r)/2c
lef t = M.M erge − Sort(A, p, q)
right = M.M erge − Sort(A, q + 1, r)
inv = M.M erge(A, p, q, r) + lef t + right
return inv
end if
return 0

11
Algorithm 7 M.Merge(A,p,q,r)
inv = 0
n1 = q − p + 1
n2 = r − q
let L[1, ..n1 ] and R[1..n2 ] be new arrays
for i = 1 to n1 do
L[i] = A[p + i − 1]
end for
for j = 1 to n2 do
R[j] = A[q + j]
end for
i=1
j=1
k=p
while i 6= n1 + 1 and j 6= n2 + 1 do
if L[i] ≤ R[j] then
A[k] = L[i]
i=i+1
else A[k] = R[j]
inv = inv + j // This keeps track of the number of inversions between
the left and right arrays.
j =j+1
end if
k =k+1
end while
if i == n1 + 1 then
for m = j to n2 do
A[k] = R[m]
k =k+1
end for
end if
if j == n2 + 1 then
for m = i to n1 do
A[k] = L[m]
inv = inv + n2 // Tracks inversions once we have exhausted the right
array. At this point, every element of the right array contributes an inversion.
k =k+1
end for
end if
return inv

12