Scribd
Upload
273 views

hw02 Solution PDF

The document summarizes solutions to homework problems on fundamental algorithms. It includes: 1) Solutions to recurrences using the master method and substitution proofs, finding recurrences for recursive functions have running time Θ(lg n). 2) Finding the exact solution to a recurrence of the form T(n) = aT(n-1) + k as T(n) = a^n c + (1 - a^n)/(1 - a) k. 3) Proving correctness of iterative binary search using a loop invariant showing the algorithm runs in Θ(lg n) time. 4) Giving pseudocode for recursive binary search and finding its Θ(lg n

Uploaded by

siddharth1k
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
273 views

hw02 Solution PDF

The document summarizes solutions to homework problems on fundamental algorithms. It includes: 1) Solutions to recurrences using the master method and substitution proofs, finding recurrences for recursive functions have running time Θ(lg n). 2) Finding the exact solution to a recurrence of the form T(n) = aT(n-1) + k as T(n) = a^n c + (1 - a^n)/(1 - a) k. 3) Proving correctness of iterative binary search using a loop invariant showing the algorithm runs in Θ(lg n) time. 4) Giving pseudocode for recursive binary search and finding its Θ(lg n

Uploaded by

siddharth1k
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 9

Fundamental Algorithms

CSCI-GA.1170-001/Summer 2016
Solution to Homework 2
Problem 1. (2 points) For the following recurrences, obtain a solution using the master method
(or explain why not applicable), then give a substitution proof for the upper bound:
(a) T (n) = 4T ( 3n ) + n. (CLRS 4.3-7)
(b) T (n) = 4T ( 2n ) + n2 . (CLRS 4.3-8)
(c) T (n) = 4T ( 2n ) + n2 lg n. (CLRS 4.5-4)
Solution:
(a) The recurrence T (n) = 4T ( 3n ) + n is of the form T (n) = aT ( nb ) + f (n), and n = O(nlog3 4 ),
so we can apply case 1 of the master method to obtain T (n) = (nlog3 4 ).
Let us verify the upper bound T (n) = O(nlog3 4 ) by substitution. (By convention, we assume
T (0) = (1) or T (1) = (1), as appropriate, and omit the base cases in this and other
inductive proofs.)
With inductive hypothesis:
T (k) cklog3 4 for all 0 < k < n,
Let us perform the inductive step by showing:
T (n) cnlog3 4 .

(1.1)

By definition of T (n) and then by inductive hypothesis:


n
+ n,
T (n) = 4T
3
 log 4
3
n
T (n) 4 c
+ n,
3
T (n) cnlog3 4 + n.
Showing (1.1) is thus equivalent to showing:
cnlog3 4 + n cnlog3 4 ,
n 0.
Which contradicts the inductive hypothesis, as well as our fundamental assumptions about
the meaning and domain of T (n).
Let us formulate a stronger inductive hypothesis by including a lower-order term:
T (k) c1 klog3 4 c2 k for all 0 < k < n.
1

And perform the inductive step by showing:


T (n) c1 nlog3 4 c2 n.

(1.2)

By definition of T (n) and then by inductive hypothesis:


n
T (n) = 4T
+ n,
3

 log 4
3
n
n
c2
+ n,
T (n) 4 c1
3
3
4
T (n) c1 nlog3 4 c2 n + n.
3
Showing (1.2) is thus equivalent to showing:
4
c1 nlog3 4 c2 n + n c1 nlog3 4 c2 n,
3
4
c2 n + n c2 n,
3
4
c2 + 1 c2 ,
3
c2 3.
Which does not contradict any of our assumptions and thus completes the inductive step
and the proof.
(b) The recurrence T (n) = 4T ( 2n ) + n2 is of the form T (n) = aT ( nb ) + f (n), and n2 = (n2 ), so
we can apply case 2 of the master method to obtain T (n) = (n2 lg n).
Let us verify the upper bound T (n) = O(n2 lg n) by substitution.
With inductive hypothesis:
T (k) ck2 lg k for all 0 < k < n.
Let us perform the inductive step by showing:
T (n) cn2 lg n.
By definition of T (n) and then by inductive hypothesis:
n
+ n2 ,
T (n) = 4T
2
 2

n
n
T (n) 4 c
lg
+ n2 ,
2
2
n
2
2
T (n) cn lg + n ,
2
2
T (n) cn lg n cn2 + n2 .

(1.3)

Showing (1.3) is thus equivalent to showing:


cn2 lg n cn2 + n2 cn2 lg n,
cn2 + n2 0,
c 1.
Which does not contradict any of our assumptions and thus completes the inductive step
and the proof.
(c) While T (n) = 4T ( 2n ) + n2 lg n is of the form T (n) = aT ( nb ) + f (n), neither of the master
method cases apply here, as the difference between f (n) = n2 lg n and nlog b a = n2 is less
than polynomial. While case 3 might seem applicable, as indeed:
n2 lg n = (n2 ).
We can see that:
n2 lg n 6= (n2+" ).
By looking at a ratio:
n2 lg n n2 lg n lg n
= 2 " = " .
n2+"
n n
n
And observing that n" grows faster than lg n for any " > 0.
There is a definition of master method that does, in fact, cover this case see CLRS exercise
4.6-2 or https://en.wikipedia.org/wiki/Master_theorem but we havent discussed
it in class.
Problem 2. (2 points) Find the exact solution of the recurrence:

c
if n = 0,
T (n) =
aT (n 1) + k if n > 0.
Solution: Computing the first few values:
T (0) = c
T (1) = aT (0) + k = ac + k
T (2) = aT (1) + k = a(ac + k) + k = a2 c + ak + k
T (3) = aT (2) + k = a(a2 c + ak + k) + k = a3 c + a2 k + ak + k
Seems to suggest:
T (n) = a c +
n

n1
X
i=0

a i k.

(1.4)

For a = 1 we have:
T (n) = c +

n1
X

k = c + nk.

(1.5)

i=0

That we can prove by induction from the base case:


T (0) = c + 0 k = c.
Taking the inductive hypothesis to be that for some n > 0:
T (n) = c + nk.
And performing inductive step to show that (1.5) holds for n + 1:
T (n + 1) = aT (n) + k
= T (n) + k
= c + nk + k
= c + (n + 1)k.
For a 6= 1, applying the formula for geometric series to (1.4):
T (n) = a n c + k

1 an
.
1a

Let us prove (1.6) by induction from the base case:


1 a0
= c.
1a
Taking the inductive hypothesis to be that for some n > 0:
T (0) = a0 c + k

1 an
.
1a
And performing the inductive step to show that (1.6) holds for n + 1:
T (n) = a n c + k

T (n + 1) = a

n+1

1 a n+1
.
c+k
1a

By definition of T (n) and then by inductive hypothesis:


T (n + 1) = aT (n) + k

1 an
n
+ k.
=a a c+k
1a
Simplifying:
1 an
+k
1a
a(1 a n ) + (1 a)
= a n+1 c + k
1a
n+1
1

a
= a n+1 c + k
.
1a

T (n + 1) = a n+1 c + ka

(1.6)

With the base case and inductive step in place for both (1.5) and (1.6), we conclude that (1.4)
holds for all natural numbers n and all values of c, a, and k.
Problem 3. (2 points) Give pseudocode for iterative binary search. Prove correctness of your
algorithm using a loop invariant, state the worst-case running time.
Solution:
Iterative binary search in Python:

def iterative_binary_search (a , key ):


lo = 0
hi = len (a)
while hi - lo > 0:
mid = lo + ( hi - lo ) / 2
if key == a[ mid ]:
return mid
elif key < a[ mid ]:
hi = mid
elif key > a[ mid ]:
lo = mid + 1
return -1
We prove the algorithm correct using the following two-part loop invariant:
At the start of each iteration of the while loop on lines 4-11:
1. lo = 0 or ke y > a[lo 1].
2. hi = len(a) or ke y < a[hi].
Initialization: Before the first iteration, lo = 0, hi = len(a) and both invariants hold.
Maintenance: The loop body has three possible executions:
1. ke y = a[mid]. The loop exits early, so there is no next iteration and no need to
demonstrate maintenance.
2. ke y < a[mid]. Observe that after line 9, hi = mid and the entry condition becomes
equivalent to ke y < a[hi] (invariant 2). Neither ke y nor lo change, so invariant 1
remains true as well.
3. ke y > a[mid]. Observe that after line 11, lo = mid + 1 and the entry condition
becomes equivalent to ke y > a[lo 1] (invariant 1). Neither ke y nor hi change,
so invariant 2 remains true as well.
Termination: We first show that the loop terminates. On entry, hi lo > 0 and there are
three possible executions:
1. ke y = a[mid]. The loop terminates immediately.
5

1
2
3
4
5
6
7
8
9
10
11
12

2. ke y < a[mid]. The adjusted range [lo, mid) is strictly smaller than the original
range [lo, hi):
mid lo < hi lo,
mid < hi,

hi lo
lo +
< hi,
2

hi lo
< hi lo.
2

For integer hi and lo, and hi lo > 0, even hi lo gives:

hi lo
hi lo
=
,
2
2
hi lo
< hi lo,
2
hi lo > 0.
Odd hi lo gives:

hi lo
hi lo 1
=
,
2
2
hi lo 1
< hi lo,
2
hi lo > 1.

Both of which hold by the loop condition hi lo > 0.


3. ke y > a[mid]. By analogous argument, the adjusted range [mid + 1, hi) is strictly
smaller that the original range [lo, hi).
As on every iteration the loop either terminates immediately or executes over a strictly
smaller range, we conclude that the loop terminates.
We now need to show that the invariants hold for both termination conditions:
1. If the loop terminates due to return mid at line 7, neither of lo, hi, or ke y have
been updated since the start of the iteration, so both invariants hold by maintenance.
With both invariants holding and with entry condition ke y = a[mid], we conclude
that return mid correctly returns the index of ke y.
2. If the loop terminates due to hi lo 0 at line 4, neither of lo, hi, or ke y have
been updated since the start of the iteration, so both invariants hold by maintenance.
The range [lo, hi), however, now includes zero or negative number of elements and
thus cannot include ke y. With both invariants holding and with no possibility of
ke y being in [lo, hi), we conclude that return -1 correctly returns the special "not
found" value.
With both invariants holding in all cases, and having considered both "key found" and
"key not found" scenarios, we conclude that the algorithm is correct.
6

The worst-case running time can be observed when ke y is not present in a and the
algorithm terminates due to hi lo 0. Each iteration of the loop reduces the range
from [lo, hi) to either [lo, mid) or [mid +1, hi) and thus reduces the number of elements
in the range to either mid lo or hi mid 1. By definition of mid:

hi lo
hi lo
lo =
,
mid lo = lo +
2
2

hi lo
hi mid 1 = hi lo
1.
2
Considering even hi lo:
mid lo =

hi lo
,
2

hi mid 1 = hi lo

hi lo
hi lo
1=
1.
2
2

Thus, hi lo, initially equal to len(a), is reduced roughly in half each iteration, leading
to the worst-case running time of (lg n) with n = len(a).
Problem 4. (2 points) Give pseudocode for recursive binary search. Formulate and solve a
recurrence describing the worst-case running time.
Solution:
Recursive binary search in Python:

def recursive_binary_search (a , key , lo =0 , hi = len (a )):


if hi - lo <= 0:
return -1
mid = lo + ( hi - lo ) / 2
if key == a[ mid ]:
return mid
elif key < a[ mid ]:
return recursive_binary_search (a , key , lo , mid )
elif key > a[ mid ]:
return recursive_binary_search (a , key , mid + 1, hi )
Observing that going from [lo, hi) to either [lo, mid) or [mid + 1, hi) approximately
halves the range size, and that only a constant number of operations are executed outside
recursive calls, we can express the worst-case running time as a recurrence (omitting the
base case by convention):
n
+ (1).
T (n) = T
2
By case 2 of the master method, T (n) = (lg n).

1
2
3
4
5
6
7
8
9
10

Problem 5. (3 points) Let A = (a1 , a2 , ..., an ) be a sequence of n distinct numbers. A pair (i, j)
is called an inversion of A if i < j and ai > a j . Give pseudocode for a modification of merge
sort to count the number of inversions of the sequence. Formulate and solve a recurrence
describing the worst-case running time.
Solution:
Considering an execution of merge(left, right) with i and j keeping track of the current element in left and right respectively, we observe that taking element right[j]
before exhausting all elements in left, indicates that right[j] forms inversions with all
len(left) - i yet unprocessed elements in left. We then extend a typical implementation of mergesort with code to:
1. Count the number of inversions right[j] forms with elements in left (line 25).
2. Return the total number of inversions detected by this invocation of merge (line 30).
3. Return the total number of inversions detected in left subarray, right subarray, and
inversions crossing the boundary between two subarrays (lines 6-8 and 10).
Resulting in the following Python code:

def mergesort (a ):
if len (a) < 2:
return a , 0
mid = len (a) / 2
left , inv_left = mergesort (a [: mid ])
right , inv_right = mergesort (a[ mid :])
merged , inv_split = merge ( left , right )
return merged , inv_left + inv_right + inv_split
def merge ( left , right ):
merged = []
i = 0
j = 0
inv = 0
while i < len ( left ) and j < len ( right ):
if left [i] <= right [j ]:
merged . append ( left [i ])
i += 1
else :
merged . append ( right [j ])
j += 1
inv += len ( left ) - i

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25

26
27
28
29
30

merged . extend ( left [i :])


merged . extend ( right [j :])
return merged , inv
Observing that we recurse on both a[0, mid) and a[mid, len(a)), each approximately
half of len(a), and that only a linear amount of work is done outside recursive calls, we
can formulate a recurrence for the worst-case running time:
n
+ (n).
T (n) = 2T
2
By case 2 of the master method, T (n) = (n lg n).

You might also like