Data Structures and Algorithms

Solving Recurrence Relations

Chris Brooks Department of Computer Science University of San Francisco

Department of Computer Science University of San Francisco p.1/30

4-0:
for (i=1; i<=n*n; i++) for (j=0; j<i; j++) sum++;

Algorithm Analysis

Department of Computer Science University of San Francisco p.2/30

4-1:
for (i=1; i<=n*n; i++) for (j=0; j<i; j++) sum++;

Algorithm Analysis

Executed n*n times Executed <= n*n times O(1)

Running Time: O(n4 ) But can we get a tighter bound?

Department of Computer Science University of San Francisco p.3/30

4-2:
for (i=1; i<=n*n; i++) for (j=0; j<i; j++) sum++;

Algorithm Analysis

Exact # of times sum++ is executed:

n2

i=1

n2 (n2 + 1) i = 2 n4 + n 2 = 2 (n4 )

Department of Computer Science University of San Francisco p.4/30

4-3:
long power(long x, long n) if (n == 0) return 1; else return x * power(x, n-1);

Recursive Functions

How many times is this executed?

Department of Computer Science University of San Francisco p.5/30

4-4:

Recurrence Relations

T (n) = Time required to solve a problem of size n Recurrence relations are used to determine the running time of recursive programs recurrence relations themselves are recursive T (0) = T (n) = time to solve problem of size 0 Base Case time to solve problem of size n Recursive Case

Department of Computer Science University of San Francisco p.6/30

4-5:
long power(long x, long n) if (n == 0) return 1; else return x * power(x, n-1);

Recurrence Relations

T (0) = c1 T (n) = c2 + T (n 1)

for some constant c1 for some constant c2

Department of Computer Science University of San Francisco p.7/30

4-6: T (0) = c1 T (n) = T (n 1) + c2

Solving Recurrence Relations

If we knew T (n 1), we could solve T (n). T (n) = T (n 1) + c2

Department of Computer Science University of San Francisco p.8/30

4-7: T (0) = c1 T (n) = T (n 1) + c2

Solving Recurrence Relations

If we knew T (n 1), we could solve T (n). T (n) = T (n 1) + c2 = T (n 2) + c2 + c2 = T (n 2) + 2c2 T (n 1) = T (n 2) + c2

Department of Computer Science University of San Francisco p.9/30

4-8: T (0) = c1 T (n) = T (n 1) + c2

Solving Recurrence Relations

If we knew T (n 1), we could solve T (n). T (n) = T (n 1) + c2 = T (n 2) + c2 + c2 = T (n 2) + 2c2 = T (n 3) + c2 + 2c2 = T (n 3) + 3c2 T (n 1) = T (n 2) + c2 T (n 2) = T (n 3) + c2

Department of Computer Science University of San Francisco p.10/30

4-9: T (0) = c1 T (n) = T (n 1) + c2

Solving Recurrence Relations

If we knew T (n 1), we could solve T (n). T (n) = T (n 1) + c2 = T (n 2) + c2 + c2 = T (n 2) + 2c2 = T (n 3) + c2 + 2c2 = T (n 3) + 3c2 = T (n 4) + 4c2 T (n 1) = T (n 2) + c2 T (n 2) = T (n 3) + c2 T (n 3) = T (n 4) + c2

Department of Computer Science University of San Francisco p.11/30

4-10: T (0) = c1 T (n) = T (n 1) + c2

Solving Recurrence Relations

If we knew T (n 1), we could solve T (n). T (n) = T (n 1) + c2 = T (n 2) + c2 + c2 = T (n 2) + 2c2 = T (n 3) + c2 + 2c2 = T (n 3) + 3c2 = T (n 4) + 4c2 = ... = T (n k ) + kc2 T (n 1) = T (n 2) + c2 T (n 2) = T (n 3) + c2 T (n 3) = T (n 4) + c2

Department of Computer Science University of San Francisco p.12/30

4-11: T (0) = c1 T (n) = T (n k ) + k c2 If we set k = n, we have: T (n) = T (n n) + nc2 = T (0) + nc2 = c1 + nc2 (n)

Solving Recurrence Relations

for all k

Department of Computer Science University of San Francisco p.13/30

4-12:

Building a Better Power

Can we avoid making a linear number of function calls?

long if if if power(long x, long n) (n==0) return 1; (n==1) return x; ((n % 2) == 0) return power(x*x, n/2); else return power(x*x, n/2) * x;

Department of Computer Science University of San Francisco p.14/30

4-13:
long if if if power(long x, long n) (n==0) return 1; (n==1) return x; ((n % 2) == 0) return power(x*x, n/2); else return power(x*x, n/2) * x;

Building a Better Power

T (0) = c1 T (1) = c2 T (n) = T (n/2) + c3 (Assume n is a power of 2)

Department of Computer Science University of San Francisco p.15/30

4-14: T (n) = T (n/2) + c3

Solving Recurrence Relations

Department of Computer Science University of San Francisco p.16/30

4-15: T (n) = T (n/2) + c3 = T (n/4) + c3 + c3 = T (n/4) + 2c3

Solving Recurrence Relations

T (n/2) = T (n/4) + c3

Department of Computer Science University of San Francisco p.17/30

4-16: T (n) = T (n/2) + c3 = T (n/4) + c3 + c3 = T (n/4) + 2c3 = T (n/8) + c3 + 2c3 = T (n/8) + 3c3

Solving Recurrence Relations

T (n/2) = T (n/4) + c3 T (n/4) = T (n/8) + c3

Department of Computer Science University of San Francisco p.18/30

4-17:

Solving Recurrence Relations

T (n/2) = T (n/4) + c3 T (n/4) = T (n/8) + c3 T (n/8) = T (n/16) + c3

T (n) = T (n/2) + c3 = T (n/4) + c3 + c3 = T (n/4) + 2c3 = T (n/8) + c3 + 2c3 = T (n/8) + 3c3 = T (n/16) + c3 + 3c3 = T (n/16) + 4c3

Department of Computer Science University of San Francisco p.19/30

4-18:

Solving Recurrence Relations

T (n/2) = T (n/4) + c3 T (n/4) = T (n/8) + c3 T (n/8) = T (n/16) + c3 T (n/16) = T (n/32) + c3

T (n) = T (n/2) + c3 = T (n/4) + c3 + c3 = T (n/4)2c3 = T (n/8) + c3 + 2c3 = T (n/8)3c3 = T (n/16) + c3 + 3c3 = T (n/16) + 4c3 = T (n/32) + c3 + 4c3 = T (n/32) + 5c3

Department of Computer Science University of San Francisco p.20/30

4-19:

Solving Recurrence Relations

T (n/2) = T (n/4) + c3 T (n/4) = T (n/8) + c3 T (n/8) = T (n/16) + c3 T (n/16) = T (n/32) + c3

T (n) = T (n/2) + c3 = T (n/4) + c3 + c3 = T (n/4)2c3 = T (n/8) + c3 + 2c3 = T (n/8)3c3 = T (n/16) + c3 + 3c3 = T (n/16) + 4c3 = T (n/32) + c3 + 4c3 = T (n/32) + 5c3 = ... = T (n/2k ) + kc3

Department of Computer Science University of San Francisco p.21/30

4-20: T (0) = c1 T (1) = c2 T (n) = T (n/2) + c3 T (n) = T (n/2k ) + kc3

Solving Recurrence Relations

We want to get rid of T (n/2k ). We get to a relation we can solve directly when we reach T (1) n/2k = 1 n = 2k lg n = k

Department of Computer Science University of San Francisco p.22/30

4-21: T (0) = c1 T (1) = c2 T (n) = T (n/2) + c3 T (n) = T (n/2k ) + kc3

Solving Recurrence Relations

We want to get rid of T (n/2k ). We get to a relation we can solve directly when we reach T (1) lg n = k T (n) = = = T (n/2lg n ) + lg nc3 T (1) + c3 lg n c2 + c3 lg n (lg n)

Department of Computer Science University of San Francisco p.23/30

4-22: Power
long if if if power(long x, long n) (n==0) return 1; (n==1) return x; ((n % 2) == 0) return power(x*x, n/2); else return power(x*x, n/2) * x;

Modications

Department of Computer Science University of San Francisco p.24/30

4-23: Power
long if if if power(long x, long n) (n==0) return 1; (n==1) return x; ((n % 2) == 0) return power(power(x,2), n/2); else return power(power(x,2), n/2) * x;

Modications

This version of power will not work. Why?

Department of Computer Science University of San Francisco p.25/30

4-24: Power
long if if if power(long x, long n) (n==0) return 1; (n==1) return x; ((n % 2) == 0) return power(power(x,n/2), 2); else return power(power(x,n/2), 2) * x;

Modications

This version of power also will not work. Why?

Department of Computer Science University of San Francisco p.26/30

4-25: Power
long if if if power(long x, long n) (n==0) return 1; (n==1) return x; ((n % 2) == 0) return power(x,n/2) * power(x,n/2); else return power(x,n/2) * power(x,n/2) * x;

Modications

This version of power does work. What is the recurrence relation that describes its running time?

Department of Computer Science University of San Francisco p.27/30

4-26: Power
long if if if power(long x, long n) (n==0) return 1; (n==1) return x; ((n % 2) == 0) return power(x,n/2) * power(x,n/2); else return power(x,n.2) * power(x,n/2) * x;

Modications

T (0) = c1 T (1) = c2 T (n) = T (n/2) + T (n/2) + c3 = 2T (n/2) + c3 (Again, assume n is a power of 2)

Department of Computer Science University of San Francisco p.28/30

4-27:

Solving Recurrence Relations

T (n/2) = 2T (n/4) + c3 T (n/4) = 2T (n/8) + c3

T (n) = 2T (n/2) + c3 = 2[2T (n/4) + c3 ]c3 = 4T (n/4) + 3c3 = 4[2T (n/8) + c3 ] + 3c3 = 8T (n/8) + 7c3 = 8[2T (n/16) + c3 ] + 7c3 = 16T (n/16) + 15c3 = 32T (n/32) + 31c3 ... = 2k T (n/2k ) + (2k 1)c3

Department of Computer Science University of San Francisco p.29/30

4-28:

Solving Recurrence Relations

T (0) = c1 T (1) = c2 T (n) = 2k T (n/2k ) + (2k 1)c3 Pick a value for k such that n/2k = 1: n/2k = 1 n = 2k lg n = k T (n) = = = = 2lg n T (n/2lg n ) + (2lg n 1)c3 nT (n/n) + (n 1)c3 nT (1) + (n 1)c3 nc2 + (n 1)c3 (n)

Department of Computer Science University of San Francisco p.30/30