Fundamentals

Data Structures and Algorithm Analysis in C, by Mark Allen Weiss, 2nd

edition, 1997, Addison-Wesley, ISBN 0-201-49840-5
Danang University of Science and Technology

Dang Thien Binh

[email protected]
 Readings and References
 Reading
 Chapters 1-2, Data Structures and Algorithm Analysis in C, Weiss

 Other References

Data Structures 2
 Mathematical Background

 Today, we will review:

 Logs and exponents
 Series
 Recursion
 Motivation for Algorithm Analysis

Data Structures 3
 Powers of 2

 Many of the numbers we use will be powers of 2

 Binary numbers (base 2) are easily represented in digital
computers
 each "bit" is a 0 or a 1
 20=1, 21=2, 22=4, 23=8, 24=16, 28=256, …
 an n-bit wide field can hold 2n positive integers:

 0  k  2n-1

Data Structures 4
 Unsigned binary numbers

 Each bit position represents a power of 2

 For unsigned numbers in a fixed width field
 the minimum value is 0
 the maximum value is 2n-1, where n is the number of bits in
the field
 Fixed field widths determine many limits
 5 bits = 32 possible values (25 = 32)
 10 bits = 1024 possible values (210 = 1024)

Data Structures 5
 Binary, Hex, and Decimal
28=256

27=128

26=64

25=32

24=16

21=2
22=4
23=8

20=1
Hex16 Decimal10
1 1 0x3 3
1 0 0 1 0x9 9
1 0 1 0 0xA 10
1 1 1 1 0xF 15
1 0 0 0 0 0x10 16
1 1 1 1 1 0x1F 31
1 1 1 1 1 1 1 0x7F 127
1 1 1 1 1 1 1 1 0xFF 255

Data Structures 6
 Logs and exponents

 Definition: log2 x = y means x = 2y

 the log of x, base 2, is the value y that gives x = 2y
 8 = 23, so log28 = 3
 65536= 216, so log265536 = 16

 Notice that log2x tells you how many bits are needed to
hold x values
 8 bits holds 256 numbers: 0 to 28-1 = 0 to 255
 log2256 = 8

Data Structures 7
 x = 0:.1:4
y = 2.^x
plot(x,y,'r')
hold on
plot(y,x,'g')
plot(y,y,'b')

2x and log2x
Data Structures
 x = 0:10
y = 2.^x
plot(x,y,'r')
hold on
plot(y,x,'g')
plot(y,y,'b')

2x and log2x
Data Structures
 Example: log2x and tree depth

 7 items in a binary tree, 3 = log27+1 levels

2 6

1 3 5 7

Data Structures 10
 Properties of logs (of the mathematical
kind)

 We will assume logs to base 2 unless specified

otherwise
 log AB = log A + log B
 A=2log2A and B=2log2B
 AB = 2log2A • 2log2B = 2log2A+log2B
 so log2AB = log2A + log2B

 note: log AB  log A•log B

Data Structures 11
 Other log properties
 log A/B = log A – log B
 log (AB) = B log A
 log log X < log X < X for all X > 0
 log log X = Y means
 log X grows slower than X
2Y
 2 X
called a “sub-linear” function

Data Structures 12
 A log is a log is a log

 Any base x log is equivalent to base 2 log within a con-

stant factor
log x B  log x B
x log x B  B
B  2log 2 B
(2log 2 x ) log x B  2log 2 B
x  2log 2 x
2log 2 x log x B  2log 2 B
log 2 x log x B  log 2 B
log 2 B
log x B 
log 2 x

Data Structures 13
 Arithmetic Series
N

 S (N )  1 2    N   i
i 1

 The sum is
 S(1) = 1
 S(2) = 1+2 = 3
 S(3) = 1+2+3 = 6
N
N ( N  1)
 
i 1
i
2
Why is this formula useful?

Data Structures 14
 Quicky Algorithm Analysis

 Consider the following program segment:

for (i = 1; i <= N; i++)
for (j = 1; j <= i; j++)
printf(“Hello\n”);
 How many times is “printf” executed?
 Or, How many Hello’s will you see?

Data Structures 15
 What is actually being executed?
 The program segment being analyzed:
for (i = 1; i <= N; i++)
for (j = 1; j <= i; j++)
printf(“Hello\n”);
 Inner loop executes “printf” i times in the ith iteration
 j goes from 1 to i
 There are N iterations in the outer loop
 i goes from 1 to N

Data Structures 16
 Lots of hellos

 Total number of times “printf” is executed =

N
N ( N  1)
1  2  3  ...   i 
2
 Congratulations
i 1
- You’ve just analyzed your first pro-
gram!
 Running time of the program is proportional to N(N+1)/2 for
all N
 Proportional to N2

Data Structures 17
 Recursion

 Classic (bad) example: Fibonacci numbers Fn

1, 1, 2, 3, 5, 8, 13, 21, 34, …

 First two are defined to be 1

 Rest are sum of preceding two
Leonardo Pisano
 F =F +F (n > 1) Fibonacci (1170-1250)
n n-1 n-2

Data Structures 18
 Recursive Procedure for Fibonacci
Numbers
int fib(int i) {
if (i < 0) return 0;
if (i == 0 || i == 1)
return 1;
else
return fib(i-1)+fib(i-2);
}
 Easy to write: looks like the definition of Fn
 But, can you spot the big problem?

Data Structures 19
 Recursive Calls of Fibonacci Procedure

 Re-computes fib(N-i) multiple times!

Data Structures 20
 Iterative Procedure for Fibonacci Num-
bers

int fib_iter(int i) {
int fib0 = 1, fib1 = 1, fibj = 1;
if (i < 0) return 0;
for (int j = 2; j <= i; j++) {
fibj = fib0 + fib1;
fib0 = fib1;
fib1 = fibj;
}
return fibj;
}
 More variables and more bookkeeping but avoids
repetitive calculations and saves time.

Data Structures 21
 Recursion Summary

 Recursion may simplify programming, but beware

of generating large numbers of calls
 Function calls can be expensive in terms of time and
space
 Be sure to get the base case(s) correct!
 Each step must get you closer to the base case

Data Structures 22
 Motivation for Algorithm Analysis

 Suppose you are

given two algos A and
B for solving a prob- TA

Run Time
lem
 The running times
TA(N) and TB(N) of A
and B as a function of TB
input size N are given
Input Size N

Which is better?

Data Structures 23
 More Motivation
 For large N, the running time of A and B is:

Now which
algorithm would
Run Time

TA(N) = 50N
you choose?

TB(N) = N2

Input Size N

Data Structures 24
 Asymptotic Behavior

 The “asymptotic” performance as N  , regard-

less of what happens for small input sizes N, is
generally most important
 Performance for small input sizes may matter in
practice, if you are sure that small N will be com-
mon forever
 We will compare algorithms based on how they
scale for large values of N

Data Structures 25