Introduction

1. Algorithms

2. Order

3. Analysis of Algorithm

4. Some Mathematical Background

Abdi M. CoSc3094 D&A Topic: Introduction 1

of Algo.
What is an algorithm?
A simple, unambiguous, mechanical
procedure to carry out some task.
 
Why algorithm instead of program?
1. Writing an algorithm is simpler (we don’t
need to worry about the detailed
implementation, or the language syntax).

2. An algorithm is easier to read than a

program written in, for instance, C.
Abdi M. CoSc3094 D&A Topic: Introduction 2
of Algo.
 How to represent an algorithm?

1. Give a description in your own language,

e.g. English, Afan Oromo, Amharic, …

2. Pseudo code

3. Graphical

Abdi M. CoSc3094 D&A of Algo. Topic: Introduction 3

 Analysis and Design of Algorithm
• Analysis: predict the cost of an algorithm in terms of
resources and performance

• Design: design algorithms which minimize the cost

Abdi M. CoSc3094 Topic: Introduction 4

D&A of Algo.
Example – multiplying two positive integers A
and B
 
For example: 45*19
 
Usually:
45
19 (x

405
45 (+
855
Abdi M. CoSc3094 D&A of Topic: Introduction 5
Algo.
A different algorithm:
 
Multiplier Multiplicand Result
(A/2) (B*2) (pick numbers in column
2 when the corresponding
number under the
multiplier is odd)
 
45 19 19
22 38
11 76 76
5 152 152
2 304
1 608 608 (+
855

Abdi M. CoSc3094 Topic: Introduction 6

D&A of Algo.
An instance of a problem is a specific assignment of values to
the parameters.

This algorithm can be used for multiplying any two positive

integers, we say that (45, 19) is an instance of this problem.
Most problems have infinite collection of instances.

It’s ok to define the domain (i.e. the set of instances) to be

considered, and the algorithm should work for all instances in
that domain.

Although the above algorithm will not work if the first operand
is negative, this does not invalidate the algorithm since (-45, 19)
is not an instance of the problem being considered.

Abdi M. CoSc3094 Topic: Introduction 7

D&A of Algo.
 Order:
Usually we use the frequency count to compare algorithms.
Consider the following 3 programs:

(a) (b) (c)

x←x+y for i ← 1 to n do for i ← 1 to n do
x←x+y for j ← 1 to n do
end x←x+y
end
end

The frequency count of stmt x ← x + y is 1, n, n2.

No matter which machine we use to run these programs, we know
that the execution time of (b) is n times the execution time of (a).
Abdi M. CoSc3094 Topic: Introduction 8
D&A of Algo.
 Asymptotic Analysis:
The study of change in performance of the algorithm with the
change in the order of the input size is defined as asymptotic
analysis.
Asymptotic Notations
•Asymptotic notations are the mathematical notations used to
describe the running time of an algorithm when the input tends
towards a particular value or a limiting value.
•For example: In bubble sort, when the input array is already
sorted, the time taken by the algorithm is linear i.e. the best case.
•But, when the input array is in reverse condition, the algorithm
takes the maximum time (quadratic) to sort the elements i.e. the
worst
Abdicase.
M. CoSc3094 Topic: Introduction 9
D&A of Algo.
 Asymptotic Notations…

There are mainly three asymptotic notations:

•Big-O notation
•Omega notation
•Theta notation

Abdi M. CoSc3094 Topic: Introduction 10

D&A of Algo.
 Big-O notation:
 
Def f(n) = O(g(n)) if and only if  2 positive constants c and n0,

such that 
|f(n)|  c  |g(n)|  n  n0.
So, g(n) actually is the upper bound of f(n).
c | g ( n) |

| f ( n) |

n0
Figure 1. Illustrating “big O”
Abdi M. CoSc3094 Topic: Introduction 11
D&A of Algo.
Big-O notation…
The general step wise procedure for Big-O runtime analysis
is as follows:  
•Figure out what the input is and what n represents.
•Express the maximum number of operations, the
algorithm performs in terms of n.
•Eliminate all excluding the highest order terms.
•Remove all the constant factors.

Abdi M. CoSc3094 Topic: Introduction 12

D&A of Algo.
Big-O notation…
Some of the useful properties of Big-O notation analysis are
as follow: 
 Constant Multiplication: 
If f(n) = c.g(n), then O(f(n)) = O(g(n)) ; where c is a
nonzero constant. 
 Polynomial Function: 
If f(n) = a0 + a1.n + a2.n2 + —- + am.nm, then O(f(n)) =
O(nm). 
 

Abdi M. CoSc3094 Topic: Introduction 13

D&A of Algo.
Big-O notation…

 Summation Function: 
If f(n) = f1(n) + f2(n) + —- + fm(n) and fi(n)≤fi+1(n) ∀ i=1, 2,
—-, m, 
then O(f(n)) = O(max(f1(n), f2(n), —-, fm(n))). 
 Logarithmic Function: 
If f(n) = logan and g(n)=logbn, then O(f(n))=O(g(n)) 
; all log functions grow in the same manner in terms of Big-
O

Abdi M. CoSc3094 Topic: Introduction 14

D&A of Algo.
Examples:
• Is 17 n2 – 5 = O(n2)?
 17n 2 5  17n 2 n  1
(c 17, n0  1)
 17n 2 5  O(n 2 )
• Is 35 n3 + 100 = O(n3)?
 35n 3  100  36n 3 n  5
(c  36, n0  5)
 35n 3  100  O(n 3 )
• Is 6  2n + n2 = O(2n)?
 6  2n  n2  7  2n n  5
(c  7, n0  5)
 6  2 n  n 2  O(2 n )
Abdi M. CoSc3094 Topic: Introduction 15
D&A of Algo.
Complexity classes

f(n) n! 2n n3
n2

n log n

n (linear time)

log n

n
Figure 2. Growth rates of some important complexity classes
Abdi M. CoSc3094 Topic: Introduction 16
D&A of Algo.
 Assume that we have a machine that can execute
1,000,000 required operations per sec

Algorithm 1 Algorithm 2 Algorithm 3 Algorithm 4 Algorithm 5

Frequency 33n 6nlogn 13n2 3.4n3 2n

count
n=10 < 1 sec < 1 sec < 1 sec < 1 sec < 1 sec
n=10,000 < 1 sec < 1 sec 22 min 39 days many many
centuries

Table 1. Execution time for algorithms with the

given time complexities

Abdi M. CoSc3094 Topic: Introduction 17

D&A of Algo.
Note:
log n
n(linear)
n log n polynomial time (easy or tractable)
n2
n3  

2n
exponential time (hard or intractable)
n!

Abdi M. CoSc3094 Topic: Introduction 18

D&A of Algo.
Whether your program is the best or not?
(Comparing algorithms)

The problem: Counting the number of 1’s in a n-bit string

algorithms:

1. c ← 0;
for i ← 1 to n do
if bi = 1 then c ← c+1;

Complexity: T(n) = O(n)

Abdi M. CoSc3094 Topic: Introduction 19

D&A of Algo.
2. Assume B = bnbn-1…b2b1
c ← 0;
while B ≠ 0 do
c ← c+1;
B ← B  (B –1 ); ( Note that  is “logical
and” )

Complexity: T(n) = O( # of 1’s)

example: B = 100011
c←1 100011 B
) 100010 B-1
c←2 100010 B’
) 100001 B’-1
c←3 100000 B’’
) 011111 B’’-1
Abdi M. CoSc3094 000000
Topic: Introduction B’’’ 20
D&A of Algo.
3. Can we do it in log n time?
B = 11010001
b8 b7 b6 b5 b4 b3 b2 b1

B 1 1 0 1 0 0 0 1

Odd 1 1 0 1
Shift even to right 1 0 0 0
B1 = Bodd + Beven 10 01 00 01
 

Odd 01 01
Shift even to right 10 00
B2 = Bodd + Beven 0011 0001

Odd 0001
Shift even to right 0011
B3 = Bodd + Beven 0100

Abdi M. CoSc3094 Topic: Introduction 21

D&A of Algo.
Idea:
4 11010001

 
3 1101 0001 1

==> T(n) = O(logn)

2 11 1 01 00 0 01 1
 
1 1 0 1 0 0 0 1
Note:
- The familiar machine that we used can only execute one
operation in each step  sequential machine.
-This stated algorithm requires the machine to have more than
one operation done in each step  parallel machine
Abdi M. CoSc3094 Topic: Introduction 22
D&A of Algo.
Ω notation
Def f ( n)  ( g (n)) iff  two positive constants c and n0,
such that
| f (n) |  c | g (n) | n  n0
So, g(n) is the lower bound of f(n).
| f ( n) |

c | g (n) |

n0
Figure 3. Illustrating Ω
Abdi M. CoSc3094 Topic: Introduction 23
D&A of Algo.
Examples:
•Is 3n  2  (n) ?
 3n  2  3  n n  1
(c  3, n0  1)
 3n  2  (n)
•Is 3n  2  (1) ?
 3n  2  1  1 n  1
(c  1, n0  1)
 3n  2  (1)

6  2 n  n 2  (n100 ) value of n0
•Is  6?  2 n  n 2  ? n100 n  ?

n 2 100
When n is bigger, 6 faster
2 will grow
n 2 than
n n . (n you
( Yes, ) can find n0 )
100

Abdi M. CoSc3094 Topic: Introduction 24

D&A of Algo.
Θ notation
Def f ( n)  ( g (n)) iff  three positive constants, c1, c2, n0 ,

such that c1  | g (n) |  | f (n) |  c 2  | g (n) | n  n0 .

c 2  | g ( n) |

| f ( n) |

c1  | g (n) |

n0 Figure 4. Illustrating Θ
Abdi M. CoSc3094 Topic: Introduction 25
D&A of Algo.
Examples:

• Is 3n + 2 = (n)?
 3n  3n  2  4n n  2
(c1  3, c 2  4, n0  2)
 3n  2  (n)

• Is 3n + 2 = (n2)?
2
 3n  2  (n )
2
 3n  2  (n )

Abdi M. CoSc3094 Topic: Introduction 26

D&A of Algo.
• Is 6  2 n  n 2  (2 n )?
 6  2n  6  2n  n2  7  2n n  4
(c1  6, c 2  7, n0  4)
 6  2 n  n 2  ( 2 n )

• Is 4n 3  3n 2  (n 2 ) ?
 4n 3  3n 2  O(n 2 )
 4n 3  3n 2  (n 2 )

Abdi M. CoSc3094 Topic: Introduction 27

D&A of Algo.
 Property of Order

1) Transitive:
If f ( n)  O( g (n)) and g (n)  O(h(nthen
)) f (n)  O(h(n))

If f (n)  ( g (n)) and g (n)  (h(nthen

)) f (n)  (h(n))

If f (n)  ( g (n)) and g (n)  (h(nthen

)) f (n)  (h(n))

2) Reflexive:
f (n)  O( f (n))
f (n)  ( f (n))
f (n)  ( f (n))

Abdi M. CoSc3094 Topic: Introduction 28

D&A of Algo.
3) Symmetric:

f (n)  ( g (n)) g (n)  ( f (n))

iff

f (n)  O( g (n)) g (n)  ( f (n))

iff

Abdi M. CoSc3094 Topic: Introduction 29

D&A of Algo.
 Analysis of Algorithms

a) Best Case, Worse Case Analysis

The problem: 
Sort n keys in nondecreasing sequence.
 
Solving strategy:
Insertion sort – insert the ith item into a sorted list of
length (i – 1) by looking at numbers one at a time,  i
>1.
  Example: sort 3,5,4,1
step 1 step 2 step 3 step 4
3 3 3 1
5 5 4 3
  4 4 5 4
Abdi M. CoSc3094 1 1 Topic: Introduction
1 5 30
D&A of Algo.
 Algorithm:
 
A(0) ← -maxint; // for efficient to stop the loop
for i ← 2 to n do
target ← A(i);
j ← i –1;
while (target < A(j))
A( j + 1) ← A(j);
j ← j –1;
A( j + 1) ← target;

Abdi M. CoSc3094 Topic: Introduction 31

D&A of Algo.
 Analysis:
 
Best Case performance is Θ(n) or Ω(n), but not
Ω(n2) since insertion sort runs in Θ(n) when
the input is sorted.

Worst Case performance is Θ(n2) or O(n2)

 

The running time of insertion sort is between Ω(n)

to O(n2)

The running time is bound to O(n2)

Abdi M. CoSc3094 Topic: Introduction 32

D&A of Algo.
b) Average Case Analysis

The problem:

Is the key x in the array S of n keys?

 
Solving strategy: 

Sequential search

Abdi M. CoSc3094 Topic: Introduction 33

D&A of Algo.
Algorithm:

Procedure search ( n, S, x, location) // n is total # of keys,

S is an array,
x is the key,
location is output
parameter
begin
location ← 1;
while (location ≤ n) and (S[location] ≠ x)
location ++;
if location > n location ← 0;
end;

Abdi M. CoSc3094 Topic: Introduction 34

D&A of Algo.
Average Case Analysis:
 
When ‘x is in the array’, we assume that
1
( x  k th ) 
Probability prob 1  k  n
n
1) When x = kth, # of key comparisons = k

1 1 1 1 n
 A(n)   1   2     n   ( k )
n n n n k 1
1 n(n  1) n  1
  
n 2 2

about half array is searched.

Abdi M. CoSc3094 Topic: Introduction 35

D&A of Algo.
2) When ‘x may not be in the array’
th 1
Assume prob (x in array) = p,  prob ( x  k )  p 
n
prob (x not in array) = 1 – p, and it takes
n comparisons to know that ‘is not in the array’
p n
 A(n)    k  (1  p )  n
n k 1
p n(n  1)
   (1  p)  n
n 2
p p
 n  (1  ) 
2 2
Abdi M. CoSc3094 Topic: Introduction 36
D&A of Algo.
 n 1
If p = 1, A(n)  (as calculated in case (1))
2

1 3 1
If p  , A(n)  n  (about ¾ of the array is
2 4 4
searched)

Abdi M. CoSc3094 Topic: Introduction 37

D&A of Algo.
 Design of Algorithms

a) Divide-and-Conquer

b) Greedy

c) Dynamic Programming

d) Backtracking & Branch-and-Bound

Abdi M. CoSc3094 Topic: Introduction 38

D&A of Algo.
 Some mathematical background

Definition. Logb x = y if b y = x b>1

1) Logb is a strictly increasing function,

if x1 < x2 then Logb x1 < Logb x2 .

2) Logb is a one-to-one function,

if Logb x1 = Logb x2 then x1 = x2 .

3) Logb 1 = 0 b

4) Logb xa = a  Logb x

5) Logb(x1  x2) = Logb x1 +Topic:

Abdi M. CoSc3094
Log b x2
Introduction 39
D&A of Algo.
6) x1Logb x2 = x2Logb x1
Log b2 x
Log b1 x 
7) to convert from one base to another, Log b2 b1
n
n(n  1)
8) sum of consecutive integers
i 
i 1 2
k 1
k
a 1
9) geometric sums 
i 0
a i

a 1

10) Suppose f is a continuous, decreasing function, a, b are integers,

b 1 b b

then  f ( x)dx   f (i)   f ( x)dx

a ia a 1

Abdi M. CoSc3094 Topic: Introduction 40

D&A of Algo.
11) Suppose f is a continuous, increasing function, a, b are integers,

b b b 1
then
 f ( x)dx   f (i)   f ( x)dx
a 1 i a a

b
12) 1
a x dx  Log e b  Log e a

Note:
Log2 = Lg
Loge = Ln

Abdi M. CoSc3094 Topic: Introduction 41

D&A of Algo.