Design and Analysis of

Algorithms
CSE 373

L1.1

Lecture 1
Dr. Sifat Momen
What is an Algorithm?
▪An algorithm is a finite set of precise
instructions for performing a
computation or for solving a problem.
▪ It must produce correct result
▪It must finish in some finite time
▪ You can represent an algorithm using pseudocode, flowchart,
or even actual code

L1.
2
Algorithm

input Algorithm output

(optional)

Computational procedure for solving a problem

Algorithm (Brief
timeline: old)
Algorithm(Brief
timeline: New Era)
Algorithm(Brief
timeline: New Era)
Algorithm Description

L1.
7
Algorithm challenge 1

Can you come up with an algorithm to compute the GCD

(Greatest Common Divisor) of two integers.

L1.
8
 Algorithm Challenge 2
You are given all numbers between 1,2,…,n except one. Your task is to find the missing
number.

Input

The first input line contains an integer n.

The second line contains n−1 numbers. Each number is distinct and
between 1 and n (inclusive).

Output

Print the missing number.

Example

Input:
5
2315
L1.
Output: 9
4
Algorithm Challenge 3

Given a sorted array and an integer value (say sum), design an algorithm that will determine if
there exists a pair of integer whose sum is equal to sum.

L1.
10
 What is this course about?

The theoretical study of design and

analysis of computer algorithms
Basic goals for an
algorithm:
• always correct
• always terminates
• performance
⮚ Performance often draws the line
between
what is possible and what is impossible.
L1.
11
Design and Analysis of
Algorithms
• Analysis: predict the cost of an algorithm in
terms of resources and performance

• Design: design algorithms which minimize the

cost

L1.
12
Why designing and
analysis of algorithm is
important?
Example:
Imagine two friends, Alice and Bob are given the
task of writing an algorithm that can sort 10 million
numbers
Alice writes an algorithm that takes 2N2
instructions and implements using computer that
executes 10 billion instructions per second.
Bob writes an algorithm that takes 50NlgN
instructions and implements using computer that
executes only 10 million instructions per second. L1.

Which one runs faster? 13

Why designing and
analysis of algorithm is
important?

L1.
14
The Problem of Sorting
Input: sequence 〈 a1, a2, …, an 〉 of
numbers.
Output: permutation 〈 a'1, a'2, …, a'n 〉
such
that a'1 ≤ a'2 ≤ … ≤ a'n .
Example
: Input: 8 2 4 9 3
6
Output: 2 3 4 6 8
9 L1.
15
Sorting Algorithms

• There are various sorting algorithms

including bubble sort, insertion sort, quick
sort, merge sort, bucket sort, shell sort
etc…
• Can either be a stable or an unstable sorting
algorithm.
• A sorting algorithm is said to be stable if
two objects with equal keys appear in the
same order in sorted output as they
L1.
appear in the input array to be sorted. 16
Stability of a sort

L1.
17
Insertion sort

L1.
18
Example of insertion sort
8 2 4 9 3 6

L1.
19
Example of insertion sort
8 2 4 9 3 6

L1.
20
Example of insertion sort
8 2 4 9 3 6
2 8 4 9 3 6

L1.
21
Example of insertion sort
8 2 4 9 3 6
2 8 4 9 3 6

L1.
22
Example of insertion sort
8 2 4 9 3 6
2 8 4 9 3 6
2 4 8 9 3 6

L1.
23
Example of insertion sort
8 2 4 9 3 6
2 8 4 9 3 6
2 4 8 9 3 6

L1.
24
Example of insertion sort
8 2 4 9 3 6
2 8 4 9 3 6
2 4 8 9 3 6
2 4 8 9 3 6

L1.
25
Example of insertion sort
8 2 4 9 3 6
2 8 4 9 3 6
2 4 8 9 3 6
2 4 8 9 3 6

L1.
26
Example of insertion sort
8 2 4 9 3 6
2 8 4 9 3 6
2 4 8 9 3 6
2 4 8 9 3 6
2 3 4 8 9 6

L1.
27
Example of insertion sort
8 2 4 9 3 6
2 8 4 9 3 6
2 4 8 9 3 6
2 4 8 9 3 6
2 3 4 8 9 6

L1.
28
Example of insertion sort
8 2 4 9 3 6
2 8 4 9 3 6
2 4 8 9 3 6
2 4 8 9 3 6
2 3 4 8 9 6
2 3 4 6 8 9 don
e L1.
29
 C++ Implementation of
Insertion Sort

int main(){
int arr[] = {10, 6, 3, 2, 1, 8};
int l = sizeof(arr)/sizeof(*arr);
print(arr, l);
insertionSort(arr, l);
print(arr, l);

}
L1.
30
 C++ Implementation of
Insertion Sort
void insertionSort(int A[], int length){
int key, i;
for(int j = 1; j < length; j++){
key = A[j];
i = j - 1;
while(i > -1 && A[i] > key){
A[i+1] = A[i];
i = i - 1;
}
A[i+1] = key;
}
L1.
} 31
Print Function

void print(int a[], int length){

for(int i = 0; i < length; i++)
cout << a[i] <<" ";
cout <<endl;
}

L1.
32
Output

L1.
33
 Running time

• The running time depends on the input: an

already sorted sequence is easier to sort.
• Major Simplifying Convention:
Parameterize the running time by the size of
the input, since short sequences are easier to
sort than long ones.
⮚TA(n) = time of A on length n inputs
• Generally, we seek upper bounds on the
running time, to have a guarantee of
performance. L1.
34
Kinds of analyses
Worst-case: (usually)
• T(n) = maximum time of algorithm
on any input of size n.
Average-case: (sometimes)
• T(n) = expected time of algorithm
over all inputs of size n.
• Need assumption of statistical
distribution of inputs.
Best-case: (NEVER)
• Cheat with a slow algorithm that
works fast on some input. L1.
35
 Insertion Sort
INSERTION-SORT (A, n) ⊳ A[1 . . n]
for j ← 2 to n
do key ← A[ j]
i←j–1
while i > 0 and A[i] > key
do A[i+1] ←
A[i]
i←i–1
A[i+1] = key
What is the estimated running time?
Depends on arrangement of numbers in the input array. We are typically
interested in the runtime of an algorithm in the worst case scenario.
Because it provides us a guarantee that the algorithm won’t take any longer than
this for any type of input.

How can you arrange the input numbers so that this

36
algorithm becomes most inefficient (worst case)?
Analyzing Algorithms
• Analyzing an algorithm means predicting the resources
that the algorithm requires.
• Typically time and memory
• Before we can analyze an algorithm, we need a model
of implementation technologies that we will use.
• Random Access Machine Model

37
Random Access Machine
model
• Single Processor
• Instructions are executed one after another, with no
concurrent operations
• The following instructions take a constant amount of
time
• Arithmetic: add, subtract, multiply, divide, remainder,
floor, ceiling etc…
• Data movement: load, store, copy
• Control: conditional/unconditional branch, subroutine call,
return
• Data type: Integer and Float

38
 Insertion Sort: Running
▪ Time

Here tj = no. of times the condition of while loop is tested for the current value of j.
In the worst case (when input is reverse-sorted), in each iteration of the for loop, all the j-1
elements need to be right shifted and the key will be inserted in the front of them, i.e., tj=j.
Using this in the above equation, we get: T(n) = An 2+Bn+C, where A, B, C are constants. 39
What is T(n) in the best case (when the input numbers are already sorted)?
Insertion Sort: Running
Time (Best case)
▪ The best case is when the input is already in the sorted
manner.
▪ Thus tj = 1

This can be expressed as T(n) = an + b, thus

T(n) = O(n) 40
Insertion Sort: Running
Time (Worst case)
▪ The worst case results when the array is in the reverse
order (in this case decreasing order)
▪ In this situation, we must compare each element A[j] with
each element in the entire sorted sub-array A[1… j-1]
▪ This results tj = j

L1.
41
Insertion Sort: Running
Time (Worst case)
▪

L1.
42
Asymptotic Analysis

To compare two algorithms with running times f(n) and

g(n), we need a rough measure that characterizes how
fast each function grows.
Hint: use rate of growth
Compare functions in the limit, that is, asymptotically!
(i.e., for large values of n)
Rate of Growth

Consider the example of buying elephants and goldfish:

Cost: cost_of_elephants + cost_of_goldfish
Cost ~ cost_of_elephants (approximation)
The low order terms, as well as constants in a function are
relatively insignificant for large n
6n + 4 ~ n
n4 + 100n2 + 10n + 50 ~ n4

i.e., we say that n4 + 100n2 + 10n + 50 and n4 have

the same rate of growth
Big-O Notation
Visualizing Orders of
Growth
On a graph, as you go to the right, a faster growing
function eventually becomes larger.

fA(n)=30n+8
Running time→

fB(n)=n2+1

Increasing n
→
Growth of Functions
Complexity Graphs

log(n)
Complexity Graphs

n log(n)

log(n)
Complexity Graphs

n10 n3

n2
n log(n)
Complexity Graphs (log
scale)
3n
nn
n20

2n

n10

1.1n
Asymptotic Notations

O notation: asymptotic “upper bound”:

Ω notation: asymptotic “lower bound”:

Θ notation: asymptotic “tight bound”:

 Asymptotic Notations
▪ O-notation

O(g(n)) is the set of functions

with smaller or same order of
growth as g(n)

Examples:
T(n) = 3n2+10nlgn+8 is O(n2), O(n2lgn), O(n3), O(n4), …
T’(n) = 52n2+3n2lgn+8 is O(n2lgn), O(n3), O(n4), …
 Asymptotic Notations
▪ Ω - notation

Ω(g(n)) is the set of functions

with larger or same order of
growth as g(n)
Examples:
T(n)=3n2+10nlgn+8 is Ω(n2), Ω(nlgn), Ω(n), Ω(nlgn),Ω(1)
T’(n) = 52n2+3n2lgn+8 is Ω(n2lgn), Ω(n2), Ω(n), …
Asymptotic Notations

Θ-notation
Θ(g(n)) is the set of functions with the same
order of growth as g(n)
* f(n) is both O(g(n)) & Ω(g(n)) ↔ f(n) is Θ(g(n))

Examples:
T(n) = 3n2+10nlgn+8 is Θ(n2)
T’(n) = 52n2+3n2lgn+8 is Θ(n2lgn)
Big-O Visualization
Some Examples

Determine the time complexity for the following algorithm.

count = 0;
for(i=0; i<10000; i++)
count++;
Some Examples

Determine the time complexity for the following algorithm.

count = 0;
for(i=0; i<10000; i++)
count++;
Some Examples

Determine the time complexity for the following algorithm.

count = 0;
for(i=0; i<n; i++)
count++;
Some Examples

Determine the time complexity for the following algorithm.

count = 0;
for(i=0; i<n; i++)
count++;
Some Examples

Determine the time complexity for the following algorithm.

sum = 0;
for(i=0; i<n; i++)
for(j=0; j<n; j++)
sum += arr[i][j];
Some Examples

Determine the time complexity for the following algorithm.

sum = 0; O(n2)
for(i=0; i<n; i++)
for(j=0; j<n; j++)
sum += arr[i][j];
Some Examples

Determine the time complexity for the following algorithm.

count = 0;
for(i=1; i<=n; i=i*2)
count++;
 Some Examples
Determine the time complexity for the following algorithm.
count = 0;
for(i=1; i<=n; i=i*2)
count++; O(lg n)
Some Examples

Determine the time complexity for the following algorithm.

sum = 0;
for(i=1; i<n; i=i*2)
for(j=0; j<n; j++)
sum += i*j;
Some Examples

Determine the time complexity for the following algorithm.

sum = 0;
O(n lg n)
for(i=1; i<n; i=i*2)
for(j=0; j<n; j++)
sum += i*j;
Some Examples

Determine the time complexity for the following algorithm.

sum = 0;
for(i=1; i<=n; i=i*4)
for(j=0; j<=n; j*=2)
sum += i*j;
Asympotic Tight Bound: Θ(lg n) WHY?
Some Examples

Determine the time complexity for the following algorithm.

sum = 0;
for(i=1; i<n; i=i*2)
for(j=0; j<i; j++)
sum += i*j;
Some Examples

Determine the time complexity for the following algorithm.

sum = 0;
for(i=1; i<n; i=i*2)
for(j=0; j<i; j++)
sum += i*j;
Loose Upper Bound: O(n lg n)
Tight Upper Bound: O(n) WHY?
Asympotic Tight Bound: Θ(n)
Some Examples

Determine the time complexity for the following algorithm.

char someString[10];
gets(someString);
for(i=0; i<strlen(someString); i++)
someString[i] -= 32;
Some Examples

Determine the time complexity for the following algorithm.

char someString[10];
gets(someString);
for(i=0; i<strlen(someString); i++)
someString[i] -= 32;
Types of Analysis