DS&A- Chapter One

Data Structures and Algorithms Analysis

1. Introduction to Data Structures and Algorithms Analysis

A program is written in order to solve a problem. A solution to a problem actually consists of two
things:
 A way to organize the data
 Sequence of steps to solve the problem
The way data are organized in a computer’s memory is said to be Data Structure and the sequence
of computational steps to solve a problem is said to be an algorithm. Therefore, a program is
nothing but data structures plus algorithms.
1.1. Introduction to Data Structures
Given a problem, the first step to solve the problem is obtaining one’s own abstract view, or model,
of the problem. This process of modeling is called abstraction.

The model defines an abstract view to the problem. This implies that the model focuses only on
problem related stuff and that a programmer tries to define the properties of the problem.

These properties include

 The data which are affected and
 The operations that are involved in the problem
With abstraction you create a well-defined entity that can be properly handled. These entities
define the data structure of the program.

An entity with the properties just described is called an abstract data type (ADT).

1.1.1. Abstract Data Types

An ADT consists of an abstract data structure and operations. Put in other terms, an ADT is an
abstraction of a data structure.
The ADT specifies:
 What can be stored in the Abstract Data Type
 What operations can be done on/by the Abstract Data Type

1
 DS&A- Chapter One

For example, if we are going to model employees of an organization:

 This ADT stores employees with their relevant attributes and discarding irrelevant
attributes.
 This ADT supports hiring, firing, retiring … operations.

A data structure is a language construct that the programmer has defined in order to implement an
abstract data type.

There are lots of formalized and standard Abstract data types such as Stacks, Queues, Trees, etc.

Do all characteristics need to be modeled?

Not at all
 It depends on the scope of the model
 It depends on the reason for developing the model

1.1.2. Abstraction
Abstraction is a process of classifying characteristics as relevant and irrelevant for the particular
purpose at hand and ignoring the irrelevant ones. Applying abstraction correctly is the essence of
successful programming.
How do data structures model the world or some part of the world?
 The value held by a data structure represents some specific characteristic of the world
 The characteristic being modeled restricts the possible values held by a data structure
 The characteristic being modeled restricts the possible operations to be performed on the
data structure.
Note: Notice the relation between characteristic, value, and data structures
Where are algorithms, then?

1.2. Algorithms
An algorithm is a well-defined computational procedure that takes some value or a set of values
as input and produces some value or a set of values as output. Data structures model the static part
of the world. They are unchanging while the world is changing. In order to model the dynamic
part of the world we need to work with algorithms. Algorithms are the dynamic part of a program’s
world model.

An algorithm transforms data structures from one state to another state in two ways:

 An algorithm may change the value held by a data structure

 An algorithm may change the data structure itself

The quality of a data structure is related to its ability to successfully model the characteristics of
the world. Similarly, the quality of an algorithm is related to its ability to successfully simulate the
changes in the world.

However, independent of any particular world model, the quality of data structure and algorithms
is determined by their ability to work together well. Generally speaking, correct data structures

2
 DS&A- Chapter One

lead to simple and efficient algorithms and correct algorithms lead to accurate and efficient data
structures.

1.2.1. Properties of an algorithm

 Finiteness: Algorithm must complete after a finite number of steps.

 Definiteness: Each step must be clearly defined, having one and only one interpretation. At
each point in computation, one should be able to tell exactly what happens next.
 Sequence: Each step must have a unique defined preceding and succeeding step. The first step
(start step) and last step (halt step) must be clearly noted.
 Feasibility: It must be possible to perform each instruction.
 Correctness: It must compute correct answer for all possible legal inputs.
 Language Independence: It must not depend on any one programming language.
 Completeness: It must solve the problem completely.
 Effectiveness: It must be possible to perform each step exactly and in a finite amount of time.
 Efficiency: It must solve with the least amount of computational resources such as time and
space.
 Generality: Algorithm should be valid on all possible inputs.
 Input/Output: There must be a specified number of input values, and one or more result
values.

1.2.2. Algorithm Analysis Concepts

Algorithm analysis refers to the process of determining the amount of computing time and storage
space required by different algorithms. In other words, it’s a process of predicting the resource
requirement of algorithms in a given environment.

In order to solve a problem, there are many possible algorithms. One has to be able to choose the
best algorithm for the problem at hand using some scientific method. To classify some data
structures and algorithms as good, we need precise ways of analyzing them in terms of resource
requirement.

The main resources are:

 Running Time
 Memory Usage
 Communication Bandwidth

Running time is usually treated as the most important since computational time is the most
precious resource in most problem domains.

There are two approaches to measure the efficiency of algorithms:

 Empirical: Programming competing algorithms and trying them on different instances.
 Theoretical: Determining the quantity of resources required mathematically (Execution
time, memory space, etc.) needed by each algorithm.

3
 DS&A- Chapter One

However, it is difficult to use actual clock-time as a consistent measure of an algorithm’s

efficiency, because clock-time can vary based on many things. For example,

 Specific processor speed

 Current processor load
 Specific data for a particular run of the program
o Input Size
o Input Properties
 Operating Environment

Accordingly, we can analyze an algorithm according to the number of operations required, rather
than according to an absolute amount of time involved. This can show how an algorithm’s
efficiency changes according to the size of the input.

1.2.3. Complexity Analysis

Complexity Analysis is the systematic study of the cost of computation, measured either in time
units or in operations performed, or in the amount of storage space required.

The goal is to have a meaningful measure that permits comparison of algorithms independent of
operating platform.
There are two things to consider:
 Time Complexity: Determine the approximate number of operations required to solve a
problem of size n.
 Space Complexity: Determine the approximate memory required to solve a problem of
size n.

Complexity analysis involves two distinct phases:

 Algorithm Analysis: Analysis of the algorithm or data structure to produce a function T
(n) that describes the algorithm in terms of the operations performed in order to measure
the complexity of the algorithm.
 Order of Magnitude Analysis: Analysis of the function T (n) to determine the general
complexity category to which it belongs.

There is no generally accepted set of rules for algorithm analysis. However, an exact count of
operations is commonly used.

1.2.3.1. Analysis Rules:

We assume an arbitrary time unit.
Execution of one of the following operations takes time 1:
 Assignment Operation
 Single Input/output Operation
 Single Boolean Operations
 Single Arithmetic Operations
 Function Return
Running time of a selection statement (if, switch) is the time for the condition evaluation
+ the maximum of the running times for the individual clauses in the selection.

4
 DS&A- Chapter One

Loops: Running time for a loop is equal to the running time for the statements inside the
loop * number of iterations.
The total running time of a statement inside a group of nested loops is the running time of
the statements multiplied by the product of the sizes of all the loops.
For nested loops, analyze inside out.
 Always assume that the loop executes the maximum number of iterations possible.
Running time of a function call is 1 for setup + the time for any parameter calculations +
the time required for the execution of the function body.
Examples:

1. int count(){
int k=0;
cout<< “Enter an integer”;
cin>>n;
for (i=0;i<n;i++)
k=k+1;
return 0;}
Time Units to Compute
-------------------------------------------------
1 for the assignment statement: int k=0
1 for the output statement.
1 for the input statement.
In the for loop:
1 assignment, n+1 tests, and n increments.
n loops of 2 units for an assignment, and an addition.
1 for the return statement.
-------------------------------------------------------------------
T (n)= 1+1+1+(1+n+1+n)+2n+1 = 4n+6 = O(n)
2. int total(int n)
{
int sum=0;
for (int i=1;i<=n;i++)
sum=sum+1;
return sum;
}
Time Units to Compute
-------------------------------------------------
1 for the assignment statement: int sum=0
In the for loop:
1 assignment, n+1 tests, and n increments.
n loops of 2 units for an assignment, and an addition.
1 for the return statement.
-------------------------------------------------------------------
T (n)= 1+ (1+n+1+n)+2n+1 = 4n+4 = O(n)
3. void func()

5
 DS&A- Chapter One

{
int x=0;
int i=0;
int j=1;
cout<< “Enter an Integer value”;
cin>>n;
while (i<n){
x++;
i++;
}
while (j<n)
{
j++;
}
}
Time Units to Compute
-------------------------------------------------
1 for the first assignment statement: x=0;
1 for the second assignment statement: i=0;
1 for the third assignment statement: j=1;
1 for the output statement.
1 for the input statement.
In the first while loop:
n+1 tests
n loops of 2 units for the two increment (addition) operations
In the second while loop:
n tests
n-1 increments
-------------------------------------------------------------------
T (n)= 1+1+1+1+1+n+1+2n+n+n-1 = 5n+5 = O(n)
4. int sum (int n)
{
int partial_sum = 0;
for (int i = 1; i <= n; i++)
partial_sum = partial_sum +(i * i * i);
return partial_sum;
}
Time Units to Compute
-------------------------------------------------
1 for the assignment.
1 assignment, n+1 tests, and n increments.
n loops of 4 units for an assignment, an addition, and two multiplications.
1 for the return statement.
-------------------------------------------------------------------
T (n)= 1+(1+n+1+n)+4n+1 = 6n+4 = O(n)

6
 DS&A- Chapter One

1.2.3.2. Formal Approach to Analysis

In the above examples we have seen that analysis is a bit complex. However, it can be simplified
by using some formal approach in which case we can ignore initializations, loop control, and book
keeping.

for Loops: Formally

 In general, a for loop translates to a summation. The index and bounds of the summation
are the same as the index and bounds of the for loop.
N

1  N
for (int i = 1; i <= N; i++) {
sum = sum+i;
}
i 1
 Suppose we count the number of additions that are done. There is 1 addition per iteration of
the loop, hence N additions in total.

Nested Loops: Formally

 Nested for loops translate into multiple summations, one for each for loop.

for (int i = 1; i <= N; i++) {

for (int j = 1; j <= M; j++) { N M N

}
sum = sum+i+j;   2   2M
i 1 j 1 i 1
 2 MN
}
 Again, count the number of additions. The outer summation is for the outer for loop.

Consecutive Statements: Formally

 Add the running times of the separate blocks of your code
for (int i = 1; i <= N; i++) {
sum = sum+i;
}  N   N N 
1    2  N  2 N
2
for (int i = 1; i <= N; i++) {
for (int j = 1; j <= N; j++) {  i 1   i 1 j 1 
sum = sum+i+j;
}
}

Conditionals: Formally
 If (test) s1 else s2: Compute the maximum of the running time for s1 and s2.
if (test == 1) {
for (int i = 1; i <= N; i++) {  N N N 
sum = sum+i; max 
  1,   2 

}}  i 1 i 1 j 1 
 
else for (int i = 1; i <= N; i++) {
for (int j = 1; j <= N; j++) { max N , 2 N 2
 2N 2
sum = sum+i+j;
}}

7
 DS&A- Chapter One

Example:

Suppose we have hardware capable of executing 106 instructions per second. How long would it
take to execute an algorithm whose complexity function was:
T (n) = 2n2 on an input size of n=108?
The total number of operations to be performed would be T (108):

T(108) = 2*(108)2 =2*1016

The required number of seconds required would be given by
T(108)/106 so:

Running time =2*1016/106 = 2*1010

The number of seconds per day is 86,400 so this is about 231,480 days (634 years).

Exercises
Determine the run time equation and complexity of each of the following code segments.
1. for (i=0;i<n;i++)
for (j=0;j<n; j++)
sum=sum+i+j;

2. for(int i=1; i<=n; i++)

for (int j=1; j<=i; j++)
sum++;
What is the value of the sum if n=20?
3. int k=0;
for (int i=0; i<n; i++)
for (int j=i; j<n; j++)
k++;
What is the value of k when n is equal to 20?
4. int k=0;
for (int i=1; i<n; i*=2)
for(int j=1; j<n; j++)
k++;
What is the value of k when n is equal to 20?

5. int x=0;
for(int i=1;i<n;i=i+5)
x++;
What is the value of x when n=25?
6. int x=0;
for(int k=n;k>=n/3;k=k-5)
x++;
What is the value of x when n=25?

7. int x=0;
for (int i=1; i<n;i=i+5)

8
 DS&A- Chapter One

for (int k=n;k>=n/3;k=k-5)

x++;
What is the value of x when n=25?

8. int x=0;
for(int i=1;i<n;i=i+5)
for(int j=0;j<i;j++)
for(int k=n;k>=n/2;k=k-3)
x++;

What is the correct big-Oh Notation for the above code segment?

1.3. Measures of Times

In order to determine the running time of an algorithm it is possible to define three functions
Tbest(n), Tavg(n) and Tworst(n) as the best, the average and the worst case running time of the
algorithm respectively.

Average Case (Tavg): The amount of time the algorithm takes on an "average" set of inputs.
Worst Case (Tworst): The amount of time the algorithm takes on the worst possible set of inputs.
Best Case (Tbest): The amount of time the algorithm takes on the smallest possible set of inputs.

We are interested in the worst-case time, since it provides a bound for all input – this is called the
“Big-Oh” estimate.

1.4. Asymptotic Analysis

Asymptotic analysis is concerned with how the running time of an algorithm increases with the
size of the input in the limit, as the size of the input increases without bound.
There are three major notations used to describe a running time function. These are:
 Big-Oh Notation (O)
 Omega Notation ()
 Theta Notation (θ)
 Little-o Notation (o)
 Little-Omega Notation (ω)

1.4.1. Big Oh Notation, Ο

The Ο(n) is the formal way to express the upper bound of an algorithm's running time. It measures
the worst case time complexity or longest amount of time an algorithm can possibly take to
complete.

 Big-Oh notation is a way of comparing algorithms and is used for computing the
complexity of algorithms; i.e., the amount of time takes for computer program to run.

9
 DS&A- Chapter One

 It’s only concerned with what happens for very large value of n. Therefore only largest
term in the expression (function) is needed.
 For ex, if the number of operations in an algorithm is n2 – n, n is insignificant compared
to n2 for large values of n. Hence the n term is ignored. Of course, for small values of n,
it may be important.
 However, Big-Oh is mainly concerned with large values of n.
 The notation Ο(n) is a formal way to express the upper bound of an algorithm's running
time.
 It measures worst case time complexity or the longest amount of time an algorithm can
possibly take to complete. Ex...

1.4.2. Big-Omega Notation

 Just as O-notation provides an asymptotic upper bound on a function, Ω notation provides

an asymptotic lower bound.

10
 DS&A- Chapter One

 f(n)= Ω( g (n)) means that f(n) is greater than or equal to some constant multiple of g(n)
for all values of n greater than or equal to some k.

Example: If f(n) =n2, then f(n)= Ω( n)

In simple terms, f(n)= Ω( g (n)) means that the growth rate of f(n) is greater than or equal to g(n).

 The notation Ω(n) is the formal way to express the lower bound of an algorithm's running
time.
 It measures the best case time complexity or the best amount of time an algorithm can
possibly take to complete.

1.4.3. Theta Notation

The θ(n) is the formal way to express both the lower bound and upper bound of an algorithm's
running time. It is represented as following

A function f (n) belongs to the set of Θ (g(n)) if there exist positive constants c1 and c2 such that
it can be sandwiched between c1.g(n) & c2.g(n), for sufficiently large values of n.

11
 DS&A- Chapter One

 If f(n)= Θ (g(n)), then g(n) is an asymptotically tight bound for f(n). In simple terms, f(n)=
Θ (g(n)) means that f(n) and g(n) have the same rate of growth.

1.4.4. Little-o Notation

Big-Oh notation may or may not be asymptotically tight, for example: 2n2 = O(n2)=O(n3)

1.4.5. Little-Omega (ω notation)

 Little-omega (ω) notation is to big-omega (Ω) notation as little-o notation is to Big Oh

notation.
 We use ω notation to denote a lower bound that is not asymptotically tight.

Typical Orders
Here is a table of some typical cases. This uses logarithms to base 2, but these are simply
proportional to logarithms in other base.

N O(1) O(log n) O(n) O(n log n) O(n2) O(n3)

1 1 1 1 1 1 1
2 1 1 2 2 4 8
4 1 2 4 8 16 64
8 1 3 8 24 64 512
16 1 4 16 64 256 4,096
1024 1 10 1,024 10,240 1,048,576 1,073,741,824

12
 DS&A- Chapter One

1.5. Relational Properties of the Asymptotic Notations

Transitivity
 if f(n)=(g(n)) and g(n)= (h(n)) then f(n)=(h(n)),
 if f(n)=O(g(n)) and g(n)= O(h(n)) then f(n)=O(h(n)),
 if f(n)=(g(n)) and g(n)= (h(n)) then f(n)= (h(n)),
Symmetry
 f(n)=(g(n)) if and only if g(n)=(f(n)).
Transpose symmetry
 f(n)=O(g(n)) if and only if g(n)=(f(n)),
 f(n)=o(g(n)) if and only if g(n)=(f(n)).
Reflexivity
 f(n)=(f(n)),
 f(n)=O(f(n)),
 f(n)=(f(n)).

13