Modern Education Society’s

College of Engineering Pune

Fundamentals of Data Structures – VC21

Class : SE COMP(I,II,SS)
Date : 4 Sept 2020 (FRIDAY)

Dr. Archana P. Kale

Mute your microphone and Switch off your camera

Revision
UNIT I
• Abstract Data Type
• Data Structure Classifications
• Introduction to Algorithms
• Characteristics of Algorithm
• Algorithm Design Tools
• Pseudo-code
• Flowchart
• C++
• Decision, Loop
• Function
• Analysis of programming constructs-
UNIT I
• Asymptotic Notation

• Finding Complexity using Step count

Method
• Space Complexity
• S(P)=C+Sp
Revision of last Lecture
UNIT I
• Space Complexity
• S(P)=C+Sp

• Algorithmic Strategies
• Recursive Algorithm
• Backtracking Algorithm
Fundamentals of Data Structures
Introduction to Algorithm and Data Structure

Algorithm
Program
Programming Language
Introduction to Algorithm and Data Structure
Algorithm is a sequence of clear and precise step-by-
step instructions for solving a problem in a finite
amount of time.
Algorithms are implemented by translating the step-
by-step instructions into a computer program that
can be executed by a computer. This translation
process is called computer programming or simply
programming.
Computer programs are constructed using a
programming language appropriate to the
problem.
Introduction to Algorithm and Data Structure
 Data structure is a representation of data and the
operations allowed on that data.

• A data structure is a way to store and organize data in order

to facilitate the access and modifications.

• Data Structures are the method of representing of logical

relationships between individual data elements related to
the solution of a given problem.
Fundamentals of Data Structures
Introduction : From Problem to Program

 Problem
 Solution
 Algorithm
 Data Structure
 Program
Introduction : From Problem to Program

 Problem
 Solution
 Algorithm
 Data Structure
 Program
Algorithm
Algorithm is a sequence of clear and precise step-by-
step instructions for solving a problem in a finite
amount of time.

Problem − Design an algorithm to add two numbers

and display the result.
Data Structure

 Data
 Information
 Knowledge
 Data Structure
Data
Data Structure
Information
Knowledge
Knowledge
Abstract Data Type
 Data Abstraction
 What a data type can do
 How it is done is hidden
 Defines a particular data structure in terms of data
and operations
 An ADT consists of
Definition
Declaration of data
Declaration/List of operations
Basic Data Structure
Data Structure Classification
 Linear and Non Linear
 Static and Dynamic
 Persistent and Ephemeral
 Sequential Access and Direct Access
Linear Data Structure
Non-Linear Data Structure
Data Structure Classification
Static and Dynamic
Sr. Static Data Structure Dynamic Data Structure
No.

1 DS referred to as a static DS that is created at run time is

DS if it is created before called Dynamic DS
program execution begins

2 Array Linked List

Persistent and Ephemeral
Sr. Persistent Data Ephemeral Data
No. Structure Structure

1 DS that supports DS that supports operations

operations on the most only on the most recent
recent version as well as version
the previous version
2 Functional DS (Effect Imperative DS (Effect Full)
free)
3 Stack, Queue Variable in program
Int a=2;
Int a=5;
Cout<<a;
Persistent and Ephemeral
Persistent – Partially and Fully
Persistent DS – Functional
Content Beyond the Syllabus -
Sequential Access and Direct Access
Sr. Sequential Access Direct Access Data
No. Data Structure Structure

1 DS in which to access the DS in which any element

nth element, we must can be accessed without
access the preceding n-1 accessing its predecessor or
elements successor, we can directly
access the nth element
2 Linked List Array
Fundamentals of Data Structures
Algorithm
Algorithm is a sequence of clear and precise step-by-
step instructions for solving a problem in a finite
amount of time.

Problem − Design an algorithm to add two numbers

and display the result.
Characteristics of an Algorithm
 Unambiguous − Algorithm should be clear and unambiguous.
Each of its steps (or phases), and their inputs/outputs should be clear
and must lead to only one meaning.
 Input − An algorithm should have 0ne or more well-defined inputs.
 Output − An algorithm should have one or more well-defined
outputs and should match the desired output.
 Finiteness − Algorithms must terminate after a finite number of
steps.
 Feasibility − Should be feasible with the available resources.
 Independent − An algorithm should have step-by-step directions,
which should be independent of any programming code.
Algorithm Design Tools – Pseudo-code and Flowchart

 Pseudo-code
 Flowchart
Pseudocode
Pseudo code is a method of describing algorithms using a
combination of natural language and programming lang.

It allows the programmer to formulate their thoughts on the

organization and sequence of a computer algorithm without
the need for actually following the exact coding syntax.

Some rules that are frequently followed when writing

pseudocode:

The usual symbols are used for

Arithmetic operations (+, -, *, / , **).
Keywords - PRINT, WRITE, READ, etc.
Flowchart
A flowchart is simply a graphical representation of steps.
It shows steps in sequential order and is widely used in presenting the
flow of algorithms, workflow or processes.
Shows the steps as boxes of various kinds and their order by connecting
them with arrows.
Algorithm
Psudo-code Algorithm

BEGIN
NUMBER s1, s2, sum
OUTPUT("Input number1:")
INPUT s1
OUTPUT("Input number2:")
INPUT s2
sum=s1+s2
OUTPUT sum
END

Python Program

num1 = input('Enter #1 : ')

num2= input('Enter #2 ')
sum=float(num1)+float(num2)
print("SUM:{0} ".format(sum))
Examples
Q.1 Area of circle
Algorithm
Flowchart
Psudo-code

Q.2 Factorial of a given Number

Algorithm
Flowchart
Psudo-code

Q.3 To read any 10 Numbers and Display its total and average
Algorithm
Flowchart
Psudo-code

Q.4 Menu driven program for addition, subtraction, multiplication, division

Algorithm, Flowchart, Psudo-code
Examples
Algorithm Area of Circle
Examples
Factorial of a given Number
Q.3 To read any 10 Numbers and Display its total and
average
Flowchart for
Menu Driven
Examples
Q.1 Addition of two matrix
Algorithm
Flowchart
Psudo-code

Q.2 Fibonacci series of a given Number

Algorithm
Flowchart
Psudo-code

Q.3 Menu driven program for area calculation of circle, rectangle and
triangle
Algorithm
Flowchart
Psudo-code
Fundamentals of Data Structures
Algorithm Analysis

To analyze the proposed solution algorithms and

Implement the best suitable solution.
Complexity of Algorithms
X - Algorithm
N - Size of input data

The time and space implemented by the Algorithm X are the two main factors
which determine the efficiency of X.

Time Factor − The time is calculated or measured by counting the number of

key operations such as comparisons in sorting algorithm.

Space Factor − The space is calculated or measured by counting the maximum

memory space required by the algorithm.

The complexity of an algorithm f(N) provides the running time and / or storage
space needed by the algorithm with respect of N.

Time Complexity
Space Complexity
Time Complexity

Problem- Addition of two number

Time Complexity
The time complexity is the number of operations an algorithm performs to
complete its task with respect to input size.

The algorithm that performs the task in the smallest number of operations is
considered the most efficient one.

Input Size: Input size is defined as total number of elements present in the
input. For a given problem we characterize the input size n appropriately.

For example:
Sorting problem: Total number of item to be sorted
Graph Problem: Total number of vertices and edges
Numerical Problem: Total number of bits needed to represent a number

External Factors – Computing Speed

The number of times a particular statement is being executed with respect to
the input size.
Time Complexity
Time Complexity
Space Complexity
Fundamentals of Data Structures
Asymptotic Notation
 Used to analyze any algorithm and based
on that we find the most efficient algorithm.
 Three Asymptotic Notation

 Why Asymptotic Notation

Time Complexity
The time complexity is the number of operations an algorithm performs to
complete its task with respect to input size (considering that each operation
takes the same amount of time).

The algorithm that performs the task in the smallest number of operations is
considered the most efficient one.

Input Size: Input size is defined as total number of elements present in the
input. For a given problem we characterize the input size n approproately.

Fot example:
Sorting problem: Total number of item to be sorted
Graph Problem: Total number of vertices and edges
Numerical Problem: Total number of bits needed to represent a number

External Factors – Computing Speed

The number of times a particular statement is being executed with respect to
the input size.
Time Complexity
One problem and three algorithms for the same problem

Run all the three algorithms on three different computers, provide same input
and find the time taken by all the three algorithms and choose the one that is
taking the least amount of time.

Run all the three algorithms on the same computer and try to find the time
taken by the algorithm and choose the best.

So, we have seen that we can't judge an algorithm by calculating the time taken
during its execution in a particular system.

We need some standard notation to analyze the algorithm.

Asymptotic Notation
 To analyze any algorithm and based on that we find the most
efficient algorithm.

 Here in Asymptotic notation, we do not consider the system

configuration, rather we consider the order of growth of the
input.

 Order of growth of an algorithm is a way of saying/predicting

how execution time of a program and the space/memory
occupied by it changes with the input size.

 The most famous way is the Big-Oh notation.

 It gives the worst case possibility for an algorithm.
Asymptotic Notation and Order of Growth
Asymptotic Notation and Order of Growth

Plot of Various Functions

Asymptotic Notation and Order of Growth
Asymptotic Notation
Worst

Best

Average
Asymptotic Notation

For x=-2,2,1 evaluate f(x)

Fundamentals of Data Structures
Analysis of programming constructs-
linear quadratic cubic logarithmic
Algorithmic Complexity also called "Running Time" or
"Order of Growth"

Refers to the number of steps a program takes as a

function of the size of its inputs.
Analysis of programming constructs-
linear quadratic cubic logarithmic
Here is a list of some common running times:
Constant O(1)

Logarithmic O(log n)

Linear O(n)

Quadratic O(n2)

Cubic O(n3)

Exponential O(2n)
Analysis of programming constructs-
linear quadratic cubic logarithmic
Constant-Time Algorithms - O(1)
A constant-time algorithm is one that takes the same
amount of time, regardless of its input.

●Given two numbers, report the sum

●Given a list, report the first element
Analysis of programming constructs-
linear quadratic cubic logarithmic
Logarithmic-Time Algorithm - O(log n)
A logarithmic-time algorithm is one that requires a number of steps
proportional to the log(n).

In most cases, we use 2 as the base of the log, but it doesn't matter
which base because we ignore constants.

●Binary search
●Searching a tree data structure
Analysis of programming constructs-
linear quadratic cubic logarithmic

Linear-Time Algorithms - O(n)

A linear-time algorithm is one that takes a number of steps directly
proportional to the size of the input.

In other words, if the size of the input doubles, the number of

steps doubles.

●Given a list of words, say each item of a list

●Given a list of numbers, add each pair of numbers together (item
1 + item 2, item 3 + item 4, etc.)
●Given a list of numbers, multiply every 3rd number by 2
Analysis of programming constructs-
linear quadratic cubic logarithmic
Quadratic-Time Algorithms - O(n2)
A quadratic-time algorithm is one takes a number of steps
proportional to n2.

If the size of the input doubles, the number of steps quadruples.

A typical pattern of quadratic-time algorithms is performing a

linear-time operation on each item of the input
(n steps per item * n items = n2steps).

● Compare each item of a list against all the other items in the list
● Fill in a n-by-n game board
Analysis of programming constructs-
linear quadratic cubic logarithmic
Cubic-Time Algorithms - O(n3)
A cubic-time algorithm is one that takes a number of steps proportional to n3.

In other words, if the input doubles, the number of steps is multiplied by 8.

Similarly to the quadratic case, this could be the result of applying an n2

algorithm to n items, or applying a linear algorithm to n2 items.

●Fill in a 3D board (or environment)

●For each object in a list, construct an n-by-n bitmap drawing of the object
Analysis of programming constructs-
linear quadratic cubic logarithmic
Exponential-Time Algorithms - O(2n)
An exponential-time algorithm is one that takes time proportional to 2n.

In other words, if the size of the input increases by one, the number of steps
doubles.

Note that logarithms and exponents are inverses of each other.

Algorithms in this category are often considered too slow to be practical,

especially if the input is typically large.

●Given a number n, generate a list of every n-bit binary number

Analysis of programming constructs-
linear quadratic cubic logarithmic
Growth rate
Asymptotic Notation
Worst

Best

Average
Big-O Notation (O-notation)
Represents the upper bound of the running time of an algorithm.
Thus, it gives the worst case complexity of an algorithm.
Definition: f(n) = O(g(n)) (read as “f(n) is Big Oh of g(n)”) iff positive constants
c and n0 exist such that f(n) ≤ cg(n) for all n, n ≥ n0.
Asymptotic Notation
Big-O Notation (O-notation)

The above expression can be described as a function

f(n) belongs to the set O(g(n)) if there exists a positive constant c such that it
lies between 0 and cg(n), for sufficiently large n.

For any value of n, the running time of an algorithm does not cross time
provided by O(g(n)).

Since it gives the worst case running time of an algorithm, it is widely used to
analyze an algorithm as we are always interested in the worst case scenario.
Big-O Notation (O-notation)
f(n) = n2+2n+1
f(n) = O g(n)
f(n) <= c g(n)

n2+2n+1 <= c g(?)

n2+2n+1 <= c g(n2)
n2+2n+1 <= 4n2

Proof put n=4

25<=64

c>0, n>=n0 , n0>=0

Omega Notation (Ω-notation)
Represents the lower bound of the running time of an algorithm.
Thus, it provides best case complexity of an algorithm.
Definition: f(n) = Ω(g(n)) (read as “f(n) is omega of g(n)”) if positive constants c
and n0 exist such that f(n) ≥ cg(n) for all n, n ≥ n0 −eeAsymptoticNotatin
Omega Notation (Ω-notation)

Omega gives the lower bound of a function

The above expression can be described as a function

f(n) belongs to the set Ω(g(n)) if there exists a positive constant c such that it
lies above cg(n), for sufficiently large n.

For any value of n, the minimum time required by the algorithm is given by
Omega Ω(g(n)).
Theta Notation, θ
Represents the upper and the lower bound of the running time of an algorithm
Used for analyzing the average case complexity of an algorithm
Definition: f(n) = Q(g(n)) (read as “f(n) is theta of g(n)”) if positive constants c1,
c2 and n0 exist such that c1g(n) ≤ f(n) ≤ c2g(n) for all n, n ≥ n0.
eeAsymptotic Notation
Theta Notation, θ
For a function g(n), Θ(g(n)) is given by the relation:

The above expression can be described as a function

f(n) belongs to the set Θ(g(n)) if there exist positive constants c1 and c2 such
that it can be between c1g(n) and c2g(n), for sufficiently large n.

If a function f(n) lies anywhere in between c1g(n) and c2 > g(n) for all n ≥ n0,
then f(n) is said to be asymptotically tight bound.
Big-O Notation (O-notation)

Let f(n) = 7n+ 8 and g(n) =n. Is f(n) ∈ O(g(n)) ?

Big-O Notation (O-notation)

Let f(n) = 7n+ 2 and g(n) =n. Is f(n) ∈ O(g(n)) ?

7n+2≤_________

O(?)
Omega Notation (Ω-notation)

Prove f(n) = Ω(n)

f(n) g(n) 0<=c.g(n)<=f(n) f(n)=Ω(n) C

n2 n 0<=c.n<=n2 n2=Ω(n)

n2 n2 0<=c. n2<=n2 n2=Ω(n2)

n2 n2/2 0<=c. n2/2<=n2 n2=Ω(n2/2)

n2/2 n2 0<=c. n2<=n2/2 n2/2=Ω(n2)

Asymptotic Notation
Asymptotic Notation
Fundamentals of Data Structures
Finding Complexity using Step
count Method
Time complexity

Need to calculate cost and frequency for every step

Frequency - Count number of times one instruction is

executing
Analysis of Algorithms
Finding Complexity using Step
count Method
Cost Frequency Total Cost
C1 1 C1
C2 1 C2
C3 1 C3
C4 1 C4

Total Cost=C1+C2+C3+C4
=O(1)
Finding Complexity using Step
count Method
Cost Frequency Total Cost

Total Cost=
Finding Complexity using Step
count Method
Cost Frequency Total Cost

C1 1 C1

C2 1 C2

C3 1 C3

Total Cost=C1+C2+C3
O(1)
Finding Complexity using Step
count Method

Total cost = C1+c2(n+1)+c3n+c4+c5

=c1+c2n+c2+c3n+c4+c5
=O(n)
Finding Complexity using Step
count Method
Finding Complexity using Step
count Method
Finding Complexity using Step
count Method
Finding Complexity using Step
count Method
Finding Complexity using Step
count Method
Finding Complexity using Step
count Method
Finding Complexity using Step
count Method
Cost Frequency Total Cost
Finding Complexity using Step
count Method Cost Frequency Total Cost
C1 1 C1
C2 n+1 C2(n+1)
C3 n nC3
C4 n nC4
C5 n nC5

Total Cost = C1 + (n+1)C2 + nC3 + nC4 + nC5 + 1

= (C2 + C3 + C4 + C5)n + (C1 + C2 + 1) = An + B,
=O(n)
where
A = C2 + C3 + C4 + C5
B = C1 + C2 + 1 are both constants.
Finding Complexity using Step
count Method
Finding Complexity using Step
count Method
Finding Complexity using Step
count Method
Finding Complexity using Step
count Method

Total Cost = O(n^2)

Finding Complexity using Step
count Method
Finding Complexity using Step
count Method
Finding Complexity using Step
count Method
Finding Complexity using Step
count Method
Finding Complexity using Step
count Method
Math Background : Exponent
Analysis of programming constructs-
linear quadratic cubic logarithmic
Fundamentals of Data Structures
Space Complexity
Space complexity is the amount of memory used by the
algorithm (including the input values to the algorithm) to
execute and produce the result

Whenever a solution to a problem is written some memory

is required to complete.

For any algorithm memory may be used for the following:

1. Variables (This include the constant values, temporary
values)
2. Program Instruction
3. Execution
.
Space Complexity
Space Complexity
Space Complexity
{
int z = a + b + c;
return(z);
}

S(P)=C+Sp

C – Fixed part(needed for instruction space, variable and

constant space)
Sp- Variable Part(dependent on a particular instance of
input and output data)
Space Complexity
{
int z = a + b + c;
return(z);
}

Variables a, b, c and z are all integer types

Total memory requirement will be (4(4) + 4) = 20 bytes

Additional 4 bytes is for return value.

And because this space requirement is fixed for the above

example, hence it is called Constant Space Complexity.
Space Complexity
// n is the length of array a[]

int sum(int a[], int n)

{
int x = 0;
for(int i = 0; i < n; i++)
{
x = x + a[i];
}
return(x);
}
Space Complexity
// n is the length of array a[]

int sum(int a[], int n)

{
int x = 0; // 4 bytes for x
for(int i = 0; i < n; i++) // 4 bytes for i
{
x = x + a[i];
}
return(x);
}
Space Complexity
// n is the length of array a[]

int sum(int a[], int n)

{
int x = 0;
for(int i = 0; i < n; i++)
{
x = x + a[i];
}
return(x);
}
Space Complexity
// n is the length of array a[]

int sum(int a[], int n)

{
int x = 0; // 4 bytes for x
for(int i = 0; i < n; i++) // 4 bytes for i
{
x = x + a[i];
}
return(x);
}
Space Complexity
4*n bytes of space is required for the array a[].

4 bytes each for x, n, i and the return value.

Total memory requirement - (12 + 4n)

which is increasing linearly with the increase in the

input value n, hence it is called as Linear Space
Complexity.
Similarly, we can have quadratic and other complex space complexity as
well, as the complexity of an algorithm increases.
Fundamentals of Data Structures
Algorithmic Strategies
Approach to solving a problem
May combine several approaches

Algorithm structure
–Iterative (execute action in loop)
–Recursive

Problem type
Satisfying
Optimization
Algorithmic Strategies
Algorithmic Strategies
•Recursive algorithms
•Backtracking algorithms
•Divide and conquer algorithms
•Dynamic programming algorithms
•Greedy algorithms
•Brute force algorithms
•Branch and bound algorithms
•Heuristic algorithms
Algorithmic Strategies -Recursive
algorithms
Algorithmic Strategies -Recursive algorithms
Algorithmic Strategies
Backtracking algorithms
Algorithmic Strategies
Backtracking algorithms
N - Queens problem is to place n - queens in such a manner on an n x n
chessboard that no queens attack each other by being in the same row, column
or diagonal.
Algorithmic Strategies
4-Queens Problem
Algorithmic Strategies
•Backtracking algorithms
Algorithmic Strategies
•Backtracking algorithms
8 Queens Problem
Algorithmic Strategies
Algorithmic Strategies
•Recursive algorithms
•Backtracking algorithms
•Divide and conquer algorithms
•Dynamic programming algorithms
•Greedy algorithms
•Brute force algorithms
•Branch and bound algorithms
•Heuristic algorithms
Algorithmic Strategies - Divide and
conquer
Algorithmic Strategies - Divide and
conquer
Algorithmic Strategies - Divide and
conquer
Algorithmic Strategies - Divide and
conquer
Merge Sort
Fundamentals of Data Structures
Algorithmic Strategies – Greedy
Algorithm
 For most optimization problems you want
to find, not just a solution, but the best
solution.
 A greedy algorithm sometimes works
well for optimization problems. It works in
phases. At each phase:
 Take the best you can get right now, without
regard for future consequences.
 Hope that by choosing a local optimum at each
step, you will end up at a global optimum.
Algorithmic Strategies – Greedy
Algorithm
 Suppose you want to count out a certain
amount of money, using the fewest
possible coins
 A greedy algorithm to do this would be:
At each step, take the largest possible
coin
 Example: To make 12 RS, you can choose:
 10 RS Coin, 5 RS Coin, 2 RS Coin, 1 Rs Coin
Algorithmic Strategies
•Recursive algorithms
•Backtracking algorithms
•Divide and conquer algorithms
•Dynamic programming algorithms
•Greedy algorithms
•Brute force algorithms
•Branch and bound algorithms
•Heuristic algorithms
Fundamentals of Data Structures
Case Study
 Multiplication Technique by the
mathematician Carl Friedrich Gauss and
Karatsuba Algorithm for Fast
Multiplication
Case Study
 Multiplication Technique
Case Study
 Multiplication Technique by Divide and
Conquer
Case Study
 Multiplication Technique by Divide and
Conquer
Case Study
 Multiplication Technique by Divide and
Conquer
 x0*y0, x0*y1, x0*y0, x0*y1,
Case Study
 Multiplication Technique by Karatsuba
Algorithm for Fast Multiplication
Fundamentals of Data Structures
UNIT II
 Thank You