Data Structure and

Algorithms
Roles of Data Structures
What is Data Structure ?

Data Structure is a specialized format for organizing, processing, retrieving and storing
data.

Why is the study of Data Structure ?

Computer science is all about storing and computing from a given data. So studying data
structures helps to deal with different ways of arranging, processing and storing data.

2
Choice of Data Model
• Data must be rich enough to mirror the actual relationships of the data in real world.
• Structure should be simple enough to process effectively when data is necessary.

Classification of data structure

• Primitive and Non-Primitive

• Homogeneous and Heterogeneous
• Static and Dynamic
• Linear and Non-Linear

3
Classification of Data Structure
Primitive and Non-Primitive
• Primitive : basic data structure which are directly operated upon machine instructions.
• Non-Primitive : data structure is derived from primitive data structure . Eg. Array

Homogeneous and Heterogeneous

• Homogeneous : all the elements in data structures are the same type.
• Heterogeneous : all the elements in data structures are different types.

Static and Dynamic

• Static : data structure which is allocated at compile time. Eg. Arrays
• Dynamic : data structure which is allocated at runtime . Eg. LinkedList, Vector
4
Classification of Data Structure ( Cont’d.)
Linear and Non-Linear

• Linear : data structure has linear relationship between its elements and each element has
unique predecessor and successor. Eg. Array

• Non-Linear : data structure which maintain hierarchal relationship between elements. Eg. Tree

Types of Data Structures

• Arrays • Hash Tables
• Linked List • Stack
• Tree • Queues
• Graph
5
Arrays
• Array in data structure is for storing more than one data which is similar data types.

• The items of an array are allocated at adjacent memory locations.

Real World Example

• One dimensional Array

Board of Game to store score and arrange in ascending or
descending order

• Two- Dimensional Array

Image Processing, Speech Processing, QR code

6
Linked List
• Linked list is a linear data structure in which elements are not stored at adjacent memory
location.
• The elements in a linked list are linked using pointers.

Real World Example

• Web Pages, Music player, Image Viewer

7
 Linked List Example
Consider the linear array Name. The values of first and Link[k] determines a linear ordering of the names. First gives the
location of the first name in the list, link[k] gives the location of the name following name[k], with 0 denoting the end of the
list. Find the linear ordering of the names.
Name Link
1 Rogers 7
First 2 Clark 8
3
5
4 Hansen 10
5 Brooks 2
6 Pitt 1
The Linear ordering of the names is 7 Walker 0
8 Fisher 4
Brooks, Clark, Fisher, Hansen, Leary, Pitt, Rogers, 9
Walker,
10 Leary 6
8
Trees
• Tree is a non-linear data structure which has roots, branches and leaves.
• Tree structure is in hierarchical relationships. Eg. Family Tree

Real World Example

• Database , Domain Name Server , Decision Based Algorithm in

Machine Learning

9
Tree Example
Consider the algebraic expression ( 7x + y ) ( 5a – b ) 3. Draw the corresponding tree diagram.

+
- 3
* y

* b
7 x

5 a 10
Practice Exercises for Tree
*
Consider the algebraic expressions
(a) ( 4x2+ 5x+7 ) ( 3x2-8x+12)
+ +
(b) (4xy2)2(5x2y3)3

+ 7 - 12
Draw the corresponding tree diagram.
* *
* *

5 x 3 ^ 8 x
4 ^

x 2 x 2
11
 (4xy2)2(5x2y3)3

(4xy2)2 (5x2y3)3

^ ^

4xy2 2 5x2y3 2

* *

4x y2 5x2 y3

12
 Classification of Trees
General Tree
• A tree is called general tree in
Binary Tree
which every node can has infinite
numbers of children. • A tree in which each node can
have at most two children known
as left and right kid.

13
Classification of Trees (Cont’d.)

Binary Search Tree AVL Tree

• Binary Search Tree : A tree in • AVL tree : A tree which is based on balancing. The
height of the kid nodes is at most 1. In AVL tree,
which left child value of anode is the correct balance factor is 1,0 and -1. If the tree
less than or equal to the parent has new node, it will be rotated to ensure that the
tree is balanced.
vale and the right child value is
greater than or equal to the parent
value.

14
Graphs
• A graph is non-linear data structure consisting of nodes and edges and each node have any
number of edges.
• There is no unique node called root and a cycle can be formed.

Real World Example

• Finding shortest path in networking, GPS navigation

15
Graph Example

The set of vertices is V = {v1, v2, v3, v4, v5, v6, v7}

The set of edges is E = {{v1, v2}, {v1, v4}, {v2, v3}, {v2,

v4}, {v2, v6}, {v3, v4},{v4, v6}, {v5, v6}, {v6, v7}}

16
Graph Example

Adjacency Matrix
• To represent the graph.
• The adjacency matrix is built by putting 1 at line i
The adjacency matrix of the graph
and column j if {i,j} is an edge, and putting 0
otherwise. Note that it is always symmetric.
17
Graph Example
v1 v2 v4

v2 v1 v3 v4 v6

v3 v2 v4

v4 v1 v2 v3

v5 v6

v6 v2 v4 v5 v7

v7 v6

The adjacency List of the graph 18

 Practice for Graph
Draw the adjacency matrix and list of the following graphs

A B x1 x2

x5
D C

x3 x4
E

19
Graph’s Path
• A path is an ordered sequence of edges which is obtained by joining
two consecutive vertices.
• The length of a path is the length of the sequence of edges.
• In a path, an edge cannot appear twice.
• The leave vertices of the sequence are called the extremities of the
graph.

20
Connected and Not-Connected Graph
• A graph is called connected graph where there is a path between every
pair of vertices.
• A graph is disconnected ( not connected ) if at least two vertices of the
graph are not connected by a path.

Connected graph Not connected graph

21
Graph’s Circles
• A circle in a graph is a path in which the beginning vertex and the
ending vertex are the same.

22
Exercise
The daily flights of an airline company appears in the following figure. CITY lists the cities, and ORIG[K] and DEST[K] denote the cities of
origin and destination, respectively of the flight NUMBER[K]. Draw the corresponding directed graph of the data. ( The graph is directed
because the flight numbers represent flights from one city to another but not returning.)
NUMBER ORIG DEST
1 701 2 3
2 702 3 2
CITY 3 705 5 3
1 Atlanta 4 708 3 4
2 Boston 5 711 2 5
3 Chicago 6 712 5 2
4 Miami 7 713 5 1
5 Philadelphia 8 715 1 4
9 717 5 4
10 718 4 5 23
Exercise Answer

24
Hash Tables
• In hash tables, data is stored in array format where each data value has its own unique
index value.
• It becomes very first in insertion and search operations.

Real World Example

• Google search

25
Stack
• Stack is a linear data structure in which insertion and deletion of elements takes place at
one end called top of the stack.
• Stack is Last In First Out (LIFO) data structure which means element inserted last will be
removed first.

Real World Example

• Undo operations, syntax in languages, virtual machine like

JVM

26
Stack Example

27
Stack Exercises
A letter means push and an asterisk means pop in the following
sequence. Give the sequence of values returned by the pop operations
when this sequence of operations is performed on an initially empty
LIFO stack.

E A S * Y * Q U E * * * S T * * * I O * N * * *

L A * S T I * N * F I R * S T * * O U * T * * * * * *

28
Queue
• Queue is a linear data structure in which first element is inserted form one end called Rear
and removes the element at the other end called Front.
• Queue is First In First Out (FIFO) data structure which means element inserted first will be
removed first.

Real World Example

• Operating systems for job scheduling, network congestion

29
Queue Example

30
Queue Exercise
A letter means put and an asterisk means get in the following
sequence. Give the sequence of values returned by the get
operation when this sequence of operations is performed on an
initially empty FIFO queue.

E A S * Y * Q U E * * * S T * * * I O * N * * *

31
32
Binary Tree Transversal

Inorder Left Root Right A

B C

Preorder Root Left Right

D E F

Postorder Left Right Root

G H I

Inorder : B D A G E C H F I

Preorder : A B D C E G F H I

Postorder: D B G E H I F C A 33
Binary Tree Transversal Exercises
Write the inorder, preorder and postorder transversal for this tree.

14

53
11

60
19
7 12

17 20
4 8 13

16

34
Inorder

14

53
11

60
19
7 12

17 20
4 8 13

16

35
Pre Order

14

53
11

60
19
7 12

17 20
4 8 13

16

36
 14

53
11

60
19
7 12

17 20
4 8 13

16

37
Write the inorder, preorder and postorder
transversal for this tree

38
 Arithmetic Expression Polish Notation using Pre-order
(A+B*C)/(D-E-F)

/
/

+ -
A+B*C D-E-F

A * - F
+ -

A B*C D-E F B C D E

* -

B C D E /+A*BC--DEF
39
Arithmetic Expression Polish Notation using Post-
order
(A+B*C)/(D-E-F)
/

-
+ ABC*+DE-F -/

- F
A *

B C D E

40
Exercises
• Convert the following arithmetic expression to Pre-order and Post-
order notation using tree structure.
• (A+(B/C)*(D+E)-F)
-

+ F

A *

/ + ABC/DE+*+F-

B C D E 41
Infix and Postfix Conversation
Infix=A+B
Postfix=AB+
Operands :A,B,X,Y,P,Q
Operators :+,-,/,*,(,)

• Priority of Operator
Rule 1 • () ^ */ + -

• No two operator of same priority can stay

Rule 2 together in stack column

Rule 3 • Lowest priority cannot be placed before

highest priority
42
 Place Operand to Postfix
Operator to Stack

Infix and Postfix Conversation Examples (A+B/C*(D+E)-F)

Symbol Stack Postfix (Output) Symbol Stack Postfix
( ( ) (+*(+) ABC/DE
A ( A (+* ABC/DE+
+ (+ A - (+ ABC/DE+*
B (+ AB ( ABC/DE+*+
/ (+/ AB (- ABC/DE+*+
C (+/ ABC F (- ABC/DE+*+F

* (+ ABC/ ) (-) ABC/DE+*+F

ABC/DE+*+F-
(+* ABC/
( (+*( ABC/
D (+*( ABC/D
+ (+*(+ ABC/D
E (+*(+ ABC/DE
ANS : ABC/DE+*+F- 43
 (A+B/C*(D+E)-F)
Symbol Stack Postfix Symbol Stack Postfix
( ( - (+ ABC/DE+*
A ( A ( ABC/DE+*+
+ (+ A (- ABC/DE+*+
B (+ AB F (- ABC/DE+*+F
/ (+/ AB ) (-) ABC/DE+*+F
C (+/ ABC ABC/DE+*+F-
* (+ ABC/
(+* ABC/
( (+*( ABC/
D (+*( ABC/D
+ (+*(+ ABC/D
E (+*(+ ABC/DE
) (+*(+) ABC/DE
(+* ABC/DE+ 44
 (A+B/C*(D+E)-F)

Symbol Stack Postfix

( (
Symbol Stack Postfix
A ( A
- (+ ABC/DE+*
+ (+ A
( ABC/DE+*+
B (+ AB
(- ABC/DE+*+
/ (+/ AB
F (- ABC/DE+*+F
C (+/ ABC
) (-) ABC/DE+*+F
* (+ ABC/
ABC/DE+*+F-
(+* ABC/
( (+*( ABC/
D (+*( ABC/D
+ (+*(+ ABC/D
E (+*(+ ABC/DE
) (+*(+) ABC/DE
(+* ABC/DE+ 45
 Symbol Stack Postfix
X X
= = X
X=(A+B)×(C - ( =( X
D÷E)
A =( XA
+ =(+ XA
B =(+ XAB
) =(+) XAB
= XAB+
* =* XAB+
( =*( XAB+
C =*( XAB+C
- =*(- XAB+C
D =*(- XAB+CD
/ =*(-/ XAB+CD
E =*(-/ XAB+CDE
) =*(-/) XAB+CDE/-
= XAB+CDE/-* 46
 Place Operand to Postfix
Operator to Stack

Infix and Postfix Conversation Examples (10+6/2*(9+10)-12)

Symbol Stack Postfix Symbol Stack Postfix
( ( - (+* 10 6 2 / 9 10 + *
10 ( 10 (- 10 6 2 / 9 10 + *+
+ (+ 10 12 (- 10 6 2 / 9 10 + *+12
6 (+ 10 6
) (-) 10 6 2 / 9 10 + *+12-
/ (+/ 10 6
2 (+/ 10 6 2
* (+* 10 6 2/
( (+*( 10 6 2 /
9 (+*( 10 6 2 / 9
+ (+*(+ 10 6 2 / 9
10 (+*(+ 10 6 2 / 9 10
) (+*(+) 10 6 2 / 9 10 +
47
AVL Tree Example
An AVL tree is a balanced binary search tree. In an AVL tree, balance factor of every node
is either -1, 0 or +1

Balance factor = heightOfLeftSubtree - heightOfRightSubtree

48
AVL Tree Single Left Rotation (LL Rotation)

In LL Rotation, every node moves one position to left from the current position.

49
AVL Tree Single Right Rotation (RR Rotation)

Every node moves one position to right from the current position.

50
AVL Tree Right Left Rotation (RL Rotation)
The RL Rotation is sequence of single right rotation followed by single left rotation.
In RL Rotation, at first every node moves one position to right and one position to left from the current
position

51
AVL Tree Left Right Rotation (LR Rotation)
The LR Rotation is a sequence of single left rotation followed by a single right rotation.
In LR Rotation, at first, every node moves one position to the left and one position to right from the current
position.

52
Example: Construct an AVL Tree by inserting numbers from 1 to 8.

53
Example: Construct an AVL Tree by inserting numbers from 1 to 8.

54
Example: Construct an AVL Tree by inserting numbers from 1 to 8.

55
Example: Construct an AVL Tree by inserting numbers from 1 to 8.

56
Example: Construct an AVL Tree by inserting numbers from 1 to 8.

57
Practice Exercises for AVL Tree

Construct the AVL tree by Inserting the following data

14,17,11,7,53,4,13,12,8,60,19,16,20

58
Data Structure Operations
• Traversing : Accessing each record exactly once so that certain items in the record may be
processed.
• Searching : Finding the location of the record with a given key value or finding the
locations of all records which satisfy one or more conditions.
• Inserting : Adding a new record to the structure.
• Deleting : Removing the record from the structure.

The two special situations for data structures:

• Sorting : Arranging the records in some logical order ( eg. Alphabetically )
• Merging : Combining the records in two different sorted files into a single sorted file.

59
Data Structure Operations
Sorting Algorithms
Traversing and Searching Algorithms
• • Bubble Sort
Sequential Search ( Linear Search )
• Binary Search • Selection Sort
• Depth-First Search ( in Tree data Structure ) • Insertion Sort
• Breadth-First Search • Merge Sort
• Traveling Sales Man Problem ( in Graph Data Structure )
• Quick Sort
• Knapsack Problem
• Assignment Problem • Heap Sort

60
Roles of Algorithms in Computing
What is Algorithm?

An algorithm is any well-defined computational procedure that takes some value, or set of
values, as input and produces some value, or set of values as output.

Why is the study of Algoirthms?

• Cause the computers has limitations like computational time, memories and so on, we need to
use the computer resources at least as possible and to give the best solutions of the problem.
• Therefore, an algorithm becomes a tool for solving a well-specified computational problem and
need to learn how algorithms works.
61
Types of Algorithms
Algorithms can be classified into the following types:

• Burte Force Algorithms

• Decrease and conquer Algorithms
• Divide and Conquer Algorithms
• Transform and Conquer Algorithms
• Backtracking Algorithms
• Dynamic Programming Algorithms ( knapsack, binary search tree, Warshall and Floyd’s algorithms )
• Greedy Algorithms ( minimum spanning tree, prim algorithm, Kruskal’s algorithm, Dijkstra’s Algorithm )

62
Types of Algorithms
Dynamic Programming Greedy Technique

At each step, the choice is At each step, we quickly make a choice

determined based on solutions of that currently looks best.
sub problems. --A local optimal (greedy) choice.

Sub-problems are solved first Greedy choice can be made first before
solving further sub-problems.

Bottom-up approach Top-down approach

Can be slower, more complex Usually faster, simpler

63
Searching Algorithms
Linear Search

You can look how sequential search and binary search in the following link

https://www.cs.usfca.edu/~galles/visualization/Search.html
64
Searching Algorithms
Binary Search

65
Binary Search Noted

Sorted Find Item Start

End Mid
66
Searching Algorithms
0 1 2 3 4 5 6 7 8 9 10 11 12
A K=70 L=0 R=12

n=13 M=
70>55
L=m+1=7 R=12

M=

L=7 R=m-1=8-1=7
70<74
M=
Return 67
m=7
Searching Algorithms
Find K =35 in the following sequences using Binary Search Algorithms.
93,79,85,64,81,85,83,63,79,73,27,5,8,14,35,25

93,79,85,64,81,85,83,63,79,73,27,5,8,14,35,25

68
• For the list D, H, O, L, A, E, I, M, B, J, N, C, G, K and searching for
the value J using binary search and also search Z.

69
Depth-First Search ( Stack Manner)
• Depth-first search starts a graph at an arbitrary vertex. On each iteration, the algorithm
proceeds to an unvisited vertex that is adjacent to the current one.
• The process continues until a dead end ( a vertex with no adjacent unvisited vertices) is
encountered.
• At a dead end, the algorithm backs up one edge to the vertex it came from and tries to
continue visiting unvisited vertices from there.

70
Depth-First Search ( Stack Manner)

71
 Depth-First Search
a g
Push order

c h A,C,D,F,B,E,G,H,I, J

f Pop order
d i
D,E,B,F,C,A,J,I,H,G
b
A B C D E F
j
G 7,10
insert 1 5 2 3 6 4 e
A 1,6 H 8,9
F 4,4
C 2,5 I 9,8
delete 6 3 5 1 2 4 B 5,3
D 3,1 J 10,7
E 6,2 72
Depth First Search Algorithm Example
12
4
2
1

0 3 5
8

9
7 6

10 11

73
Searching the path using Breath First Search and depth first algorithms.

74
Breadth-First Search ( Queue Manner)
• Breadth-first search starts a graph at an arbitrary vertex. On each iteration, the algorithm
visits first all the vertices that are adjacent to a starting vertex, then all unvisited vertices
two edges apart from it and so on.
• The process continues until all the vertices in the same connected component as the starting
vertex are visited.
• If there remain unvisited vertices, the algorithm has to be restarted at an arbitrary of another
connected component of the graph.

75
76
77
78
5
Breadth-First Search ( Queue Manner)
4
2
12 2 4
1 12
1
6 5
3
3 8
8 10
7 6
10
9
11
11
7 0
9
0
79
 Breadth-First Search ( Queue Manner)
BFS (G, s) //Where G is the graph and s is the source node
let Q be queue.
Q.enqueue( s ) //Inserting s in queue until all its neighbor vertices are marked.
mark s as visited.
while ( Q is not empty)
//Removing that vertex from queue ,whose neighbor will be visited now
v = Q.dequeue( )
//processing all the neighbors of v
for all neighbors w of v in Graph G
if w is not visited
Q.enqueue( w ) //Stores w in Q to further visit its neighbour
mark w as visited.
80
 Breadth-First Search
a g

c d e h j

f b i

A B C D E F A1 G7
C2 F5 H8
insert 1 6 2 3 4 5 B6
D3 J9
E4 I 10
81
Traveling Sales Man Problem
• This is the problem for finding the shortest tour through a given set of cities and visits each
city exactly once before returning to the city where it is started.
• This problem can be conveniently modeled by a weighted graph to find the shortest
Hamiltonian circuit of the graph.
• Hamiltonian circuit is a cycle that passes through all the vertices of the graph exactly once.

Real World Example

• Transportation : service calls, school bus routes

• Manufacturing : industrial robot that drills holes in printed circuit boards
• Communication : planning new telecommunication networks
• Computer wiring : connecting computer components using minimum wire length

82
Traveling Sales Man Example

83
Traveling Sale Man Problem
A B A B C D
A 0 16 11 6
B 8 0 13 16
C 4 7 0 9
D 5 12 2 0
C D
A B C D
A 0 46 51 66
B 48 0 53 86
C 54 74 0 69
D 65 52 42 0

84
 A B C D
A 0 16 11 6
16+22=38 B 8 0 13 16
A 11+28=39 C 4 7 0 9
6+17=23
D 5 12 2 0
13+14=27 B 7+21=28 C 12+17=29 D
16+6=22 2+15=17
9+20=29

C D B D B C

9+5=14 2+4=6 16+5=21 12+8=20 13+4=17 7+8=15

D C D B C B

5 4 5 8 4 8
A A A A A A

85
 A B C D
46+182=228
51+169=220 A 0 46 51 66
A
66+159=225 B 48 0 53 86
C 54 74 0 69
53+134=187 B 74+151=225 C 52+107=159 D D 65 52 42 0
86+96=182 42+122=164
69+100=169

C D B D B C

69+65=134 42+54=96 86+65=151 52+48=100 53+54=107 74+48=122

D C D B C B

65 54 65 48 54 48
A A A A A A

86
Knapsack Problem
• This is the problem for finding the most valuable subset of the items that fits into the
knapsack.

Real World Example

• Power Allocation Management

• Production planning problem
• Network selection for mobile nodes
• Formulation and solution method of touring conducting

87
Knapsack Example

The most valuable subset of the items = {3,4} .

88
Assignment Problem
• This is the problem for finding an assignment with the minimum total cost.
• The general case of the assignment problem is n!.

Assignment Problem Example

You work as a sales manager for a toy manufacturer, and you currently have three salespeople on the
road meeting buyers. Your salespeople are in Austin, Boston, and Chicago. You want them to fly to
three other cities: Denver, Edmonton, and Fargo.

89
Assignment Problem Example (Cont’d.)
• The table below shows the cost of airplane tickets in dollars between these cities.

• Where should you send each of your salespeople in order to minimize airfare?
90
Assignment Problem Example (Cont’d.)
• The table below shows the cost of airplane tickets in dollars between these cities.

• Where should you send each of your salespeople in order to minimize airfare?
91
Assignment Problem Example (Cont’d.)
D E F
A 250 400 350
B 400 600 350
C 200 400 250

<D, E, F> cost = 250 + 600 + 250 = 1100

<D, F, E> cost = 250 + 350 + 400 = 1000
<E, D, F> cost = 400 + 400 + 250 = 1050
<E, F, D> cost = 400 + 350 + 200 = 950
<F, D, E> cost = 350 + 400 + 400 = 1150
<F, E, D> cost = 350 + 600 + 200 = 1150

Thus your salesperson should travel Austin to Edmonton, Boston to Fargo and Chicago to Denver.

92
Assignment Problem Exercises

From/ To Kyine Tone Mandalay Myike

Yangon 250 150 270

Sittwe 450 200 380

Naypyitaw 320 120 320

93
94
95
Defining the Problem Example

96
Algorithms for The Above Example

Sorting Algorithms

• Bubble Sort
• Selection Sort
You can look how sorting works in the following link
• Insertion Sort
• Merge Sort https://visualgo.net/en
• Quick Sort

97
 𝑛−1 𝑛−1−𝑖 𝑛−1 𝑛−1 𝑛−1
(𝑛−1)𝑛 2 2 ¿ 𝑛 −𝑛 2
2
∑ ∑ 1=∑ 𝑛−𝑖=𝑛∑ 1−∑ 𝑖=𝑛 − 2 =2𝑛 −𝑛 +𝑛¿ 2= 2 =𝑂(𝑛 )
2

𝑖=0 𝑗=0 𝑖=0 𝑖=0 𝑖= 0

Bubble Sort Algorithm
Algorithm BubbleSort(A,n)
for i<-0 to n-1 do
{
for j<-0 to n-i-1 do
{
if A[j]>A[j+1] then
{
swap(A[j],A[j+1])
}
}
}
98
 i=0,j=7 =n-i-1=8-0-1=7
i=1,j=6 =8-1-1=6
i=2,j=5
i=3,j=4
i=4,j=3
Bubble Sort Algorithm i=5,j=2
i=6,j=1=
i=0 j 0 1 2 3 4 5 6 7 i=1 j 0 1 2 3 4 5 6 7
0 5 3 1 9 8 2 4 7 0 3 1 5 8 2 4 7 9
1 3 5 1 9 8 2 4 7 1 1 3 5 8 2 4 7 9
2 3 1 5 9 8 2 4 7 2 1 3 5 8 2 4 7 9
3 3 1 5 9 8 2 4 7 3 1 3 5 8 2 4 7 9
4 8 9 2 4 7 4 1 3 5 2 8 4 7 9
5 2 9 4 7 5 1 3 5 2 4 8 7 9
6 3 1 5 8 2 4 9 7 7 8

8-0-1

8-1-1

99
Bubble Sort Algorithm
i=2 j 0 1 2 3 4 5 6 7 i=3 j 0 1 2 3 4 5 6 7
0 1 3 5 2 4 7 8 9 0 1 3 2 4 5 7 8 9
1 1 3 5 2 4 7 8 9 1 1 3 2 4 5 7 8 9
2 1 3 5 2 4 7 8 9 2 1 2 3 4 5 7 8 9
3 1 3 2 5 4 7 8 9 3 1 2 3 4 5 7 8 9
4 1 3 2 4 5 7 8 9 i=5 j 0 1 2 3 4 5 6 7
0 1 2 3 4 5 7 8 9
i=4 j 0 1 2 3 4 5 6 7
1 1 2 3 4 5 7 8 9
0 1 2 3 4 5 7 8 9
1 1 2 3 i=6 j 0 1 2 3 4 5 6 7
4 5 7 8 9
2 1 2 3 0 1 2 3 4 5 7 8 9
4 5 7 8 9

1 2 3 4 5 7 8 9
100
Bubble Sort Quiz
What is the best time complexity of bubble sort?

A O(n^2) Explanation: The bubble sort is at its best if

the input data is sorted. i.e. If the input data is
sorted in the same order as expected output.
B O(Log(n)) This can be achieved by using one boolean
variable. The boolean variable is used to
check whether the values are swapped at least
C O(n) once in the inner loop.

D O(Log(n)^2)

101
Bubble Sort Quiz
The number of swappings needed to sort the numbers 8, 22, 7, 9, 31, 5, 13 in
ascending order, using bubble sort is

A 11
Explanation:
1 : 8, 7, 9, 22, 5, 13, 31 = 4 swaps
B 12
2 : 7, 8, 9, 5, 13, 22, 31 = 3 swaps
3 : 7, 8, 5, 9, 13, 22, 31 = 1 swap
4 : 7, 5, 8, 9, 13, 22, 31 = 1 swap
C 13
5 : 5, 7, 8, 9, 13, 22, 31 = 1 swap
Total 10 swaps are required to sort the array.
D 10

102
Worst Case Analysis for Bubble Sort
At Pass1:Number of Comparison =n-1
Number of Swaps =n-1
At Pass2:Number of Comparison =n-2
Number of Swaps =n-2
.
.
.
At Passn-1 :Number of Comparison =1
Number of Swaps =1
C(n)=(n-1)+(n-2)+(n-3)+…+1==

Total number of Comparison= =O(n^2)

Total number of swap =

103
 i j Min
0 1 =>5 0=> 7,1
2 =>4 1=>5, 2
3=>2 2=>4,3
Swap(A[0],A[3] 2547
Selection Sort Algorithm 1 2=>4
3=>7
1=>5, 2
2=>4
Swap(A[1],A[2]) 2457
2 3=>7 2=>5
Swap(A[2],A[2]) 2457

i=0 7 5 4 2
i=1 2 5 4 7
i=2 2 4 5 7 104
 0 1 2 3
i=0 17 15 14 12
i J < min
i=1 12 15 14 17
0 1 = 15 0 =17, 1
2 = 14 1 =15, 2
3 =12 2 =14 ,3
Swap(A[i],A[min]=> swap(17,12)
12 15 14 17
1 2 = >14 1 =>15, 2
3 => 17 2 =>14
Swap(1],A[2]=> swap(15,14)
12 14 15 17
2 3 =>17 2 =>15
Swap(A[2],A[2])
12 14 15 17

105
Time Complexity of Selection Sort Algorithm
• T(n)

• O(n^2)

106
 7 5 4 2

Find Smallest Element => Swap

7 5 4 2

2 5 4 7

2 4 5 7

107
 i J< min
0 1(28) 0(15)
2(17) 0(15)
Exercises For Selection Sort 3(12) < 0(15),3
4(18) 3(12)
15 28 17 12 18 9 6 5(9) 3(12),5
6(6) 5(9),6
0 1 2 3 4 5 6 0 6
A 15 28 17 12 18 9 6 6 28 17 12 18 9 15
i=0 6 28 17 12 18 9 15 1 2(17) 1(28),2
i=1 6 9 17 12 18 28 15 3(12) 2(17)
i=2 6 9 12 17 18 28 15 4(18) 3(12)
i=3 6 9 12 15 18 28 17 5(9) 3(12),5
i=4 6 9 12 15 17 28 18 6(15) 5(9)
i=5 6 9 12 15 17 18 28 1 5
To be Continue…

108
Selection Sort Algorithm Quiz
Consider a situation where swap operation is very costly. Which of the following sorting algorithms should be
preferred so that the number of swap operations are minimized in general?

A Heap Sort

Explanation:
B Selection Sort Selection sort makes O(n) swaps which is
minimum among all sorting algorithms
mentioned above.
C Insertion Sort

D Merge Sort

109
Insertion Sort Algorithm n=6

110
7 8 5 2 4 6 3
2 4 5 7 8 6

7 5 8 2 4 5 7 6 8

5 7 8 2 2 4 5 6 7 8 3

5 7 2 8 2 4 5 6 7 3 8

2 4 5 6 3 7
5 2 7

2 4 5 3 6
2 5 7 8 4
2 4 3 5
2 5 7 4 8
2 3 4 5 6 7 8
2 5 4 7 8

2 4 5 7 8 6
111
Merge Sort Algorithm

112
113
Merge Sort Algorithm

114
Merge Sort Algorithm

115
 Merge Sort
Algorithm
Complexity

+1
E.g.=> n=8=>
=O(n )
116
Merge Sort Algorithm Quiz
In a modified merge sort, the input array is splitted at a position one-third of the length(N) of the
array. Which of the following is the tightest upper bound on time complexity of this modified Merge
Sort.

A NlogN base 3
Explanation: The time complexity is given by:
T(N) = T(N/3) + T(2N/3) + N
B NlogN base 1/3
Solving the above recurrence relation gives,
T(N) = N(logN base 3/2)
C NlogN base 2/3

D NlogN base 3/2

117
Merge Sort Algorithm Quiz
A list of n string, each of length n, is sorted into lexicographic order using the merge-sort algorithm.
The worst-case running time of this computation is

A O(nlogn)
Explanation: The recurrence tree for merge sort will have height
Log(n). And O(n^2) work will be done at each level of the
B O(n2logn) recurrence tree (Each level involves n comparisons, and a
comparison takes O(n) time in worst case). So, time complexity of
this Merge Sort will be .
C O(n2+logn)

D O(n2)
118
Heap Sort Algorithm (Simple Heapify)
15 20 7 9 30

20
30

20 20
15 20

20
30
15 7
20
15 15 7 15 7

30
15
9 9

119
 Heap Sort Algorithm (Build Max Heap)
30 20 7 9 15

15
30
9
20
15 15
9

15
20 7
15
9 7
20 7

9
15
9
20 15 7 9 30 15 9 7 20 30

15 20 7 9 30 9 15 7 20 30 7 9 15 20 30

120
Heap Sort Algorithm
7 9 15 20 30
7
7
9

7 9 15 20 30
9
7

9 7 15 20 30

7 9 15 20 30

121
 25 50 80 45 35 15 20 90

90 80 50 45 35 15 20 25
25 80 50 45 35 15 20 90

80 45 50 25 35 15 20 90 50 45 20 25 35 15 80 90
20 45 50 25 35 15 80 90 15 45 20 25 35 50 80 90

45 35 20 25 15 50 80 90
15 35 20 25 45 50 80 90 35 25 20 15 45 50 80 90
15 25 20 35 45 50 80 90

122
MAX HEAPIFY1 i=Length(n)/2 to 1

1
2 3
90 57 20 30 27 17 5 2 1 3 16
16 5

4 5 6 7 To be continue with build heap

30 27 17 20

8 9 10 11

2 57 3 90

123
 Quick Sort Algorithm
QuickSort(A,start,end) Partition(A,start,end)
{
{ if (start<end)
pivot<-A[end]
{ pindex<-start
pindex<-Partition(A,start,end) for i<-start to end-1
QuickSort(A,start,pindex-1) {
if(A[i]<=pivot){
QuickSort(A,pindex+1,end)
swap(A[i],A[pindex])
} pindex<-pindex+1
} }
swap(A[pindex],A[end])
return pindex
}
124
 i Pivot pindex start end
2 1 3 0(2) 3 0,1 0 2
i Pivot pindex start end
2 1 3 1(1) 3 1,2
7 2 1 6 8 5 3 4 0(7) 4 0 0 7
Swap(A[2],A[2])
2 7 1 6 8 5 3 4 1(2) 4 0,1
2 1 7 6 8 5 3 4 2(1) 4 1,2 i Pivot pindex start end
2 1 7 6 8 5 3 4 3(6) 4 2 2 1 0(2) 1 0 0 1
2 1 7 6 8 5 3 4
4(8) 4 2 1 2 Swap(A[0],A[1]
5(5) 4 2
2 1 7 6 8 5 3 4 i Pivot pindex start end
6(3) 4 2,3
2 1 3 6 8 5 7 4 8 5 7 6 4(8) 6 4 4 7
Swap(A[3],A[7])
2 1 3 4 8 5 7 6 5 8 7 6 5(5) 6 4,5
5 8 7 6 6(7) 6 1
2 1 3 8 5 7 6
5 6 7 8 Swap(A[5],A[7])

2 1 5 6 7 8
i Pivot pindex start end
1 2 5 7 8 7 8 6(7) 8 6 6 7
7
Swap(A[7],A[7] 125
 0 4 7
35 15 25 20 80 50 90 45

35 15 25 20 45 50 90 80

35 15 25 20 50 90 80

15 20 25 35 50 80 90

15 25 35 50 90

25 35

15 20 20 35 45 50 80 90
126
 Best Case Time Complexity(Quick Sort)

• T(1)=c1
• T(n) =2T(n/2)+cn
=2{2T(n/4)+cn/2)+cn
=4T(n/4)+2cn
=4{2T(n/8)+cn/4)+2cn
=8T(n/8)+3cn
.
.
.
=2k T(n/ 2k )+kcn
=
=nc1+cn
=O(n )
127
Worst Case Time Complexity(Order sorted)
T(1)=C1
T(n) =T(n-1)+cn
={T(n-2)+c(n-1)}+cn=T(n-2)+2cn-c
={T(n-3)+c(n-2)}+2cn-c=T(n-3)+3cn-3c
={T(n-4)+c(n-3)}+3cn-3c =T(n-4) +4cn-6c
.
.
.
={T(n-k)+kcn-k(k-1)/2
=T(1)+cn^2-(n(n-1)/2)c
=O(n^2) 128
Analysis of Algorithms
• Efficiency of an algorithm can be analyzed at two different stages, before implementation and
after implementation. They are the following −
• A Priori Analysis − This is a theoretical analysis of an algorithm. Efficiency of an algorithm is
measured by assuming that all other factors, for example, processor speed, are constant and have
no effect on the implementation.
• A Posterior Analysis − This is an empirical analysis of an algorithm. The selected algorithm is
implemented using programming language. This is then executed on target computer machine. In
this analysis, actual statistics like running time and space required, are collected.

129
Posterior Analysis of Algorithms
• The time and Space the algorithms uses are the two major measures of the effiency of an
algorithm.
• Time Factor : Time is measured by counting the number of key operations such as comparisons in
sorting algorithms.
• Space Factor : Space is measured by the amount of memory required by the algorithm.
• The space required by an algorithm is equal to the sum of the following two components :
• A Fixed Part : A space required to store certain data and variables that are independent of the size
of the problem Eg. Simple variables, constants, program size.

• A Variable Part : A space required by variables whose size depends on the size of the problem. Eg.
Dynamic memory allocation, recursion stack space,

130
Time Complexity
• Time Factor : Time is measured by counting the number of key operations such as comparisons in
sorting algorithms.

Input size

T(n) ≈ cop C(n)

Execution time for No: of times key operation

Running Time is executed.
basic operation

131
Key Operations Example

Problem Input size measure Key operation

Searching for key in a list of n

Number of list’s items, i.e. n Key comparison
items

Matrix dimensions or total Multiplication of two

Multiplication of two matrices
number of elements numbers

Visiting a vertex or
Typical graph problem #vertices and/or edges
traversing an edge

132
Asymptotic Analysis
• Asymptotic analysis refers in defining the mathematical boundary / framing of the run-time
performance of the algorithm.
• By using the asymptotic analysis, algorithms can be conclude into three cases :
• Worst Case ( maximum time for execution and maximum space for storage )
• Average Case
• Best Case
Asymptotic Notations
Asymptotic notations are used to express the calculation of the running time.
• Ο Notation
• Ω Notation
• θ Notation
133
Basic Asymptotic Efficiency Classes
Class Name
1 constant
log n logarithmic
n linear
n log n linearithmic
n2 quadratic
n3 cubic
2n exponential
n! factorial
134
Efficiency Among Sorting Algorithms

Algorithm Best Case Average Case Worst Case

Bubble Sort n n2 n2
Selection Sort n n2 n2
Insertion Sort n n2 n2
Merge Sort n log n n log n n log n
Quick Sort n log n n log n n2

135
Efficiency Among Searching Algorithms

Algorithm Best Case Average Case Worst Case

Linear Search 1 n /2 n
Binary Search 1 log n log n

136
 General Plan for Analyzing the Time Efficiency of Non recursive
Algorithms
1. Decide on a parameter (or parameters) indicating an input’s size.

2. Identify the algorithm’s basic operation. (As a rule, it is located in the innermost loop.)

3. Check whether the number of times the basic operation is executed depends only on
the size of an input. If it also depends on some additional property, the worst-case,
average-case, and, if necessary, best-case efficiencies have to be investigated separately.

4. Set up a sum expressing the number of times the algorithm’s basic operation is
executed.

5. Using standard formulas and rules of sum manipulation, either find a closed form
formula for the count or, at the very least, establish its order of growth. 137
 Rule and Solutions

Rule 1

𝑢 𝑢 𝑢
Rule2 ∑ 𝑎𝑖 ± 𝑏𝑖 = ∑ 𝑎𝑖 ± ∑ 𝑏𝑖
𝑖= 𝑙 𝑖= 𝑙 𝑖= 𝑙

S1
where L<=u are some upper and lower integer limit

𝑛
S2 𝑛 ( 𝑛+1 ) 1 2
∑ 𝑖=1+2 … +𝑛=
2
≈
2
𝑛 ∈ 𝑂 (𝑛2
)
𝑖= 0 138
Analysis of Algorithms

139
Basic asymptotic efficiency
classes

140
Mathematical Analysis of Non-Recursive
Algorithms
• In analysis of algorithms, we use especially frequently two basic rules
of sum manipulation

and two summation formulas:

141
Finding the Maximum

Basic operation: Comparison in loop

142
Unique Elements

• Basic operation: Comparison

in innermost loop

143
Unique Elements ( Cont’d.)

144
Matrix Multiplication

145
Matrix Multiplication(Cont’d.)

• Basic operation: Multiplication in innermost loop

146
Number of Bits in a integer

Basic operation: Comparison

C(n) = └ log2 n ┘+1 = Ѳ(log 2 n)

147
Number of Bits in a integer

Basic operation: Comparison

C(n) = └ log2 n ┘+1 = Ѳ(log2 n)

148
Number of Bits in a integer ( Cont’d.)
How to get log2n complexity for n/2 ?
For 16, we have that
• 16 / 2 = 8
• 8/2=4
• 4/2=2
• 2/2=1

Notice that this ends up taking four steps to complete. Interestingly, we also have that log 2 16 = 4.
For 128, we have
n / 2k ≥ 1
• 128 / 2 = 64
• 64 / 2 = 32
n ≥ 2k
• 32 / 2 = 16 log2 n ≥ k
• 16 / 2 = 8 k ≤ log2 n
• 8/2=4
• 4/2=2
k = └ log2 n ┘
• 2 / 2 =1
149
This took seven steps, and log 2 128 = 7.
Mathematical Analysis for Recursive Algorithms

• Recursive Algorithms example :

• Factorial

• BinRec

• Fibonacci

150
Factorial Algorithm

Basic operation: Multiplication

151
Factorial Algorithm(Cont’d.)

152
Factorial Algorithm(Cont’d.)
• M(n) = M(n - 1) + 1 for n>0,
• M(0) = 0
• M(n) = M(n - 1) + 1
= [M(n - 2) + 1] + 1 = M(n - 2) + 2
= [M(n - 3) + 1] + 2 = M(n - 3) + 3
= ...
= M(n - i) + i
=...
= M(n - n) + n
= M(0) + n
=0+n
= n = Ѳ(n)
153
Number of Bits in an Integer

Basic Operation: Addition

154
Number of Bits in an Integer
• Forward substitution.
A(2) = A(1) + 1 = 1
A(4) = A(2) + 1 = 2
A(8) = A(4) + 1 = 3
• Backward substitution.
A(n) = A(n/2) + 1
= [A(n/4) + 1] + 1 = A(n/4) + 2
= [A(n/8) + 1] + 2 = A(n/8) + 3
155
Number of Bits in an Integer (Cont’d)
• Let n = 2k
• A(2k) = A(2k−1) +1 for k > 0,
• A(20) = 0
• A(2k) = A(2k−1) + 1
= [ A(2k−2) +1 ] + 1 = A(2k−2) + 2
= [ A(2k−3) + 1] + 2 = A(2k−3) + 3
=...
=A(2k−i)+i
=...
=A(2k−k) + k
= A(20) + k
= 0 + k = log2 n = Ѳ(log2 n) 156
Number of Bits in an Integer (Cont’d)
• A(n)=A(└ n/2 ┘)+1 for n > 1,
• A(1) = 0
• substitute n = 2k (also k = log2 n)
• A(n) = A(└ n/21 ┘)+1
= [ A(└ n/22 ┘) + 1 ] + 1 = A(└ n/22 ┘) + 2
= [A(└ n/23 ┘) + 1 ] + 2 = A(└ n/23 ┘) + 3
=...
=+k
=

157
Fibonacci
Fib (n)
if n <=1 return n
else
return Fib(n-1)+ Fib (n-2)

158
Fibonacci
T(n)=T(n-1)+T(n-2)+4
T(0)=T(1)=1
T(n-1) ≈ T(n-2) T(n)∞ 2n/2
T(n) = 2T(n-2)+C
Lower Bound
=2[2T(n-4)+C]+C=4T(n-4)+2C+C=4T(n-4)+3C
=8T(n-6)+7C
=16T(n-8)+15C
=2kT(n-2k)+(2k-1)C
n-2k =0 =>k=n/2
T(n)=2n/2T(n-2n/2)+(2n/2-1)C
=2n/2T(0)+(2n/2-1)C
=2n/2+(2n/2C-C)
159
= 2n/2 (1+C)-C
Fibonacci
T(n)=T(n-1)+T(n-2)+4
T(0)=T(1)=1
T(n-2) ≈ T(n-1) T(n)∞ 2n
T(n) = 2T(n-1)+C
Upper Bound
=2[2T(n-2)+C]+C=4T(n-2)+2C+C=4T(n-2)+3C
=8T(n-3)+7C
T(n)∞ 2n/2
=16T(n-4)+15C
=2kT(n-k)+(2k-1)C Lower Bound

n-k =0 =>k=n
T(n)=2nT(n-n)+(2n-1)C O(2n)
=2nT(0)+(2n-1)C
=2n+(2nC-C)
160
= 2n (1+C)-C
Thanks you