Chapter 1

Data Structure and Algorithms

DSA
 Introduction
• A program is written in order to solve a problem.
A solution to a problem actually consists of two
things:
 A way to organize the data
 Sequence of steps to solve the problem
• The way data are organized in a computers
memory is said to be Data Structure and the
sequence of computational steps to solve a
problem is said to be an algorithm.
• Therefore, a program is nothing but data
structures plus algorithms.
 Introduction
• Given a problem, the first step to solve the problem is obtaining
ones own abstract view, or model, of the problem. This process
of modeling is called abstraction.
• The model defines an abstract view to the problem. This implies
that the model focuses only on problem related stuff and that a
programmer tries to define the properties of the problem.
• These properties include
 The data which are affected and
 The operations that are involved in the problem.
• With abstraction you create a well-defined entity that can be
properly handled. These entities define the data structure of the
program.
• An entity with the properties just described is called an abstract
data type (ADT).
 Abstraction
• Abstraction is a process of classifying characteristics as
relevant and irrelevant for the particular purpose at hand
and ignoring the irrelevant ones.
• Applying abstraction correctly is the essence of successful
programming . How do data structures model the world or
some part of the world?
The value held by a data structure represents some specific
characteristic of the world
The characteristic being modeled restricts the possible values
held by a data structure
The characteristic being modeled restricts the possible operations
to be performed on the data structure.
• Note: Notice the relation between characteristic, value, and
data structures
 Abstract Data Types (ADT)
• Abstract Data Types An ADT consists of an abstract data structure and
operations.
• The ADT specifies:
1. What can be stored in the Abstract Data Type
2. What operations can be done on/by the Abstract Data Type? For example, if
we are going to model employees of an organization:
This ADT stores employees with their relevant attributes and discarding irrelevant
attributes.
This ADT supports hiring, firing, retiring, … operations.
• A data structure is a language construct that the programmer has defined in
order to implement an abstract data type. There are lots of formalized and
standard Abstract data types such as Stacks, Queues, Trees, etc.
• Do all characteristics need to be modeled? Not at all
It depends on the scope of the model
It depends on the reason for developing the model
 Data Structure
• Data Structure is a systematic way to organize data in
order to use it efficiently. Following terms are the
foundation terms of a data structure.
• Interface − Each data structure has an interface. Interface
represents the set of operations that a data structure
supports. An interface only provides the list of supported
operations, type of parameters they can accept and
return type of these operations.
• Implementation − Implementation provides the internal
representation of a data structure. Implementation also
provides the definition of the algorithms used in the
operations of the data structure.
Why to Learn Data Structure and Algorithms?
• As applications are getting complex and data rich, there are three common
problems that applications face now-a-days.
• Data Search − Consider an inventory of 1 million(106) items of a store. If the
application is to search an item, it has to search an item in 1 million(10 6)
items every time slowing down the search. As data grows, search will become
slower.
• Processor speed − Processor speed although being very high, falls limited if
the data grows to billion records.
• Multiple requests − As thousands of users can search data simultaneously on
a web server, even the fast server fails while searching the data.
• To solve the above-mentioned problems, data structures come to rescue.
Data can be organized in a data structure in such a way that all items may not
be required to be searched, and the required data can be searched almost
instantly.
Characteristics of a Data Structure
• Correctness − Data structure implementation
should implement its interface correctly.
• Time Complexity − Running time or the
execution time of operations of data structure
must be as small as possible.
• Space Complexity − Memory usage of a data
structure operation should be as little as
possible.
 Applications of Data Structure and
Algorithms
• Algorithm is a step-by-step procedure, which defines a set of
instructions to be executed in a certain order to get the desired
output. Algorithms are generally created independent of underlying
languages, i.e. an algorithm can be implemented in more than one
programming language.
• From the data structure point of view, following are some important
categories of algorithms −
• Search − Algorithm to search an item in a data structure.
• Sort − Algorithm to sort items in a certain order.
• Insert − Algorithm to insert item in a data structure.
• Update − Algorithm to update an existing item in a data structure.
• Delete − Algorithm to delete an existing item from a data structure.
 Algorithms
• An algorithm is a well-defined computational procedure
that takes some value or a set of values as input and
produces some value or a set of values as output.
• Data structures model the static part of the world. They
are unchanging while the world is changing.
• In order to model the dynamic part of the world we
need to work with algorithms.
• Algorithms are the dynamic part of a program’s world
model. An algorithm transforms data structures from
one state to another state in two ways:
 An algorithm may change the value held by a data structure
 An algorithm may change the data structure itself
 Algorithms
• The quality of a data structure is related to its ability
to successfully model the characteristics of the world.
• Similarly, the quality of an algorithm is related to its
ability to successfully simulate the changes in the
world.
• However, independent of any particular world model,
the quality of data structure and algorithms is
determined by their ability to work together well.
• Generally speaking, correct data structures lead to
simple and efficient algorithms and correct
algorithms lead to accurate and efficient data
structures.
 Algorithm
• Algorithm is a step-by-step procedure, which defines a set of
instructions to be executed in a certain order to get the desired
output. Algorithms are generally created independent of
underlying languages, i.e. an algorithm can be implemented in
more than one programming language.
• From the data structure point of view, following are some
important categories of algorithms −
• Search − Algorithm to search an item in a data structure.
• Sort − Algorithm to sort items in a certain order.
• Insert − Algorithm to insert item in a data structure.
• Update − Algorithm to update an existing item in a data structure.
• Delete − Algorithm to delete an existing item from a data
structure.
 Properties of an algorithm
• Finiteness: Algorithm must complete after a finite number of
steps.
• Definiteness: Each step must be clearly defined, having one and
only one interpretation. At each point in computation, one should
be able to tell exactly what happens next.
• Sequence: Each step must have a unique defined preceding and
succeeding step. The first step (start step) and last step (halt step)
must be clearly noted.
• Feasibility: It must be possible to perform each instruction.
• Correctness: It must compute correct answer for all possible legal
inputs.
• Language Independence: It must not depend on any one
programming language
 Properties of an algorithm
• Completeness: It must solve the problem
completely.
• Effectiveness: It must be possible to perform
each step exactly and in a finite amount of time.
• Efficiency: It must solve with the least amount of
computational resources such as time and space.
• Generality: Algorithm should be valid on all
possible inputs.
• Input/Output: There must be a specified number
of input values, and one or more result values.
 How to Write an Algorithm?
• There are no well-defined standards for writing algorithms.
Rather, it is problem and resource dependent. Algorithms
are never written to support a particular programming code.
• As we know that all programming languages share basic
code constructs like loops (do, for, while), flow-control (if-
else), etc. These common constructs can be used to write an
algorithm.
• We write algorithms in a step-by-step manner, but it is not
always the case. Algorithm writing is a process and is
executed after the problem domain is well-defined. That is,
we should know the problem domain, for which we are
designing a solution.
Example
• Let's try to learn algorithm-writing by using an example.
• Problem − Design an algorithm to add two numbers and display the result.
• Step 1 − START
• Step 2 − declare three integers a, b & c
• Step 3 − define values of a & b
• Step 4 − add values of a & b
• Step 5 − store output of step 4 to c
• Step 6 − print c
• Step 7 − STOP Algorithms tell the programmers how to code the program.

Alternatively, the algorithm can be written as −

• Step 1 − START ADD
• Step 2 − get values of a & b
• Step 3 − c ← a + b
• Step 4 − display c
• Step 5 − STOP
• Writing step numbers, is optional.
• We design an algorithm to get a solution of a
given problem.
• A problem can be solved in more than one ways.
• Hence, many solution algorithms can be derived
for a given problem.
• The next step is to analyze those proposed
solution algorithms and implement the best
suitable solution.
 Algorithm Analysis
• Algorithm analysis refers to the process of determining
the amount of computing time and storage space
required by different algorithms.
• In other words, it’s a process of predicting the resource
requirement of algorithms in a given environment.
• In order to solve a problem, there are many possible
algorithms.
• One has to be able to choose the best algorithm for the
problem at hand using some scientific method.
• To classify some data structures and algorithms as
good, we need precise ways of analyzing them in terms
of resource requirement.
 Algorithm Analysis
• The main resources are:
 Running Time
 Memory Usage
 Communication Bandwidth
 Algorithm Analysis
• Efficiency of an algorithm can be analyzed at two different stages,
before implementation and after implementation. They are the
following −
• A Priori Analysis − This is a theoretical analysis of an algorithm.
Efficiency of an algorithm is measured by assuming that all other
factors, for example, processor speed, are constant and have no
effect on the implementation.
• A Posterior Analysis − This is an empirical analysis of an algorithm.
The selected algorithm is implemented using programming language.
This is then executed on target computer machine. In this analysis,
actual statistics like running time and space required, are collected.
• We shall learn about a priori algorithm analysis. Algorithm analysis
deals with the execution or running time of various operations
involved. The running time of an operation can be defined as the
number of computer instructions executed per operation.
 Complexity Analysis
• Complexity Analysis is the systematic study of the cost of
computation, measured either in time units or in
operations performed, or in the amount of storage space
required.
• The goal is to have a meaningful measure that permits
comparison of algorithms independent of operating
platform.
• There are two things to consider:
Time Complexity: Determine the approximate number of
operations required to solve a problem of size n.
Space Complexity: Determine the approximate memory
required to solve a problem of size n.
 Algorithm Efficiency
• Algorithm Efficiency is used to describe
properties of an algorithm relating to how
much of various types of resources it
consumes. (Time and space are the most
frequently encountered resources).
• At the design stage of solving a particular
problem, there are two conflicting goals. These
are:
 Complexity Analysis
• To design an algorithm that is easy to understand,
code, and design.
 This goal is the concern of software engineers.
• To design an algorithm that makes efficient use of
computer resources such as CPU and memory (In
terms of hardware).
• This is a factor of time and space and results a
quantitative analysis of algorithm.
• This goal is the concern of data structure and
algorithm analysis.
 Algorithm Complexity
• Suppose X is an algorithm and n is the size of input data,
the time and space used by the algorithm X are the two
main factors, which decide the efficiency of X.
• Time Factor − Time is measured by counting the number
of key operations such as comparisons in the sorting
algorithm.
• Space Factor − Space is measured by counting the
maximum memory space required by the algorithm.
• The complexity of an algorithm f(n) gives the running
time and/or the storage space required by the algorithm
in terms of n as the size of input data.
 Space Complexity
• Space complexity of an algorithm represents the amount of
memory space required by the algorithm in its life cycle. The
space required by an algorithm is equal to the sum of the
following two components −
• A fixed part that is a space required to store certain data and
variables, that are independent of the size of the problem. For
example, simple variables and constants used, program size, etc.
• A variable part is a space required by variables, whose size
depends on the size of the problem. For example, dynamic
memory allocation, recursion stack space, etc.
• Space complexity S(P) of any algorithm P is S(P) = C + SP(I), where
C is the fixed part and S(I) is the variable part of the algorithm,
which depends on instance characteristic I.
• Following is a simple example that tries to
explain the concept −
• Algorithm: SUM(A, B)
• Step 1 - START
• Step 2 - C ← A + B + 10
• Step 3 - Stop
• Here we have three variables A, B, and C and
one constant.
• Hence S(P) = 1 + 3. Now, space depends on
data types of given variables and constant
types and it will be multiplied accordingly.
 Time Complexity
• Time complexity of an algorithm represents the amount
of time required by the algorithm to run to completion.
Time requirements can be defined as a numerical function
T(n), where T(n) can be measured as the number of steps,
provided each step consumes constant time.
• For example, addition of two n-bit integers takes n steps.
Consequently, the total computational time is T(n) = c ∗
n, where c is the time taken for the addition of two bits.
Here, we observe that T(n) grows linearly as the input size
increases.
 Measures of Times
• In order to determine the running time of an algorithm it is possible to
define three functions Tbest(n), Tavg(n) and Tworst(n) as the best, the
average and the worst case running time of the algorithm respectively.
 Average Case (Tavg): The amount of time the algorithm takes on an
"average" set of inputs.
• Average time required for program execution.
 Worst Case (Tworst): The amount of time the algorithm takes on the
worst possible set of inputs.
• Maximum time required for program execution.
 Best Case (Tbest): The amount of time the algorithm takes on the
smallest possible set of inputs.
 Minimum time required for program execution.
• We are interested in the worst-case time, since it provides a bound for
all input – this is called the “Big-Oh” estimate.
 Asymptotic Analysis
• Asymptotic analysis is concerned with how the
running time of an algorithm increases with the
size of the input in the limit, as the size of the
input increases without bound.
• There are five notations used to describe a
running time function. These are:
 Big-Oh Notation (O)
 Big-Omega Notation (Ω)
 Theta Notation (Θ)
 Little-o Notation (o)
 Little-Omega Notation (ω)
 Big Oh Notation, Ο
• The notation Ο(n) is the formal way to express
the upper bound of an algorithm's running
time.
• It measures the worst case time complexity or
the longest amount of time an algorithm can
possibly take to complete.
 Big Oh Notation, Ο
• Big-Oh notation is a way of comparing
algorithms and is used for computing the
complexity of algorithms; i.e., the amount of
time that it takes for computer program to
run.
• It’s only concerned with what happens for a
very large value of n.
• Therefore only the largest term in the
expression (function) is needed.
 Big Oh Notation, Ο
• For example, for a function f(n)
• Ο(f(n)) = { g(n) : there exists c > 0 and n0 such
that f(n) ≤ c.g(n) for all n > n0. }
 Big Oh Notation, Ο
• For example, if the number of operations in an
algorithm is n2 – n, n is insignificant compared
to n2 for large values of n. Hence the n term is
ignored.
• Of course, for small values of n, it may be
important. However, Big-Oh is mainly
concerned with large values of n.
 Omega Notation, Ω
• The notation Ω(n) is the formal way to express
the lower bound of an algorithm's running
time.
• It measures the best case time complexity or
the best amount of time an algorithm can
possibly take to complete.
 Omega Notation, Ω
• For example, for a function f(n)
• Ω(f(n)) ≥ { g(n) : there exists c > 0 and n0 such
that g(n) ≤ c.f(n) for all n > n0. }
 Theta Notation, θ
• The notation θ(n) is the formal way to express
both the lower bound and the upper bound of
an algorithm's running time.
• It is represented as follows −
• θ(f(n)) = { g(n) if and only if g(n) = Ο(f(n)) and
g(n) = Ω(f(n)) for all n > n0. }
 Common Asymptotic Notations

constant − Ο(1)

logarithmic − Ο(log n)

linear − Ο(n)

n log n − Ο(n log n)

quadratic − Ο(n2)

cubic − Ο(n3)

polynomial − nΟ(1)

exponential − 2Ο(n)
• Typical Orders Here is a table of some typical
cases. This uses logarithms to base 2, but these
are simply proportional to logarithms in other
base.
Orders of Common Functions
End of chapter 1