Data Structure and Algorithms

Data Structures are the programmatic way of storing data so that data can be used efficiently.
Almost every enterprise application uses various types of data structures in one or the other
way. Understanding on Data Structures needed to understand the complexity of enterprise
level applications and need of algorithms, and data structures.
Why to Learn Data Structure and Algorithms?
As applications are getting complex and data rich, there are three common problems that
applications face now-a-days.
 Data Search − Consider an inventory of 1 million(10 6) items of a store. If the application
is to search an item, it has to search an item in 1 million(10 6) items every time slowing
down the search. As data grows, search will become slower.
 Processor speed − Processor speed although being very high, falls limited if the data
grows to billion records.
 Multiple requests − As thousands of users can search data simultaneously on a web
server, even the fast server fails while searching the data.
To solve the above-mentioned problems, data structures come to rescue. Data can be
organized in a data structure in such a way that all items may not be required to be searched,
and the required data can be searched almost instantly.
Applications of Data Structure and Algorithms
Algorithm is a step-by-step procedure, which defines a set of instructions to be executed in a
certain order to get the desired output. Algorithms are generally created independent of
underlying languages, i.e. an algorithm can be implemented in more than one programming
language.
From the data structure point of view, following are some important categories of algorithms −
 Search − Algorithm to search an item in a data structure.
 Sort − Algorithm to sort items in a certain order.
 Insert − Algorithm to insert item in a data structure.
 Update − Algorithm to update an existing item in a data structure.
 Delete − Algorithm to delete an existing item from a data structure.
The following computer problems can be solved using Data Structures −

 Fibonacci number series

 Knapsack problem
 Tower of Hanoi
 All pair shortest path by Floyd-Warshall
 Shortest path by Dijkstra
 Project scheduling
Data Structure is a systematic way to organize data in order to use it efficiently. Following
terms are the foundation terms of a data structure.
 Interface − Each data structure has an interface. Interface represents the set of
operations that a data structure supports. An interface only provides the list of supported
operations, type of parameters they can accept and return type of these operations.
 Implementation − Implementation provides the internal representation of a data
structure. Implementation also provides the definition of the algorithms used in the
operations of the data structure.
Characteristics of a Data Structure
 Correctness − Data structure implementation should implement its interface correctly.
 Time Complexity − Running time or the execution time of operations of data structure
must be as small as possible.
 Space Complexity − Memory usage of a data structure operation should be as little as
possible.
Need for Data Structure
As applications are getting complex and data rich, there are three common problems that
applications face now-a-days.
 Data Search − Consider an inventory of 1 million(10 6) items of a store. If the application
is to search an item, it has to search an item in 1 million(10 6) items every time slowing
down the search. As data grows, search will become slower.
 Processor speed − Processor speed although being very high, falls limited if the data
grows to billion records.
 Multiple requests − As thousands of users can search data simultaneously on a web
server, even the fast server fails while searching the data.
To solve the above-mentioned problems, data structures come to rescue. Data can be
organized in a data structure in such a way that all items may not be required to be searched,
and the required data can be searched almost instantly.
Execution Time Cases
There are three cases which are usually used to compare various data structure's execution
time in a relative manner.
 Worst Case − This is the scenario where a particular data structure operation takes
maximum time it can take. If an operation's worst case time is ƒ(n) then this operation
will not take more than ƒ(n) time where ƒ(n) represents function of n.
 Average Case − This is the scenario depicting the average execution time of an
operation of a data structure. If an operation takes ƒ(n) time in execution, then m
operations will take mƒ(n) time.
 Best Case − This is the scenario depicting the least possible execution time of an
operation of a data structure. If an operation takes ƒ(n) time in execution, then the actual
operation may take time as the random number which would be maximum as ƒ(n).
Basic Terminology
 Data − Data are values or set of values.
 Data Item − Data item refers to single unit of values.
 Group Items − Data items that are divided into sub items are called as Group Items.
 Elementary Items − Data items that cannot be divided are called as Elementary Items.
 Attribute and Entity − An entity is that which contains certain attributes or properties,
which may be assigned values.
 Entity Set − Entities of similar attributes form an entity set.
 Field − Field is a single elementary unit of information representing an attribute of an
entity.
 Record − Record is a collection of field values of a given entity.
 File − File is a collection of records of the entities in a given entity set.
ALGORITHM
Algorithm is a step-by-step procedure, which defines a set of instructions to be executed in a
certain order to get the desired output. Algorithms are generally created independent of
underlying languages, i.e. an algorithm can be implemented in more than one programming
language.
From the data structure point of view, following are some important categories of algorithms −
 Search − Algorithm to search an item in a data structure.
 Sort − Algorithm to sort items in a certain order.
 Insert − Algorithm to insert item in a data structure.
 Update − Algorithm to update an existing item in a data structure.
 Delete − Algorithm to delete an existing item from a data structure.
Characteristics of an Algorithm
Not all procedures can be called an algorithm. An algorithm should have the following
characteristics −
 Unambiguous − Algorithm should be clear and unambiguous. Each of its steps (or
phases), and their inputs/outputs should be clear and must lead to only one meaning.
 Input − An algorithm should have 0 or more well-defined inputs.
 Output − An algorithm should have 1 or more well-defined outputs, and should match
the desired output.
 Finiteness − Algorithms must terminate after a finite number of steps.
 Feasibility − Should be feasible with the available resources.
 Independent − An algorithm should have step-by-step directions, which should be
independent of any programming code.
How to Write an Algorithm?
There are no well-defined standards for writing algorithms. Rather, it is problem and resource
dependent. Algorithms are never written to support a particular programming code.
As we know that all programming languages share basic code constructs like loops (do, for,
while), flow-control (if-else), etc. These common constructs can be used to write an algorithm.
We write algorithms in a step-by-step manner, but it is not always the case. Algorithm writing is
a process and is executed after the problem domain is well-defined. That is, we should know
the problem domain, for which we are designing a solution.

Example
Let's try to learn algorithm-writing by using an example.
Problem − Design an algorithm to add two numbers and display the result.
Step 1 − START
Step 2 − declare three integers a, b & c
Step 3 − define values of a & b
Step 4 − add values of a & b
Step 5 − store output of step 4 to c
Step 6 − print c
Step 7 − STOP
Algorithms tell the programmers how to code the program. Alternatively, the algorithm can be
written as −
Step 1 − START ADD
Step 2 − get values of a & b
Step 3 − c ← a + b
Step 4 − display c
Step 5 − STOP
In design and analysis of algorithms, usually the second method is used to describe an
algorithm. It makes it easy for the analyst to analyze the algorithm ignoring all unwanted
definitions. He can observe what operations are being used and how the process is flowing.
Writing step numbers, is optional.
We design an algorithm to get a solution of a given problem. A problem can be solved in more
than one ways.
Hence, many solution algorithms can be derived for a given problem. The next step is to
analyze those proposed solution algorithms and implement the best suitable solution.
Algorithm Analysis
Efficiency of an algorithm can be analyzed at two different stages, before implementation and
after implementation. They are the following −
 A Priori Analysis − This is a theoretical analysis of an algorithm. Efficiency of an
algorithm is measured by assuming that all other factors, for example, processor speed,
are constant and have no effect on the implementation.
 A Posterior Analysis − This is an empirical analysis of an algorithm. The selected
algorithm is implemented using programming language. This is then executed on target
computer machine. In this analysis, actual statistics like running time and space
required, are collected.
We shall learn about a priori algorithm analysis. Algorithm analysis deals with the execution or
running time of various operations involved. The running time of an operation can be defined
as the number of computer instructions executed per operation.
Algorithm Complexity
Suppose X is an algorithm and n is the size of input data, the time and space used by the
algorithm X are the two main factors, which decide the efficiency of X.
 Time Factor − Time is measured by counting the number of key operations such as
comparisons in the sorting algorithm.
 Space Factor − Space is measured by counting the maximum memory space required
by the algorithm.
The complexity of an algorithm f(n) gives the running time and/or the storage space required
by the algorithm in terms of n as the size of input data.
Space Complexity
Space complexity of an algorithm represents the amount of memory space required by the
algorithm in its life cycle. The space required by an algorithm is equal to the sum of the
following two components −
 A fixed part that is a space required to store certain data and variables, that are
independent of the size of the problem. For example, simple variables and constants
used, program size, etc.
 A variable part is a space required by variables, whose size depends on the size of the
problem. For example, dynamic memory allocation, recursion stack space, etc.
Space complexity S(P) of any algorithm P is S(P) = C + SP(I), where C is the fixed part and
S(I) is the variable part of the algorithm, which depends on instance characteristic I. Following
is a simple example that tries to explain the concept −
Algorithm: SUM(A, B)
Step 1 - START
Step 2 - C ← A + B + 10
Step 3 - Stop
Here we have three variables A, B, and C and one constant. Hence S(P) = 1 + 3. Now, space
depends on data types of given variables and constant types and it will be multiplied
accordingly.
Time Complexity
Time complexity of an algorithm represents the amount of time required by the algorithm to run
to completion. Time requirements can be defined as a numerical function T(n), where T(n) can
be measured as the number of steps, provided each step consumes constant time.

time is T(n) = c ∗ n, where c is the time taken for the addition of two bits. Here, we observe that
For example, addition of two n-bit integers takes n steps. Consequently, the total computational

T(n) grows linearly as the input size increases.

ASYMPTOTIC ANALYSIS

Asymptotic analysis of an algorithm refers to defining the mathematical boundation/framing of

its run-time performance. Using asymptotic analysis, we can very well conclude the best case,
average case, and worst case scenario of an algorithm.
Asymptotic analysis is input bound i.e., if there's no input to the algorithm, it is concluded to
work in a constant time. Other than the "input" all other factors are considered constant.
Asymptotic analysis refers to computing the running time of any operation in mathematical
units of computation. For example, the running time of one operation is computed as f(n) and
may be for another operation it is computed as g(n2). This means the first operation running
time will increase linearly with the increase in n and the running time of the second operation
will increase exponentially when n increases. Similarly, the running time of both operations will
be nearly the same if n is significantly small.
Usually, the time required by an algorithm falls under three types −
 Best Case − Minimum time required for program execution.
 Average Case − Average time required for program execution.
 Worst Case − Maximum time required for program execution.
Asymptotic Notations
Following are the commonly used asymptotic notations to calculate the running time complexity
of an algorithm.

 Ο Notation
 Ω Notation
 θ Notation

Big Oh Notation, Ο
The notation Ο(n) is the formal way to express the upper bound of an algorithm's running time.
It measures the worst case time complexity or the longest amount of time an algorithm can
possibly take to complete.

For example, for a function f(n)

Ο(f(n)) = { g(n) : there exists c > 0 and n0 such that f(n) ≤ c.g(n) for
all n > n0. }

Omega Notation, Ω
The notation Ω(n) is the formal way to express the lower bound of an algorithm's running time.
It measures the best case time complexity or the best amount of time an algorithm can possibly
take to complete.
For example, for a function f(n)
Ω(f(n)) ≥ { g(n) : there exists c > 0 and n0 such that g(n) ≤ c.f(n) for
all n > n0. }

Theta Notation, θ
The notation θ(n) is the formal way to express both the lower bound and the upper bound of an
algorithm's running time. It is represented as follows −

θ(f(n)) = { g(n) if and only if g(n) = Ο(f(n)) and g(n) = Ω(f(n)) for
all n > n0. }

Common Asymptotic Notations

Following is a list of some common asymptotic notations −

constant − Ο(1)

logarithmic − Ο(log n)

linear − Ο(n)
 n log n − Ο(n log n)

quadratic − Ο(n2)

cubic − Ο(n3)

polynomial − nΟ(1)

exponential − 2Ο(n)

Asymptotic Notations

What is Asymptotic Notation?

Whenever we want to perform analysis of an algorithm, we need to calculate the complexity of that algorithm.
But when we calculate the complexity of an algorithm it does not provide the exact amount of resource
required. So instead of taking the exact amount of resource, we represent that complexity in a general form
(Notation) which produces the basic nature of that algorithm. We use that general form (Notation) for analysis
process.

Asymptotic notation of an algorithm is a mathematical representation of its complexity.

Note - In asymptotic notation, when we want to represent the complexity of an algorithm, we use only the
most significant terms in the complexity of that algorithm and ignore least significant terms in the complexity
of that algorithm (Here complexity can be Space Complexity or Time Complexity).

For example, consider the following time complexities of two algorithms...

 Algorithm 1 : 5n2 + 2n + 1

 Algorithm 2 : 10n2 + 8n + 3

Generally, when we analyze an algorithm, we consider the time complexity for larger values of input data
(i.e. 'n' value). In above two time complexities, for larger value of 'n' the term '2n + 1' in algorithm 1 has least
significance than the term '5n2', and the term '8n + 3' in algorithm 2 has least significance than the term '10n2'.
Here, for larger value of 'n' the value of most significant terms ( 5n2 and 10n2 ) is very larger than the value of
least significant terms ( 2n + 1 and 8n + 3 ). So for larger value of 'n' we ignore the least significant terms to
represent overall time required by an algorithm. In asymptotic notation, we use only the most significant terms
to represent the time complexity of an algorithm.

Majorly, we use THREE types of Asymptotic Notations and those are as follows...

1. Big - Oh (O)
 2. Big - Omega (Ω)

3. Big - Theta (Θ)

Big - Oh Notation (O)

Big - Oh notation is used to define the upper bound of an algorithm in terms of Time Complexity.
That means Big - Oh notation always indicates the maximum time required by an algorithm for all input
values. That means Big - Oh notation describes the worst case of an algorithm time complexity.
Big - Oh Notation can be defined as follows...

Consider function f(n) as time complexity of an algorithm and g(n) is the most significant term. If f(n) <= C
g(n) for all n >= n0, C > 0 and n0 >= 1. Then we can represent f(n) as O(g(n)).

f(n) = O(g(n))

Consider the following graph drawn for the values of f(n) and C g(n) for input (n) value on X-Axis and time
required is on Y-Axis

In above graph after a particular input value n0, always C g(n) is greater than f(n) which indicates the
algorithm's upper bound.

Example

Consider the following f(n) and g(n)...

f(n) = 3n + 2
g(n) = n
If we want to represent f(n) as O(g(n)) then it must satisfy f(n) <= C g(n) for all values of C > 0 and n0>= 1

⇒3n + 2 <= C n
f(n) <= C g(n)

Above condition is always TRUE for all values of C = 4 and n >= 2.

By using Big - Oh notation we can represent the time complexity as follows...
3n + 2 = O(n)
Big - Omege Notation (Ω)

Big - Omega notation is used to define the lower bound of an algorithm in terms of Time Complexity.
That means Big-Omega notation always indicates the minimum time required by an algorithm for all input
values. That means Big-Omega notation describes the best case of an algorithm time complexity.
Big - Omega Notation can be defined as follows...

Consider function f(n) as time complexity of an algorithm and g(n) is the most significant term. If f(n) >= C
g(n) for all n >= n0, C > 0 and n0 >= 1. Then we can represent f(n) as Ω(g(n)).

f(n) = Ω(g(n))

Consider the following graph drawn for the values of f(n) and C g(n) for input (n) value on X-Axis and time
required is on Y-Axis

In above graph after a particular input value n0, always C g(n) is less than f(n) which indicates the algorithm's
lower bound.

Example

Consider the following f(n) and g(n)...

f(n) = 3n + 2
g(n) = n
If we want to represent f(n) as Ω(g(n)) then it must satisfy f(n) >= C g(n) for all values of C > 0 and n0>= 1

⇒3n + 2 >= C n
f(n) >= C g(n)

Above condition is always TRUE for all values of C = 1 and n >= 1.

By using Big - Omega notation we can represent the time complexity as follows...
3n + 2 = Ω(n)

Big - Theta Notation (Θ)

Big - Theta notation is used to define the average bound of an algorithm in terms of Time Complexity.
That means Big - Theta notation always indicates the average time required by an algorithm for all input
 values. That means Big - Theta notation describes the average case of an algorithm time complexity.
Big - Theta Notation can be defined as follows...

Consider function f(n) as time complexity of an algorithm and g(n) is the most significant term. If C 1 g(n) <=
f(n) <= C2 g(n) for all n >= n0, C1 > 0, C2 > 0 and n0 >= 1. Then we can represent f(n) as Θ(g(n)).

f(n) = Θ(g(n))

Consider the following graph drawn for the values of f(n) and C g(n) for input (n) value on X-Axis and time
required is on Y-Axis

In above graph after a particular input value n0, always C1 g(n) is less than f(n) and C2 g(n) is greater than f(n)
which indicates the algorithm's average bound.

Example

Consider the following f(n) and g(n)...

f(n) = 3n + 2
g(n) = n
If we want to represent f(n) as Θ(g(n)) then it must satisfy C1 g(n) <= f(n) <= C2 g(n) for all values of C1 > 0,
C2 > 0 and n0>= 1

⇒C1 n <= 3n + 2 <= C2 n

C1 g(n) <= f(n) <= C2 g(n)

Above condition is always TRUE for all values of C1 = 1, C2 = 4 and n >= 2.

By using Big - Theta notation we can represent the time compexity as follows...
3n + 2 = Θ(n)

Properties of Asymptotic Notations

Prerequisite: Asymptotic Notations
Assuming f(n), g(n) and h(n) be asymptotic functions the mathematical definitions are:
1. If f(n) = Θ(g(n)), then there exists positive constants c1, c2, n0 such that 0 ≤ c1.g(n) ≤ f(n) ≤ c2.g(n), for all n ≥ n0
2. If f(n) = O(g(n)), then there exists positive constants c, n0 such that 0 ≤ f(n) ≤ c.g(n), for all n ≥ n0
3. If f(n) = Ω(g(n)), then there exists positive constants c, n0 such that 0 ≤ c.g(n) ≤ f(n), for all n ≥ n0
4. If f(n) = o(g(n)), then there exists positive constants c, n0 such that 0 ≤ f(n) < c.g(n), for all n ≥ n0
5. If f(n) = ω(g(n)), then there exists positive constants c, n0 such that 0 ≤ c.g(n) < f(n), for all n ≥ n0
Properties:
1. Reflexivity:
If f(n) is given then
f(n) = O(f(n))

If f(n) = n3 ⇒ O(n3)
Example:

Similarly,
f(n) = Ω(f(n))
f(n) = Θ(f(n))
2. Symmetry:
f(n) = Θ(g(n)) if and only if g(n) = Θ(f(n))
Example:
If f(n) = n2 and g(n) = n2 then f(n) = Θ(n2) and g(n) = Θ(n2)
Proof:
1.


f(n) = Θ(g(n)) ⇒ g(n) = Θ(f(n))
Necessary part:

⇒ g(n) ≤ (1/c1).f(n) and g(n) ≥ (1/c2).f(n)

By the definition of Θ, there exists positive constants c1, c2, no such that c1.g(n) ≤ f(n) ≤ c2.g(n) for all n ≥ no

⇒ (1/c2).f(n) ≤ g(n) ≤ (1/c1).f(n)

Since c1 and c2 are positive constants, 1/c1 and 1/c2 are well defined. Therefore, by the definition of Θ, g(n) = Θ(f(n))
 Sufficiency part:
g(n) = Θ(f(n)) ⇒ f(n) = Θ(g(n))

⇒ f(n) ≤ (1/c1).g(n) and f(n) ≥ (1/c2).g(n)

By the definition of Θ, there exists positive constants c1, c2, no such that c1.f(n) ≤ g(n) ≤ c2.f(n) for all n ≥ no

⇒ (1/c2).g(n) ≤ f(n) ≤ (1/c1).g(n)

By the definition of Theta(Θ), f(n) = Θ(g(n))

f(n) = O(g(n)) and g(n) = O(h(n)) ⇒ f(n) = O(h(n))

2. Transistivity:

Example:

⇒ n is O(n2) and n2 is O(n3) then n is O(n3)

If f(n) = n, g(n) = n2 and h(n) = n3

f(n) = O(g(n)) and g(n) = O(h(n)) ⇒ f(n) = O(h(n))

Proof:

⇒ f(n) ≤ c1.g(n)
By the definition of Big-Oh(O), there exists positive constants c, no such that f(n) ≤ c.g(n) for all n ≥ no

⇒ g(n) ≤ c2.h(n)
⇒ f(n) ≤ c1.c2h(n)
⇒ f(n) ≤ c.h(n), where, c = c1.c2 By the definition, f(n) = O(h(n))

f(n) = Θ(g(n)) and g(n) = Θ(h(n)) ⇒ f(n) = Θ(h(n))

Similarly,

f(n) = Ω(g(n)) and g(n) = Ω(h(n)) ⇒ f(n) = Ω(h(n))

f(n) = o(g(n)) and g(n) = o(h(n)) ⇒ f(n) = o(h(n))
f(n) = ω(g(n)) and g(n) = ω(h(n)) ⇒ f(n) = ω(h(n))
3. Transpose Symmetry:
f(n) = O(g(n)) if and only if g(n) = Ω(f(n))
Example:
If f(n) = n and g(n) = n2 then n is O(n2) and n2 is Ω(n)
Proof:
 Necessary part:
f(n) = O(g(n)) ⇒ g(n) = Ω(f(n))
By the definition of Big-Oh (O) ⇒ f(n) ≤ c.g(n) for some positive constant c ⇒ g(n) ≥ (1/c).f(n)
By the definition of Omega (Ω), g(n) = Ω(f(n))
 Sufficiency part:
g(n) = Ω(f(n)) ⇒ f(n) = O(g(n))
By the definition of Omega (Ω), for some positive constant c ⇒ g(n) ≥ c.f(n) ⇒ f(n) ≤ (1/c).g(n)
By the definition of Big-Oh(O), f(n) = O(g(n))
Similarly,
 f(n) = o(g(n)) if and only if g(n) = ω(f(n))
4. Since these properties hold for asymptotic notations, analogies can be drawn between functions f(n) and g(n) and two real numbers a and b.
 g(n) = O(f(n)) is similar to a ≤ b
 g(n) = Ω(f(n)) is similar to a ≥ b
 g(n) = Θ(f(n)) is similar to a = b
 g(n) = o(f(n)) is similar to a < b
 g(n) = ω(f(n)) is similar to a > b
5. Observations:
max(f(n), g(n)) = Θ(f(n) + g(n))

Without loss of generality, assume f(n) ≤ g(n), ⇒ max(f(n), g(n)) = g(n)

Proof:

⇒ g(n) ≤ max(f(n), g(n)) ≤ f(n) + g(n)

Consider, g(n) ≤ max(f(n), g(n)) ≤ g(n)

⇒ g(n)/2 + g(n)/2 ≤ max(f(n), g(n)) ≤ f(n) + g(n)

⇒ f(n)/2 + g(n)/2 ≤ max(f(n), g(n)) ≤ f(n) + g(n)

From what we assumed, we can write

⇒ (f(n) + g(n))/2 ≤ max(f(n), g(n)) ≤ f(n) + g(n)

By the definition of Θ, max(f(n), g(n)) = Θ(f(n) + g(n))
6. O(f(n)) + O(g(n)) = O(max(f(n), g(n)))
Proof:

⇒ O(f(n)) + O(g(n)) = c1.f(n) + c2.g(n)

Without loss of generality, assume f(n) ≤ g(n)

From what we assumed, we can write

O(f(n)) + O(g(n)) ≤ c1.g(n) + c2.g(n)
≤ (c1 + c2) g(n)
≤ c.g(n)
≤ c.max(f(n), g(n))
By the definition of Big-Oh(O),
O(f(n)) + O(g(n)) = O(max(f(n), g(n)))

1. If lim n→∞ f(n)/g(n) = c, c ∈ R+ then f(n) = Θ(g(n))

Note:

2. If lim n→∞ f(n)/g(n) ≤ c, c ∈ R (c can be 0) then f(n) = O(g(n))

4. If lim n→∞ f(n)/g(n) ≥ c, c ∈ R (c can be ∞) then f(n) = Ω(g(n))

3. If lim n→∞ f(n)/g(n) = 0, then f(n) = O(g(n)) and g(n) = O(f(n))

5. If lim n→∞ f(n)/g(n) = ∞, then f(n) = Ω(g(n))and g(n) = Ω(f(n))

Linear & Non-Linear Data Structures

What is Data Structure?
Whenever we want to work with a large amount of data, then organizing that data is very important. If that data is not organized effectively, it is very difficult to perform any task on that
data. If it is organized effectively then any operation can be performed easily on that data.
A data structure can be defined as follows...
Data structure is a method of organizing a large amount of data more efficiently so that any operation on that data becomes easy
Note -
Every data structure is used to organize the large amount of data
Every data structure follows a particular principle
The operations in data structure should not violate the basic principle of that data structure.
Based on the organizing method of data structure, data structures are divided into two types.
 Linear Data Structures
 Non - Linear Data Structures
Linear Data Structures
If a data structure organizes the data in sequential order, then that data structure is called a Linear Data Structure.
Example
1. Arrays
2. List (Linked List)
3. Stack
4. Queue
Non - Linear Data Structures
If a data structure organizes the data in random order, then that data structure is called as Non-Linear Data Structure.
Example
1. Tree
2. Graph
3. Dictionaries
4. Heaps
5. Tries, Etc.,
GREEDY ALGORITHM

An algorithm is designed to achieve optimum solution for a given problem. In greedy algorithm
approach, decisions are made from the given solution domain. As being greedy, the closest
solution that seems to provide an optimum solution is chosen.
Greedy algorithms try to find a localized optimum solution, which may eventually lead to
globally optimized solutions. However, generally greedy algorithms do not provide globally
optimized solutions.
Counting Coins
This problem is to count to a desired value by choosing the least possible coins and the greedy
approach forces the algorithm to pick the largest possible coin. If we are provided coins of ₹ 1,
2, 5 and 10 and we are asked to count ₹ 18 then the greedy procedure will be −
 1 − Select one ₹ 10 coin, the remaining count is 8
 2 − Then select one ₹ 5 coin, the remaining count is 3
 3 − Then select one ₹ 2 coin, the remaining count is 1
 4 − And finally, the selection of one ₹ 1 coins solves the problem
Though, it seems to be working fine, for this count we need to pick only 4 coins. But if we
slightly change the problem then the same approach may not be able to produce the same
optimum result.
For the currency system, where we have coins of 1, 7, 10 value, counting coins for value 18 will
be absolutely optimum but for count like 15, it may use more coins than necessary. For
example, the greedy approach will use 10 + 1 + 1 + 1 + 1 + 1, total 6 coins. Whereas the same
problem could be solved by using only 3 coins (7 + 7 + 1)
Hence, we may conclude that the greedy approach picks an immediate optimized solution and
may fail where global optimization is a major concern.

Examples
Most networking algorithms use the greedy approach. Here is a list of few of them −

 Travelling Salesman Problem

 Prim's Minimal Spanning Tree Algorithm
 Kruskal's Minimal Spanning Tree Algorithm
 Dijkstra's Minimal Spanning Tree Algorithm
 Graph - Map Coloring
 Graph - Vertex Cover
 Knapsack Problem
 Job Scheduling Problem
There are lots of similar problems that uses the greedy approach to find an optimum solution.
DIVIDE AND CONQUER ALGORITHM
In divide and conquer approach, the problem in hand, is divided into smaller sub-problems and
then each problem is solved independently. When we keep on dividing the subproblems into
even smaller sub-problems, we may eventually reach a stage where no more division is
possible. Those "atomic" smallest possible sub-problem (fractions) are solved. The solution of
all sub-problems is finally merged in order to obtain the solution of an original problem.

Broadly, we can understand divide-and-conquer approach in a three-step process.

Divide/Break
This step involves breaking the problem into smaller sub-problems. Sub-problems should
represent a part of the original problem. This step generally takes a recursive approach to
divide the problem until no sub-problem is further divisible. At this stage, sub-problems become
atomic in nature but still represent some part of the actual problem.
Conquer/Solve
This step receives a lot of smaller sub-problems to be solved. Generally, at this level, the
problems are considered 'solved' on their own.
Merge/Combine
When the smaller sub-problems are solved, this stage recursively combines them until they
formulate a solution of the original problem. This algorithmic approach works recursively and
conquer & merge steps works so close that they appear as one.
Examples
The following computer algorithms are based on divide-and-conquer programming approach
−

 Merge Sort
 Quick Sort
 Binary Search
 Strassen's Matrix Multiplication
 Closest pair (points)
There are various ways available to solve any computer problem, but the mentioned are a
good example of divide and conquer approach.
DYNAMIC PROGRAMMING
Dynamic programming approach is similar to divide and conquer in breaking down the problem
into smaller and yet smaller possible sub-problems. But unlike, divide and conquer, these sub-
problems are not solved independently. Rather, results of these smaller sub-problems are
remembered and used for similar or overlapping sub-problems.
Dynamic programming is used where we have problems, which can be divided into similar sub-
problems, so that their results can be re-used. Mostly, these algorithms are used for
optimization. Before solving the in-hand sub-problem, dynamic algorithm will try to examine the
results of the previously solved sub-problems. The solutions of sub-problems are combined in
order to achieve the best solution.
So we can say that −
 The problem should be able to be divided into smaller overlapping sub-problem.
 An optimum solution can be achieved by using an optimum solution of smaller sub-
problems.
 Dynamic algorithms use Memoization.
Comparison
In contrast to greedy algorithms, where local optimization is addressed, dynamic algorithms are
motivated for an overall optimization of the problem.
In contrast to divide and conquer algorithms, where solutions are combined to achieve an
overall solution, dynamic algorithms use the output of a smaller sub-problem and then try to
optimize a bigger sub-problem. Dynamic algorithms use Memoization to remember the output
of already solved sub-problems.

Example
The following computer problems can be solved using dynamic programming approach −

 Fibonacci number series

  Knapsack problem
 Tower of Hanoi
 All pair shortest path by Floyd-Warshall
 Shortest path by Dijkstra
 Project scheduling
Dynamic programming can be used in both top-down and bottom-up manner. And of course,
most of the times, referring to the previous solution output is cheaper than recomputing in
terms of CPU cycles.

Introduction to Linked Lists

A linked list is a linear data structure consisting of a group of nodes where each node point to the next node through a pointer. Each node is composed of data
and a reference (in other words, a link) to the next node in the sequence.

Linked lists are among the simplest and most common data structures. The principal benefits of a linked list over a conventional array
are:

 The list elements can easily be inserted or removed without reallocation or reorganization of the entire structure because the
data items need not be stored contiguously in memory. Simultaneously, an array has to be declared in the source code
before compiling and running the program.
 Linked lists allow the insertion and removal of nodes at any point in the list. They can do so with a constant number of
operations if the link to the previous node is maintained during the list traversal.

On the other hand, simple linked lists by themselves do not allow random access to the data or any form of efficient indexing. Thus,
many basic operations – such as obtaining the last node of the list, finding a node containing a given data, or locating the place
where a new node should be inserted – may require sequential scanning of most or all of the list elements.

Common Types of Linked Lists:

1. Singly Linked List
Singly linked lists contain nodes with a data field and a next field, which points to the next node in a line of nodes. The operations that can be performed on singly linked lists
include insertion, deletion, and traversal.
2. Doubly Linked List
In a doubly-linked list, each node contains, besides the next-node link, a second link field pointing to the prev node in the sequence.

An XOR-linking technique allows a doubly-linked list to be implemented using a single link field in each node.

3. Circular Linked list

In the last node of a list, the link field often contains a null reference, a special value is used to indicate the lack of further nodes. In a circular doubly linked list, the “tail” is linked back to
the “head”. The major advantage of circularly linked lists is their ability to traverse the full list beginning at any given node.

Applications of Linked Lists:

Linked lists are a dynamic data structure, which can grow and be pruned, allocating, and deallocating memory while the program is
running.

 Dynamic data structures such as stacks and queues can be implemented using a linked list and several other common
abstract data types, including lists and associative arrays.
 Many modern operating systems use doubly linked lists to maintain references to active processes, threads, and other
dynamic objects.
 A hash table may use linked lists to store the chains of items that hash to the same position in the hash table.
 A binary tree can be seen as a type of linked list where the elements are themselves linked lists of the same nature. The
result is that each node may include a reference to the first node of one or two other linked lists, which, together with their
contents, form the subtrees below that node.

Note: Unless explicitly mentioned, a singly linked list will be referred to by a list or linked list throughout our website.

Linked List
A linked list is a sequence of data structures, which are connected together via links.
Linked List is a sequence of links which contains items. Each link contains a connection to another link. Linked list is
the second most-used data structure after array. Following are the important terms to understand the concept of Linked
List.
 Link − Each link of a linked list can store a data called an element.
 Next − Each link of a linked list contains a link to the next link called Next.
 LinkedList − A Linked List contains the connection link to the first link called First.
Linked List Representation
Linked list can be visualized as a chain of nodes, where every node points to the next node.

As per the above illustration, following are the important points to be considered.

 Linked List contains a link element called first.

 Each link carries a data field(s) and a link field called next.
 Each link is linked with its next link using its next link.
 Last link carries a link as null to mark the end of the list.

Types of Linked List

Following are the various types of linked list.

 Simple Linked List − Item navigation is forward only.

 Doubly Linked List − Items can be navigated forward and backward.
 Circular Linked List − Last item contains link of the first element as next and the first element has a link to the last element as previous.

Basic Operations
Following are the basic operations supported by a list.

 Insertion − Adds an element at the beginning of the list.

 Deletion − Deletes an element at the beginning of the list.
 Display − Displays the complete list.
 Search − Searches an element using the given key.
 Delete − Deletes an element using the given key.

Insertion Operation
Adding a new node in linked list is a more than one step activity. We shall learn this with diagrams here. First, create a node using the same structure and find
the location where it has to be inserted.

Imagine that we are inserting a node B (NewNode), between A (LeftNode) and C (RightNode). Then point B.next to C −
NewNode.next −> RightNode;

It should look like this −

Now, the next node at the left should point to the new node.
LeftNode.next −> NewNode;

This will put the new node in the middle of the two. The new list should look like this −

Similar steps should be taken if the node is being inserted at the beginning of the list. While inserting it at the end, the second last node of the list should point
to the new node and the new node will point to NULL.

Deletion Operation
Deletion is also a more than one step process. We shall learn with pictorial representation. First, locate the target node to be removed, by using searching
algorithms.

The left (previous) node of the target node now should point to the next node of the target node −
LeftNode.next −> TargetNode.next;
This will remove the link that was pointing to the target node. Now, using the following code, we will remove what the target node is pointing at.
TargetNode.next −> NULL;

We need to use the deleted node. We can keep that in memory otherwise we can simply deallocate memory and wipe off the target node completely.

Reverse Operation
This operation is a thorough one. We need to make the last node to be pointed by the head node and reverse the whole linked list.

First, we traverse to the end of the list. It should be pointing to NULL. Now, we shall make it point to its previous node −

We have to make sure that the last node is not the last node. So we'll have some temp node, which looks like the head node pointing to the last node. Now,
we shall make all left side nodes point to their previous nodes one by one.
Except the node (first node) pointed by the head node, all nodes should point to their predecessor, making them their new successor. The first node will point
to NULL.

We'll make the head node point to the new first node by using the temp node.

Difference between Linked List and Arrays

Arrays and Linked Lists are both linear data structures, but both have some advantages and disadvantages over each other. Now let us look at the difference
between arrays and linked list.
For example, an array is a datatype which is widely implemented as a default type, in most of the modern programming languages, which are used to store
data of similar type. But, there are many cases, like the one where we don't know the quantity of data to be stored, for which some other data structures like
Linked Lists can be used.

Arrays Vs Linked Lists

The following are some of the differences between Arrays and Linked Lists:

Arrays Linked Lists

An array is a collection of elements Linked List is an ordered collection

of a similar data type. of elements of the same type in
 which each element is connected
to the next using pointers.

Random accessing is not possible

Array elements can be accessed
in linked lists. The elements will
randomly using the array index.
have to be accessed sequentially.

New elements can be stored

Data elements are stored in anywhere and a reference is
contiguous locations in memory. created for the new element using
pointers.

Insertion and Deletion operations

Insertion and Deletion operations
are costlier since the memory
are fast and easy in a linked list.
locations are consecutive and fixed.

Memory is allocated during the Memory is allocated during the

compile time (Static memory run-time (Dynamic memory
allocation). allocation).

Size of the array must be specified at Size of a Linked list grows/shrinks

the time of array as and when new elements are
declaration/initialization. inserted/deleted.

Advantages of Linked Lists

 Size of linked lists is not fixed, they can expand and shrink during run time.
 Insertion and Deletion Operations are fast and easier in Linked Lists.
 Memory allocation is done during run-time (no need to allocate any fixed memory).
 Data Structures like Stacks, Queues, and trees can be easily implemented using Linked list.

Disadvantages of Linked Lists

 Memory consumption is more in Linked Lists when compared to arrays. Because each node contains a pointer in linked list and it requires extra
memory.
 Elements cannot be accessed at random in linked lists.
 Traversing from reverse is not possible in singly linked lists.

Applications of Linked Lists

 Linked Lists can be used to implement Stacks, Queues and Trees.
 Linked Lists can be also used to implement Graphs. (Adjacency list representation of Graph).
 Linked Lists can be used to Implement Hash Tables - Each Bucket of the hash table can be a linked list (Open chain hashing).