Design and Analysis

Algorithm
Dr. Rajesh I S
Assistant Professor
BMSIT&M
 Course Objectives:
This course will enable students to:
CLO1. Explain the methods of analyzing the algorithms and to analyze performance of

Algorithms.

CLO2. State algorithm’s efficiencies using asymptotic notations.

CLO3. Solve problems using algorithm design methods such as the brute force method,
greedy method, divide and conquer, decrease and conquer, transform and
conquer,
Dynamic programming, backtracking and branch and bound.

CLO4. Choose the appropriate data structure and algorithm design method for a
specified Application.
MODULE-1
Introduction: What is an Algorithm? It’s Properties. Algorithm Specification-using natural
language, using Pseudo code convention, Fundamentals of Algorithmic Problem solving,
Analysis Framework- Time efficiency and space efficiency, Worst-case, Best-case and Average
case efficiency.

Performance Analysis: Estimating Space complexity and Time complexity of algorithms.

Asymptotic Notations: Big-Oh notation (O), Omega notation (Ω), Theta notation ( ) with
examples, Basic efficiency classes, Mathematical analysis of Non-Recursive and Recursive
Algorithms with Examples problems.

Brute force design technique: Selection sort, sequential search, string matching algorithm
with complexity Analysis.
MODULE-2

Divide and Conquer: General method, Recurrence equation for divide and conquer,
solving it using Master’s theorem, Divide and Conquer algorithms and complexity
Analysis, Finding the maximum & minimum, Binary search, Merge sort, Quick sort.

Decrease and Conquer Approach: Introduction, Insertion sort, Topological Sorting. It’s
efficiency analysis.
MODULE-3
Greedy Method: General method, Coin Change Problem, Knapsack Problem, solving
Job sequencing with deadlines Problems.

Minimum cost spanning trees: Prim’s Algorithm, Kruskal’s Algorithm with

performance analysis.

Single source shortest paths: Dijkstra's Algorithm.

Optimal Tree problem: Huffman Trees and Codes.

 MODULE-4
Dynamic Programming: General method with Examples, Multistage Graphs, Knapsack
problem, Bellman-Ford Algorithm, Travelling Sales Person problem, Optimal Binary tree
method.

MODULE-5
Backtracking: General method, solution using back tracking to N-Queens problem, Sum of subsets
problem, Graph coloring, Hamiltonian cycles Problems.

Branch and Bound: Assignment Problem, Travelling Sales Person problem, 0/1 Knapsack problem

NP-Complete and NP-Hard problems: Basic concepts, non- deterministic algorithms, P, NP, NP-
Complete, and NP-Hard classes.
 Algorithm Types
There are several different types of algorithms. Several significant algorithms include:
1.Brute Force Algorithm: The first attempt to solve a problem is the brute force algorithm. When we
see an issue, the first method that springs to mind is a brute force algorithm.
2.Recursive Algorithm: Recursion is the foundation of a recursive algorithm. In this instance, an issue
is divided into multiple smaller components and repeatedly called by the same function.

3.Backtracking Algorithm: The idea is that we can build a solution step by step using recursion; if
during the process we realise that is not going to be a valid solution, then we stop computing that
solution and we return back to the step before ( backtrack ). and repeat this process until we either find
the answer or all possible solutions are looked after.
4.Searching Algorithm: Searching algorithms are used to find individual elements or collections of
components inside a given data structure. Depending on how they go about things or whatever data
structure the element has to be in, they can be of several forms.
5. Sorting Algorithm: Sorting is the process of organizing a set of facts in a certain way in accordance
with the needs. Sorting algorithms are the ones that assist in carrying out this duty. Data groupings are
often sorted using sorting algorithms in an increasing or decreasing order.
6. Hashing Algorithm: The hashing algorithm is comparable to the search algorithm in operation.
Nevertheless, they have an index with a key ID. In hashing, a key is given to a particular piece of data.

7. Divide and Conquer method: This method divides an issue into smaller problems, solves each of
those problems separately, and then combines the results to provide the overall answer. The procedure
entails the following three steps:
1. Divide
2. Solve
3. Combine
8. Greedy Algorithm: An algorithm is greedy when the path picked is regarded as the best option based
on a specific criterion without considering future consequences
9. Dynamic Programming technique: Dynamic programming is a computer programming technique
where an algorithmic problem is first broken down into sub-problems, the results are saved, and then the
sub-problems are optimized to find the overall solution
10. Randomized Algorithm: An algorithm that uses random numbers to decide what to do next
anywhere in its logic is called a Randomized Algorithm. For example, in Randomized Quick Sort, we use
a random number to pick the next pivot (or we randomly shuffle the array)..
 Course outcomes (Course Skill Set)
At the end of the course the student will be able to:
• CO1. Analyze various algorithms, state the efficiency using asymptotic
notations and mathematically represent the complexity of the algorithm.
• CO2. Explain the classes P, NP, and NP-Complete and be able to prove that
a certain problem is NP-Complete.
• CO3. Explain important algorithmic design paradigms (divide-and-
conquer, greedy method, dynamic-programming and Backtracking) and
apply when an algorithmic design situation calls for it.
• CO4. Apply an algorithm using appropriate design strategies for problem
solving.
 Question paper pattern:
•Assessment Details (both CIE and SEE)
1. Each internal test (two in a semester) to be conducted for 40 marks.
2. Average of both the test marks are scaled down to 25 marks, a minimum of ten marks is to be
scored by the student.
3. Two AATs are to be carried out in this scheme, but there is an exception for PBL (Project Based
Learning- alone can be offered as AAT). It is decided to offer PBL for this course.
4. PBL is evaluated for 25 marks, a minimum of ten marks is to be scored by the student.
5. A Minimum of 20 marks to be scored in CIE out of 50 marks (Both components put
together).
6. SEE examination for the Lab is to be conducted for 100 marks and reduced to 50.
7. Minimum of 18 marks is to be scored in SEE examination.
• Note: A total mark of 40 is to be scored by the students in this course from both CIE and
SEE together out of 100.
 Text books:

1. Anany Levitin, Introduction to the Design and Analysis of Algorithms -, Pearson

Education Limited. 3rd Edition, 2019.

2. Ellis Horowitz, Satraj Sahni and Rajasekaran, Computer Algorithms/C++,

Universities Press 2nd Edition, 2014
What is Algorithm?
 Why study Algorithm?

An algorithm is the best way to represent the solution of a particular problem in

a very simple and efficient way. If we have an algorithm for a specific problem,
then we can implement it in any programming language, meaning that
the algorithm is independent from any programming languages.
Properties of Algorithm
Input: It should take zero or more input.

Output: It should produce at least one output.

Definiteness: Each instruction is clear and

unambiguous.

Finiteness: It should terminate after a finite

time

Effectiveness: Every step in the algorithm must

be effective i.e. every step should do some work.
Need of Algorithm
1. To understand the basic idea of the problem.
2. To find an approach to solve the problem.
3. To improve the efficiency of existing techniques.
4. To understand the basic principles of designing the algorithms.
5. To compare the performance of the algorithm with respect to other techniques.
6. It is the best method of description without describing the implementation detail.
7. The Algorithm gives a clear description of requirements and goal of the problem to the designer.
8. A good design can produce a good solution.
9. To understand the flow of the problem.
10. To measure the behavior (or performance) of the methods in all cases (best cases, worst cases,
average cases)
11. With the help of an algorithm, we can also identify the resources (memory, input-output) cycles
required by the algorithm.
12. With the help of algorithm, we convert art into a science.
13. To understand the principle of designing.
 Why Analysis of Algorithms is important?

 To predict the behavior of an algorithm without implementing it on a specific computer.

 It is much more convenient to have simple measures for the efficiency of an algorithm than to
implement the algorithm and test the efficiency every time a certain parameter in the
underlying computer system changes.
 It is impossible to predict the exact behavior of an algorithm. There are too many influencing
factors.
 The analysis is thus only an approximation; it is not perfect.
 More importantly, by analyzing different algorithms, we can compare them to determine the
best one for our purpose.
 Algorithm Analysis
Algorithm analysis is an important part of computational complexity theory, which
provides theoretical estimation for the required resources of an algorithm to solve a
specific computational problem. Analysis of algorithms is the determination of the
amount of time and space resources required to execute it.

In designing of Algorithm, complexity analysis of an algorithm is an essential aspect.

Mainly, algorithmic complexity is concerned about its performance, how fast or slow
it works.

The complexity of an algorithm describes the efficiency of the algorithm in terms of

the amount of the memory required to process the data and the processing time.

Complexity of an algorithm is analyzed in two perspectives: Time and Space.

 Types of Algorithm Analysis:
1.Best case
2.Worst case
3.Average case

In the analysis of the algorithm, it generally focused on CPU (time) usage, Memory usage,
Disk usage, and Network usage. All are important, but the most concern is about the CPU time. Be
careful to differentiate between:
•Performance: How much time/memory/disk/etc. is used when a program is run. This depends
on the machine, compiler, etc. as well as the code we write.
•Complexity: How do the resource requirements of a program or algorithm scale, i.e. what
happens as the size of the problem being solved by the code gets larger

Note: Complexity affects performance but not vice-versa.

Performance analysis of an algorithm depends upon two factors i.e. amount of memory used and
amount of compute time consumed on any CPU. Formally they are notified as complexities in terms of:

• Time Complexity
It’s a function describing the amount of time required to run an algorithm in terms of the size of the
input. "Time" can mean the number of memory accesses performed, the number of comparisons between
integers, the number of times some inner loop is executed, or some other natural unit related to the
amount of real time the algorithm will take.
• Space Complexity
It’s a function describing the amount of memory an algorithm takes in terms of the size of input to the
algorithm. We often speak of "extra" memory needed, not counting the memory needed to store the input
itself.
Space complexity is sometimes ignored because the space used is minimal and/or obvious,
however sometimes it becomes as important an issue as time.
Best case: Define the input for which algorithm takes less time or minimum time. In
the best case calculate the lower bound of an algorithm. Example: In the linear
search when search data is present at the first location of large data then the best case
occurs.
Worst Case: Define the input for which algorithm takes a long time or maximum
time. In the worst calculate the upper bound of an algorithm. Example: In the linear
search when search data is not present at all then the worst case occurs.
Average case: In the average case take all random inputs and calculate the
computation time for all inputs.
And then we divide it by the total number of inputs.
Average case = all random case time / total no of case
 Asymptotic Analysis of Algorithms
The asymptotic analysis defines the mathematical foundation of an algorithm’s run
time performance.

•Asymptotic analysis is the running time of any process or algorithm in mathematical

terms.

•We can calculate the best, average, and worst-case scenarios for an algorithm by
using asymptotic analysis.
 Asymptotic notation
Execution time of an algorithm depends on the instruction set, processor speed, disk
I/O speed, etc. Hence, we estimate the efficiency of an algorithm asymptotically.

Time function of an algorithm is represented by T(n), where n is the input size.

Different types of asymptotic notations are used to represent the complexity of an

algorithm. Following asymptotic notations are used to calculate the running time
complexity of an algorithm.

•O − Big Oh
•Ω − Big omega
•θ − Big theta
•o − Little Oh
•ω − Little omega
 How to find time complexity?
Computing the real running time of a process is not feasible. The time taken for the
execution of any process is dependent on the size of the input.

For example, traversing through an array of 5 elements will take less time as compared
to traversing through an array of 500 elements. We can observe that time complexity
depends on the input size.

Hence, if the input size is ‘n’, then f(n) is a function of ‘n’ denoting the time
complexity.
 Asymptotic Analysis of algorithms (Growth of function)

Resources for an algorithm are usually expressed as a function regarding input. Often
this function is messy and complicated to work. To study Function growth efficiently,
we reduce the function down to the important part.

Let f (n) = a +bn+c

In this function, the term dominates the function that is when n gets sufficiently
large.

Dominate terms are what we are interested in reducing a function, in this; we ignore
all constants and coefficient and look at the highest order term concerning n.

Asymptotic notation: The word Asymptotic means approaching a value

 In computer science in the analysis of algorithms, considering the
performance of algorithms when applied to very large input datasets

The simplest example is a function ƒ (n) = +3n, the term 3n becomes insignificant
compared to when n is very large. The function "ƒ (n) is said to be asymptotically
equivalent to as n → ∞", and here is written symbolically as ƒ (n) ~ .

Asymptotic notations are used to write fastest and slowest possible running time
for an algorithm. These are also referred to as 'best case' and 'worst case' scenarios
respectively.

"In asymptotic notations, we derive the complexity concerning the size of the input.
(Example in terms of n)"

"These notations are important because without expanding the cost of running the
algorithm, we can estimate the complexity of the algorithms."
How Write and Analyze Algorithm
 How to Analyze Algorithm
Criteria to Analyze the algorithm

1. Time
2. Space
3. Network Usage
4. Power Usage
5. CPU Register Usage