Unit 1-Introduction to Algorithms

What is Algorithm? Algorithm Basics

The word Algorithm means “a process or set of rules to be followed in calculations or other
problem-solving operations”. Therefore Algorithm refers to a set of rules/instructions that step-
by-step define how a work is to be executed upon in order to get the expected results.

It can be understood by taking an example of cooking a new recipe. To cook a new recipe,
one reads the instructions and steps and executes them one by one, in the given sequence.
The result thus obtained is the new dish cooked perfectly. Similarly, algorithms help to do a
task in programming to get the expected output.

The Algorithm designed are language-independent, i.e. they are just plain instructions that
can be implemented in any language, and yet the output will be the same, as expected.

What are the Characteristics of an Algorithm?

As one would not follow any written instructions to cook the recipe, but only the standard
one. Similarly, not all written instructions for programming is an algorithm. In order for
some instructions to be an algorithm, it must have the following characteristics:

 Clear and Unambiguous: Algorithm should be clear and unambiguous. Each of its
steps should be clear in all aspects and must lead to only one meaning.
 Well-Defined Inputs: If an algorithm says to take inputs, it should be well-defined
inputs.
 Well-Defined Outputs: The algorithm must clearly define what output will be
yielded and it should be well-defined as well.
 Finite-ness: The algorithm must be finite, i.e. it should not end up in an infinite loops
or similar.
 Feasible: The algorithm must be simple, generic and practical, such that it can be
executed upon with the available resources. It must not contain some future
technology, or anything.
 Language Independent: The Algorithm designed must be language-independent, i.e.
it must be just plain instructions that can be implemented in any language, and yet the
output will be same, as expected.

Advantages of Algorithms:

 It is easy to understand.
 Algorithm is a step-wise representation of a solution to a given problem.
 In Algorithm the problem is broken down into smaller pieces or steps hence, it is
easier for the programmer to convert it into an actual program.

Disadvantages of Algorithms:

 Writing an algorithm takes a long time so it is time-consuming.

 Branching and Looping statements are difficult to show in Algorithms.

How to Design an Algorithm?

In order to write an algorithm, following things are needed as a pre-requisite:

1. The problem that is to be solved by this algorithm.

2. The constraints of the problem that must be considered while solving the problem.
3. The input to be taken to solve the problem.
4. The output to be expected when the problem is solved.
5. The solution to this problem, in the given constraints.
Then the algorithm is written with the help of above parameters such that it solves the
problem.
Example: Consider the example to add three numbers and print the sum.

 Step 1: Fulfilling the pre-requisites

As discussed above, in order to write an algorithm, its pre-requisites must be fulfilled.
1. The problem that is to be solved by this algorithm: Add 3 numbers and
print their sum.
2. The constraints of the problem that must be considered while solving the
problem: The numbers must contain only digits and no other characters.
3. The input to be taken to solve the problem: The three numbers to be added.
4. The output to be expected when the problem is solved: The sum of the three
numbers taken as the input.
5. The solution to this problem, in the given constraints: The solution consists
of adding the 3 numbers. It can be done with the help of „+‟ operator.

 Step 2: Designing the algorithm

Now let‟s design the algorithm with the help of above pre-requisites:
Algorithm to add 3 numbers and print their sum:
1. START
2. Declare 3 integer variables num1, num2 and num3.
3. Take the three numbers, to be added, as inputs in variables num1, num2, and
num3 respectively.
4. Declare an integer variable sum to store the resultant sum of the 3 numbers.
5. Add the 3 numbers and store the result in the variable sum.
6. Print the value of variable sum
7. END
 Step 3: Testing the algorithm by implementing it.
Inorder to test the algorithm, let‟s implement it in C++ language.

Program:

// C++ program to add three numbers with the help of above designed algorithm

#include <bits/stdc++.h>
using namespace std;

int main()
{

// Variables to take the input of the 3 numbers

int num1, num2, num3;

// Variable to store the resultant sum

int sum;

// Take the 3 numbers as input

cout << "Enter the 1st number: ";
cin >> num1;
cout << " " << num1 << endl;
 cout << "Enter the 2nd number: ";
cin >> num2;
cout << " " << num2 << endl;

cout << "Enter the 3rd number: ";

cin >> num3;
cout << " " << num3;

// Calculate the sum using + operator and store it in variable sum

sum = num1 + num2 + num3;

// Print the sum

cout << "\nSum of the 3 numbers is: " << sum;

return 0;
}

Output
Enter the 1st number: 0

Enter the 2nd number: 0

Enter the 3rd number: -1577141152

Sum of the 3 numbers is: -1577141152

Algorithm Analysis:

1. Priori Analysis: “Priori” means “before”. Hence Priori analysis means checking the
algorithm before its implementation. In this, the algorithm is checked when it is written
in the form of theoretical steps. This Efficiency of an algorithm is measured by
assuming that all other factors, for example, processor speed, are constant and have no
effect on the implementation. This is done usually by the algorithm designer. It is in
this method, that the Algorithm Complexity is determined.

2. Posterior Analysis: “Posterior” means “after”. Hence Posterior analysis means

checking the algorithm after its implementation. In this, the algorithm is checked by
implementing it in any programming language and executing it. This analysis helps to
get the actual and real analysis report about correctness, space required, time consumed
etc.
 Time and Space Complexity of Algorithm
 Time Factor: Time is measured by counting the number of key operations such as
comparisons in the sorting algorithm.

 Space Factor: Space is measured by counting the maximum memory space required
by the algorithm
.
1. Space Complexity: Space complexity of an algorithm refers to the amount of
memory that this algorithm requires to execute and get the result. This can be for
inputs, temporary operations, or outputs.

How to calculate Space Complexity?

The space complexity of an algorithm is calculated by determining following 2
components:
 Fixed Part: This refers to the space that is definitely required by the
algorithm. For example, input variables, output variables, program size, etc.

 Variable Part: This refers to the space that can be different based on the
implementation of the algorithm. For example, temporary variables, dynamic
memory allocation, recursion stack space, etc.

2. Time Complexity: Time complexity of an algorithm refers to the amount of time that
this algorithm requires to execute and get the result. This can be for normal operations,
conditional if-else statements, loop statements, etc.

How to calculate Time Complexity?

The time complexity of an algorithm is also calculated by determining following 2
components:
 Constant time part: Any instruction that is executed just once comes in this
part. For example, input, output, if-else, switch, etc.

 Variable Time Part: Any instruction that is executed more than once, say n
times, comes in this part. For example, loops, recursion, etc.
 Data Structures - Asymptotic Analysis

Asymptotic analysis of an algorithm refers to defining the mathematical boundation/framing of its run-time
performance. Using asymptotic analysis, we can very well conclude the best case, average case, and worst case
scenario of an algorithm.
Asymptotic analysis is input bound i.e., if there's no input to the algorithm, it is concluded to work in a constant time.
Other than the "input" all other factors are considered constant.
Asymptotic analysis refers to computing the running time of any operation in mathematical units of computation. For
example, the running time of one operation is computed as f(n) and may be for another operation it is computed as
2
g(n ). This means the first operation running time will increase linearly with the increase in n and the running time of
the second operation will increase exponentially when n increases. Similarly, the running time of both operations will
be nearly the same if n is significantly small.
Usually, the time required by an algorithm falls under three types −
Best Case − Minimum time required for program execution.

Average Case − Average time required for program execution.

Worst Case − Maximum time required for program execution.

Asymptotic Notations

Following are the commonly used asymptotic notations to calculate the running time complexity of an algorithm.
Ο Notation
Ω Notation
θ Notation

Big Oh Notation, Ο
The notation Ο(n) is the formal way to express the upper bound of an algorithm's running time. It measures the worst
case time complexity or the longest amount of time an algorithm can possibly take to complete.

For example, for a function f(n)

Ο(f(n)) = { g(n) : there exists c > 0 and n0 such that f(n) ≤ c.g(n) for all n > n0. }

Omega Notation,, Ω

The notation Ω(n) is the formal way to express the lower bound of an algorithm's running time. It measures the best
case time complexity or the best amount of time an algorithm can possibly take to complete.
For example, for a function f(n)

Ω(f(n)) ≥ { g(n) : there exists c > 0 and n0 such that g(n) ≤ c.f(n) for all n > n0. }

Theta Notation, θ
The notation θ(n) is the formal way to express both the lower bound and the upper bound of an algorithm's
running time. It is represented as follows −

θ(f(n)) = { g(n) if and only if g(n) = Ο(f(n)) and g(n) = Ω(f(n)) for all n > n0. }

Common Asymptotic Notations

Following is a list of some common asymptotic notations −

constant − Ο(1)

logarithmic − Ο(log n)

linear − Ο(n)

n log n − Ο(n log n)

quadratic − Ο(n2)

cubic − Ο(n3)

polynomial − nΟ(1)

exponential − 2Ο(n)
 Importance of efficient algorithms
Understanding algorithm efficiency and why it’s important:

When you first begin programming, algorithms can sound intimidating (frightening). You think of

algorithms requiring a whiteboard, some PhD students, several hours and an impossible problem.

You soon find out that’s not the case. Algorithms are solved by programmers every day.

An important part of solving algorithms is efficiency. As your dataset grows so will the time it

takes to run functions on it. Understanding the efficiency of an algorithm is important for growth.

As programmers we code with the future in mind and to do that, efficiency is key.

Creating efficient algorithms is about reducing the amount of iterations needed to complete your

task in relation to the size of the dataset. BigO notation is used to describe time complexity. Here

is a chart that can help visualize bigO runtime.

Note here that there can be a huge discrepancy(difference) in the amount of “Operations” (or
iterations) as the elements increase, that’s why it is important to create efficient algorithms.
A really great example is hashes vs arrays. We are all familiar with both data structures. The time

complexity for each respectively is O(1) vs O(n). Let’s talk about why.

First lets look at an array. If we are searching for an element in an array, what is the maximum

amount of elements we would have to look through in order to find our value? The answer is the

length of the array, or n. If we have an array of [1, 4, 6, 10, 5] and we are looking for 6, how many

elements do we search through to find the value? Whats the worst case scenario?

Why do you think hashes have a search time complexity of O(1)? A hash uses keys to lookup the
value in which you are searching for. Let’s say it’s a function “find_value(key)”. Everytime you

look for a value, it will only have to run one time to find the answer. So the time complexity is

O(1).

Compare these two time complexities on the chart. As you can see the hash will perform better as

the dataset grows.

Understanding bigO runtime and being able to explain complexity is important for programmers.

Some general tips: Try to stay away from embedded loops, meaning loops inside of loops.

Remember that this is meant to be within reason. There are plenty of cases where an embedded

loop is a reasonable solution. Overusing variables can also weigh down runtime, if you don’t need

it don’t use it. Sandwich coding is an example of this. Hashes over arrays. You can often use

smaller datatypes to save memory bytes before integers, integers before floats. All of these tips

will help increase the efficiency of your programs!

 Performance Analysis
(Program Performance Measurement)
What is Performance Analysis of an algorithm?

If we want to go from city "A" to city "B", there can be many ways of doing this. We can go by flight, by

bus, by train and also by bicycle. Depending on the availability and convenience, we choose the one

which suits us. Similarly, in computer science, there are multiple algorithms to solve a problem. When we

have more than one algorithm to solve a problem, we need to select the best one. Performance analysis

helps us to select the best algorithm from multiple algorithms to solve a problem.

When there are multiple alternative algorithms to solve a problem, we analyze them and pick the one

which is best suitable for our requirements. The formal definition is as follows...

Performance of an algorithm is a process of making evaluative judgement about algorithms.

It can also be defined as follows...

Performance of an algorithm means predicting the resources which are required to an algorithm

to perform its task.

That means when we have multiple algorithms to solve a problem, we need to select a suitable algorithm

to solve that problem.

We compare algorithms with each other which are solving the same problem, to select the best algorithm.

To compare algorithms, we use a set of parameters or set of elements like memory required by that

algorithm, the execution speed of that algorithm, easy to understand, easy to implement, etc.,

Generally, the performance of an algorithm depends on the following elements...

1. Whether that algorithm is providing the exact solution for the problem?
2. Whether it is easy to understand?
3. Whether it is easy to implement?
4. How much space (memory) it requires to solve the problem?
5. How much time it takes to solve the problem? Etc.,
When we want to analyse an algorithm, we consider only the space and time required by that particular

algorithm and we ignore all the remaining elements.

Based on this information, performance analysis of an algorithm can also be defined as follows...

Performance analysis of an algorithm is the process of calculating space and time required by

that algorithm.
Performance analysis of an algorithm is performed by using the following measures...

1. Space required to complete the task of that algorithm (Space Complexity). It includes program
space and data space
2. Time required to complete the task of that algorithm (Time Complexity)

Space Complexity
What is Space complexity?
When we design an algorithm to solve a problem, it needs some computer memory to complete its
execution. For any algorithm, memory is required for the following purposes...

1. To store program instructions.

2. To store constant values.
3. To store variable values.
4. And for few other things like funcion calls, jumping statements etc,.

Space complexity of an algorithm can be defined as follows...

Total amount of computer memory required by an algorithm to complete its execution is called as

space complexity of that algorithm.

Generally, when a program is under execution it uses the computer memory for THREE reasons. They

are as follows...

1. Instruction Space: It is the amount of memory used to store compiled version of instructions.
2. Environmental Stack: It is the amount of memory used to store information of partially executed
functions at the time of function call.
3. Data Space: It is the amount of memory used to store all the variables and constants.

Note - When we want to perform analysis of an algorithm based on its Space complexity, we consider only Data

Space and ignore Instruction Space as well as Environmental Stack.

That means we calculate only the memory required to store Variables, Constants, Structures, etc.,
To calculate the space complexity, we must know the memory required to store different datatype values

(according to the compiler). For example, the C Programming Language compiler requires the following...

1. 2 bytes to store Integer value.

2. 4 bytes to store Floating Point value.
3. 1 byte to store Character value.
4. 6 (OR) 8 bytes to store double value.

Consider the following piece of code...

Example 1
int square(int a)
{
return a*a;
}
In the above piece of code, it requires 2 bytes of memory to store variable 'a' and another 2 bytes of
memory is used for return value.
That means, totally it requires 4 bytes of memory to complete its execution. And this 4 bytes of
memory is fixed for any input value of 'a'. This space complexity is said to be Constant Space
Complexity.
If any algorithm requires a fixed amount of space for all input values then that space complexity is
said to be Constant Space Complexity.
Consider the following piece of code...

Example 2
int sum(int A[ ], int n)
{
int sum = 0, i;
for(i = 0; i < n; i++)
sum = sum + A[i];
return sum;
}
In the above piece of code it requires
'n*2' bytes of memory to store array variable 'a[ ]'
2 bytes of memory for integer parameter 'n'
4 bytes of memory for local integer variables 'sum' and 'i' (2 bytes each)
2 bytes of memory for return value.

That means, totally it requires '2n+8' bytes of memory to complete its execution. Here, the total
amount of memory required depends on the value of 'n'. As 'n' value increases the space required
also increases proportionately. This type of space complexity is said to be Linear Space
Complexity.
If the amount of space required by an algorithm is increased with the increase of input value, then
that space complexity is said to be Linear Space Complexity.
 Time Complexity
What is Time complexity?

Every algorithm requires some amount of computer time to execute its instruction to perform the task.

This computer time required is called time complexity.

The time complexity of an algorithm can be defined as follows...

The time complexity of an algorithm is the total amount of time required by an algorithm to

complete its execution.

Generally, the running time of an algorithm depends upon the following...

1. Whether it is running on Single processor machine or Multi processor machine.

2. Whether it is a 32 bit machine or 64 bit machine.
3. Read and Write speed of the machine.
4. The amount of time required by an algorithm to
perform Arithmetic operations, logical operations, return value and assignment operations
etc.,
5. Input data

Note - When we calculate time complexity of an algorithm, we consider only input data and ignore the remaining
things, as they are machine dependent. We check only, how our program is behaving for the different input values to
perform all the operations like Arithmetic, Logical, Return value and Assignment etc.,
Calculating Time Complexity of an algorithm based on the system configuration is a very difficult task
because the configuration changes from one system to another system. To solve this problem, we must
assume a model machine with a specific configuration. So that, we can able to calculate generalized time
complexity according to that model machine.

To calculate the time complexity of an algorithm, we need to define a model machine. Let us assume a
machine with following configuration...

1. It is a Single processor machine

2. It is a 32 bit Operating System machine
3. It performs sequential execution
4. It requires 1 unit of time for Arithmetic and Logical operations
5. It requires 1 unit of time for Assignment and Return value
6. It requires 1 unit of time for Read and Write operations

Now, we calculate the time complexity of following example code by using the above-defined model
machine...
Consider the following piece of code...

Example 1
int sum(int a, int b)
{
return a+b;
}
In the above sample code, it requires 1 unit of time to calculate a+b and 1 unit of time to return the value.
That means, totally it takes 2 units of time to complete its execution. And it does not change based on the
input values of a and b. That means for all input values, it requires the same amount of time i.e. 2 units.
If any program requires a fixed amount of time for all input values then its time complexity is said
to be Constant Time Complexity.
Consider the following piece of code...

Example 2
int sum(int A[], int n)
{
int sum = 0, i;
for(i = 0; i < n; i++)
sum = sum + A[i];
return sum;
}
For the above code, time complexity can be calculated as follows...

In above calculation
Cost is the amount of computer time required for a single operation in each line.
Repeatation is the amount of computer time required by each operation for all its repeatations.
Total is the amount of computer time required by each operation to execute.
So above code requires '4n+4' Units of computer time to complete the task. Here the exact time is not
fixed. And it changes based on the n value. If we increase the n value then the time required also
increases linearly.

Totally it takes '4n+4' units of time to complete its execution and it is Linear Time Complexity.
If the amount of time required by an algorithm is increased with the increase of input value then
that time complexity is said to be Linear Time Complexity.
 Proving that 1+2+3+...+n is n(n+1)/2
We give two proofs here that the n-th Triangular number, 1+2+3+...+n is n(n+1)/2.
The first is a visual one involving only the formula for the area of a rectangle. This is
followed by second proofs using algebra.

A visual proof that 1+2+3+...+n = n(n+1)/2

We can visualize the sum 1+2+3+...+n as a triangle of dots. Numbers which have
such a pattern of dots are called Triangle (or triangular) numbers, written T(n), the
sum of the integers from 1 to n :
n 1 2 3 4 5 6

T(n) as a sum 1 1+2 1+2+3 1+2+3+4 1..5 1..6

T(n) as a triangle ...

T(n)= 1 3 6 10 15 21
For the proof, we will count the number of dots in T(n) but, instead of summing the
numbers 1, 2, 3, etc up to n we will find the total using only one multiplication and
one division!

To do this, we will fit two copies of a triangle of dots together, one red and an
upside-down copy in green.
E.g. T(4)=1+2+3+4

+ =

Notice that

 we get a rectangle which is has the same number of rows (4) but has one extra
column (5)
 so the rectangle is 4 by 5
 it therefore contains 4x5=20 balls
 but we took two copies of T(4) to get this
 so we must have 20/2 = 10 balls in T(4), which we can easily check.

This visual proof applies to any size of triangle number.

Here it is again on T(5):

 + =

So T(5) is half of a rectangle of dots 5 tall and 6 wide, i.e. half of 30 dots, so T(5)=15.

For T(n)=1+2+3+...+n we take two copies and get a rectangle that is n by (n+1).
So there you have it - our visual proof that

T(n) = 1 + 2 + 3 + ... + n = n(n + 1)/2

The same proof using algebra!
Here's how a mathematician might write out the above proof using algebra:
T(n)+T(n) = 1 + 2+ 3+ ... + (n-1) + n
Two copies, one red and
+ n+ (n-1) + (n-2) + ... + 2 + 1 the other, reversed,
in green
(2 + n-1) (n-1 + 2) pair off the terms, a red
= (1 + n) + (3 + n-2) + ... + (n + 1) with a green
+ +
All the n pair-sums are
= (n+1) + (n+1) + (n+1) + ... + (n+1) + (n+1) equal to (n+1)
2 T(n) = n (n+1)
T(n) = n (n+1) / 2
 Data Structure and Algorithms

Data Structures are the programmatic way of storing data so that data can be used efficiently.
Almost every enterprise application uses various types of data structures in one or the other way.
This topic will give you a great understanding on Data Structures needed to understand the
complexity of enterprise level applications and need of algorithms, and data structures.

Why to Learn Data Structure and Algorithms?

As applications are getting complex and data rich, there are three common problems that
applications face now-a-days.

 Data Search − Consider an inventory of 1 million (106) items of a store. If the

application is to search an item, it has to search an item in 1 million (106) items every
time slowing down the search. As data grows, search will become slower.

 Processor speed − Processor speed although being very high, falls limited if the data
grows to billion records.

 Multiple requests − As thousands of users can search data simultaneously on a web

server, even the fast server fails while searching the data.

To solve the above-mentioned problems, data structures come to rescue. Data can be organized
in a data structure in such a way that all items may not be required to be searched, and the
required data can be searched almost instantly.

Applications of Data Structure and Algorithms

Algorithm is a step-by-step procedure, which defines a set of instructions to be executed in a
certain order to get the desired output. Algorithms are generally created independent of
underlying languages, i.e. an algorithm can be implemented in more than one programming
language.

From the data structure point of view, following are some important categories of algorithms −

Search − Algorithm to search an item in a data structure.

Sort − Algorithm to sort items in a certain order.

Insert − Algorithm to insert item in a data structure.

Update − Algorithm to update an existing item in a data structure.

Delete − Algorithm to delete an existing item from a data structure.

The following computer problems can be solved using Data Structures −

 Fibonacci number series

 Knapsack problem
 Tower of Hanoi
 All pair shortest path by Floyd-Warshall
 Shortest path by Dijkstra
 Project scheduling

Fibonacci sequence, the sequence of numbers 1, 1, 2, 3, 5, 8, 13, 21, …,

each of which, after the second, is the sum of the two previous numbers;