Lecture 2: Big-O Notation

Time Complexity and Space

Complexity of an Algorithm

Ms. Togzhan Nurtayeva

Course Code: IT 235/A
Semester 3
Week 4-5
Date: 26.10.2023
Two criteria are used to judge algorithms:
i. Time complexity
ii. Space complexity

Time complexity of an algorithm is the amount of CPU time it needs to run completion.

Space complexity of an algorithm is the amount of memory it needs to run completion.

TIME: SPACE:
- Operations - Variables
- Comparisons - Data structures
- Loops - Allocations
- Pointer references - Function call
- Function calls to outside
Time complexity of an algorithm is the amount of time (or number of steps)
needed by a program to complete its task (to execute a particular algorithm).

The time taken for an algorithm is comprised of two times: Limits

1. Compilation time

2. Run time Better

Solutions

Less More
Time
Space
Compile Time

❖ Compilation time is the time taken to compile an algorithm

❖ While compiling it checks for the syntax ( 𝑖𝑛𝑡 → 𝑖𝑡𝑛 ) and semantic errors (𝑖𝑛𝑡 12 → 𝑖𝑛𝑡 12.5)
in the program and links it with the standard libraries

Run Time

❖ It is the time to execute the compiled program

❖ The run time of an algorithm depends on the number of instructions present in the
algorithm
❖ Note that run time is calculated only for executable statements and not for declaration
statements
 Types of Time Complexity

Time complexity of an algorithm is generally classified into three types:

1. Worst Case (Longest Time) ✓ Big Oh Notation: Upper bound

2. Average Case (Average Time) ✓ Omega Notation: Lower bound

3. Best Case (Shorter Time) ✓ Theta Notation: Tighter bound

 Standard Analysis Techniques

➢ Constant time statements

➢ Analyzing Loops
Time and Space are dependent
➢ Analyzing Nested Loops
on these analysis

➢ Analyzing Sequence of Statements

➢ Analyzing Conditional Statements

 Basic Example for Time Complexity

// Input: int A[n], array of n integers

// Output: sum of all numbers in array A

int Sum (int A[], int N){

1 1, 2, 8: Once
int s = 0;
3, 4, 5, 6, 7: Once per each iteration of for
loop, N iteration
for (int i = 0 ; i < N ; i++ )
4
2 Total: 5𝑁 + 3
s = s + A[i] ; 3

7 The complexity function of the algorithm is:

5 return s;
6 𝑓 𝑁 = 5𝑁 + 3
}
8
 Space Complexity of Algorithm

Space Complexity of a program is the amount of memory consumed by the algorithm until it completes
its execution.

The space occupied by the program is generally by the following:

1. A fixed amount of memory occupied by the space for the program i.e. data types
2. Code and space occupied by the variables used in the program.
3. A variable amount of memory occupied by the component variable whose size is dependent on the
problem being solved.
4. This space increases or decreases depending on whether the program uses iterative or recursive
procedures.

Space Complexity = Auxiliary Space + Input Space

 Types of Space Complexity

➢ Type 1: A fixed part that is a space required to store certain data and variables, that are
independent of the size of the problem.

For example, simple variables and constant used, program size, etc.

➢ Type 2: A variable part is a space required by variables, whose size depends on the size of the
problem.

For example, dynamic memory allocation, recursion stack space, etc.

Space Complexity S(P) of any algorithm P is S(P) = C + SP (I)

Algorithm: SUM (A, B)
Where,
Step 1 – Start
Step 2 – C ← A+B+10
C is the fixed part
Step 3 - Stop
S(I) is the variable part
 Other Types of Space

Instruction Space: is the space in memory occupied by the complied version of the program. We
consider this space as a constant space for any value of n. The instruction space is independent of the
size of the problem.

Data Space: is the space in memory, which used to hold the variables, data structures, allocated
memory and other data elements. The data space is related to the size of the problem.

Environment Space: is the space in memory used on the run time stack for each function call. This is
related to the run time stack and holds the returning address of the previous function. Stored return value
and pointer on it.
 Basic Example for Space Complexity

𝑇𝑦𝑝𝑒 Size 1. To store program instructions.

2. To store constant values.
bool, char, unsigned char, signed char, _int8 1 byte
3. To store variable values.
_int16, short, unsigned short, wchar_t, _wchar_t 2 bytes 4. And for few other things like
function calls, jumping
float, _int32, int, unsigned int, long, unsigned long 4 bytes
statements, etc.
double, _int64, long double, long long 8 bytes

int sum(int a[], int n)

{ {
int z = a + b + c; int x = 0;
return (z); for (int i = 0; i < n; i++)
} { 4n +12
x = x + a[i];
(4(4)+4) = 20 bytes }
return (x);
}
 Difference Between

Space Complexity Time Complexity

Space Complexity is the Time Complexity is the time
space (memory) needed required for an algorithm to
for an algorithm to solve complete its process. It
the problem. An efficient allows comparing the
algorithm take space as algorithm to check which
small as possible. one is the efficient one.
Time complexity of a program is a simple measurement of how fast the time taken by a
program grows, if the input increases.

Method 1 Method 2

𝑛−2 𝑛−2

The second method is faster. That’s why time complexity is important. In real life we want
software to be fast & smooth.
Space complexity of a program is a simple measurement of how fast the space taken by a
program grows, if the input increases.

Method 2
Method 1

𝑂(1)

𝑂(𝑛)

The second method is better. There is no point in using more space to solve a problem if, we
can do the same with lesser space complexity.
Calculating Time
complexity of
Algorithms
 Time complexity

Calculations in different Cases

1. Loop 2. Nested Loop

for (𝑖 = 1 to 𝑛){ // 𝑛 for (𝑖 = 1 to 𝑛){ // 𝑛

x = y + z; // 𝑐𝑜𝑛𝑠𝑡𝑎𝑛𝑡 𝑡𝑖𝑚𝑒 for (𝑗 = 1 to 𝑛){ // 𝑛
} x = y + z; // 𝑐𝑜𝑛𝑠𝑡𝑎𝑛𝑡 𝑡𝑖𝑚𝑒
}
}

𝑂(𝑛) 𝑂(𝑛2 )

constant time can be neglected

 Time complexity

Calculations in different Cases

3. Sequential Statements 4. If-else Statements

i) a = a+ b; // 𝑐𝑜𝑛𝑠𝑡𝑎𝑛𝑡 𝑡𝑖𝑚𝑒 = 𝑐1 if (condition){

// 𝑛
ii) for (𝑖 = 1 to 𝑛){ // 𝑛 }
x = y + z; // 𝑐𝑜𝑛𝑠𝑡𝑎𝑛𝑡 𝑡𝑖𝑚𝑒 = 𝑐2 else
} {
// 𝑛2
iii) for (𝑗 = 1 to 𝑛){ // 𝑛 }
c = d + e; // 𝑐𝑜𝑛𝑠𝑡𝑎𝑛𝑡 𝑡𝑖𝑚𝑒 = 𝑐3
}

𝑂(𝑛) = 𝑐1 + 𝑐2 𝑛 + 𝑐3𝑛 = 𝑛 𝑂(𝑛2 )

 6. Loops running n times and
incrementing/decrementing by
5. Loops running constant times constant

for (𝑖 = 1 to 𝑐) for (𝑖 = 1 ; 𝑖 ≤ 𝑛; 𝑖 = 𝑖 + 𝑐)
{ {
x = y + z; x = y + z;
} }

int 𝑖 = 1; int 𝑖 = 1;
while (𝑖 ≤ 𝑐) while (𝑖 ≤ 𝑛)
{ {
x = y + z; i = i + c;
} }
𝑂(1) 𝑂(𝑛)
 7. Loops running n times and
incrementing/decrementing by
constant factor

for (𝑖 = 1 ; 𝑖 ≤ 𝑛; 𝑖 = 𝑖 ∗ 𝑐)
{
x = y + z;
}

int 𝑖 = 1;
while (𝑖 ≤ 𝑛)
{
i = i / c;
}
𝑂(𝑛) 𝑂(𝑙𝑜𝑔𝑛)
8. Loops running n times and
incrementing by some constant
1→𝑖=2
power
2 → 𝑖 = 2𝑐
2
3 → 𝑖 = 2𝑐
for (𝑖 = 2 ; 𝑖 ≤ 𝑛; 𝑖 = 𝑝𝑜𝑤 (𝑖, 𝑐)) …
{ 𝑖
x = y + z; The loop will end when: 𝑛 = 2𝑐
} 𝑖
log2( 𝑛) = log 2 (2𝑐 )
𝑙𝑜𝑔𝑛 = 𝑐 𝑖
int 𝑖 = 2;
while (𝑖 ≤ 𝑛)
log 𝑐 𝑙𝑜𝑔𝑛 = 𝑙𝑜𝑔𝑐 𝑐 𝑖
{
𝑖 = 𝑙𝑜𝑔𝑐(𝑙𝑜𝑔𝑛)
i = pow(i , c);
}
𝑂(𝑙𝑜𝑔 𝑙𝑜𝑔𝑛 )
 Time complexity

More Examples

Algo1 () Algo2 ()
{ {
int 𝑖; int 𝑖;
for (𝑖 = 1 to 𝑛) for (𝑖 = 1 to 𝑛)
//nested loop
print (“Hello World”); for (𝑗 = 1 to 𝑛)
} print (“Hello World”);
}

𝑂(𝑛)
𝑂(𝑛2 )
 Time complexity

Algo3 () Algo4 ()
{ {
int 𝑖; int 𝑖;
for (𝑖 = 1; 𝑖 < 𝑛; 𝑖 = 𝑖 ∗ 2) for (𝑖 = 1; 𝑖 < 𝑛; 𝑖 = 𝑖/5)
print (“Hello World”); print (“Hello World”);
} }
𝑖 → 1, 2, 4, 8, 16, 32, … 𝑛
𝑂(log 5 𝑛)
20 , 21, 22 , 23, 24 , 25 … 2𝑘
Algo5 ()
𝑛 = 2𝑘 {
2𝑘 = 𝑛 int 𝑖;
for (𝑖 = 1; 𝑖 < 𝑛3 ; 𝑖 = 𝑖 ∗ 5)
𝑘 = log2 𝑛 print (“Hello World”);
}
𝑂(log 2 𝑛) 𝑂(log 5 𝑛3 )
 Time complexity

Algo6 () Algo7 ()
{ {
int 𝑖; int 𝑖 = 1, 𝑘 = 1;
for (𝑖 = 1; 𝑖 2 <= 𝑛; 𝑖 + +) while (𝑘 <= 𝑛)
print (“Hello World”); {
} 𝑖 + +; // all operations
𝑘 = 𝑘 + 𝑖; inside any loop
𝑖 2 <= 𝑛 print (“Hello World”); considered
𝑖 2 <= 𝑛 }
𝑖 <= 𝑛 }
𝑘 1 3 6 10 15 … 𝑛
𝑖 1 2 3 4 5 … 𝑧
𝑂( 𝑛) 𝑧(𝑧 + 1)
=𝑛
2

𝑧 2 + 𝑧 = 2𝑛
𝑧 = 2𝑛 = 2 ∙ 𝑛 𝑂( 𝑛)
 Time complexity

Algo8 () Algo9 ()
{ {
int 𝑖, 𝑗, 𝑘; int 𝑖 = 𝑛;
for (𝑖 = 𝑛/2; 𝑖 <= 𝑛; 𝑖 + +) // 𝑛/2 while (𝑖 > 1)
for (𝑗 = 1; 𝑗 <= 𝑛/2; 𝑗 + +) // 𝑛/2 {
for (𝑘 = 1; 𝑘 <= 𝑛; 𝑘 = 𝑘 ∗ 2) // log 2 𝑛 print (“Hello World”);
print (“Hello World”); 𝑖 = 𝑖/2; // log 2 𝑛
} }
}
𝑛 𝑛 𝑛2 log 2 𝑛
∙ ∙ log2 𝑛 =
2 2 4

𝑂(𝑛2 log 2 𝑛) 𝑂(log 2 𝑛)

 Time complexity

Algo10 ()
{
int 𝑖, 𝑗, 𝑘;
for (𝑖 = 𝑛/2; 𝑖 < 𝑛; 𝑖 + +) // 𝑛/2
for (𝑗 = 1; 𝑗 <= 𝑛; 𝑗 = 2 ∗ 𝑗) // log 2 𝑛
for (𝑘 = 1; 𝑘 <= 𝑛; 𝑘 = 𝑘 ∗ 2) // log 2 𝑛
print (“Hello World”);
}
𝑛 𝑛 (log 2 𝑛)2
∙ log 2 𝑛 ∙ log 2 𝑛 =
2 2

𝑂(𝑛 (log 2 𝑛)2 )

 Time complexity

Independent Loop Dependent Loop

Algo11 () Algo12 ()

for (𝑖 = 1 to 𝑛){ for (𝑖 = 1; 𝑖 ≤ 𝑛; 𝑖 + +){

for (k = 1 to m) for (𝑘 = 1; 𝑘 <= 𝑖; 𝑘 = 𝑘 + 1)
print (“Hello World”); print (“Hello World”);
} }

𝑖 1 𝑡𝑖𝑚𝑒 2 𝑡𝑖𝑚𝑒𝑠 3 𝑡𝑖𝑚𝑒𝑠 … 𝑛 𝑡𝑖𝑚𝑒𝑠

𝑘 1 𝑡𝑖𝑚𝑒 1, 2 𝑡𝑖𝑚𝑒𝑠 1, 2, 3 𝑡𝑖𝑚𝑒𝑠 … 1, 2, 3, … , 𝑛 𝑡𝑖𝑚𝑒𝑠

𝑂(𝑛𝑚)

𝑛 𝑛+1 𝑛2 + 𝑛
𝑇𝐶 = 1 + 2 + 3 … + 𝑛 = = ( )
2 2

𝑂(𝑛2 )
 Time complexity

𝑛 𝑛
1 𝑑𝑥
෍ ≈න = 𝑙𝑜𝑔𝑛
Algo13 () 𝑘 1 𝑥
𝑘=1

for (𝑖 = 1; 𝑖 ≤ 𝑛; 𝑖 + +){
for (𝑗 = 1; 𝑗 <= 𝑛; 𝑗 = 𝑗 + 𝑖)
𝑥 = 𝑥 + 1;
}

𝑖 1 2 3 … 𝑛
𝑗 𝑛 𝑛/2 𝑛/3 … 𝑛/𝑛

1 1 1
𝑂(𝑛𝑙𝑜𝑔𝑛) 𝑇𝐶 = 𝑛 1 + + + ⋯+ = 𝑛𝑙𝑜𝑔𝑛
2 3 𝑛
 Time complexity

𝑛(𝑛 + 1)(𝑛 + 2)
1+ 1+ 2 + 1 +2 + 3 + ⋯+ 1+ 2+ 3 +⋯+ 𝑛 =
Algo14 () 6

𝑥 = 0;
for (𝑖 = 1; 𝑖 ≤ 𝑛; 𝑖 + +){
for (𝑗 = 1; 𝑗 ≤ 𝑖; 𝑗 = 𝑗 + +)
for (k = 1; 𝑘 ≤ 𝑗; 𝑘 = 𝑘 + +)
𝑥 = 𝑥 + 1;
}

𝑖 1 2 3 … 𝑛
𝑗 1 1; 2 1; 2; 3 … 1, 2, 3, … , 𝑛
𝑘 1 1; 1, 2 1; 1, 2; 1, 2, 3 … 1; 1, 2; … 1,2, 3 … 𝑛

𝑂(𝑛3 )
 int fun(int n) int a = 0, b = 0;
{ for (i = 0; i < N; i++) {
int count = 0; a = a + rand();
for (int i = n; i > 0; i /= 2) }
for (int j = 0; j < i; j++) for ( j = 0; j < M; j++) {
count += 1; b = b + rand();
return count; }
}

𝑂(𝑁 + 𝑀)
𝑛 𝑛
𝑇 𝑛 = 𝑛 + + + ⋯ + 1 = 𝑂(𝑛)
2 4
for (int i = n; i > 0; i = i / 2) int i, j, k = 0;
{ for (i = n / 2; i <= n; i++) {
for (int j = 1; j < n; j = j * 2) for ( j = 2; j <= n; j = j * 2)
{ {
for (int k = 0; k < n; k = k + 2) k = k + n / 2;
{ }
//some logic with complexity X }
}
} 𝑂(𝑛𝑙𝑜𝑔𝑛)
}
𝑂(𝑛 (log2 𝑛)2 )
 int value = 0;
for(int i=0;i<n;i++)
for(int i=0;i<n;i++){ for(int j=0;j<i;j++)
i*=k; value += 1;
}
𝑂(𝑛2)
𝑂(log 𝑘 𝑛) int a = 0, i = N;
while (i > 0) {
a += i;
i /= 2;
}
𝑂(log 𝑛)
Calculating Space
complexity of
Algorithms
 Space complexity

Space Complexity = Auxiliary Space + Input Space y axis: size in bytes

x axis: N value

Algo1 () – Addition of two numbers

function add(n1, n2)

Constant Space (behavior)
{
sum = n1 + n2
return sum
} n1 – 4 bytes
n2 – 4 bytes
sum – 4 bytes
𝑂(1) Aux (function call, return) – 4 bytes

Total (estimated): 16 bytes = C

 Space complexity

Space Complexity = Auxiliary Space + Input Space

Algo2 () – Sum of all elements in array

function sumOfNumbers(arr[], N)
{ Linear Space Complexity
sum = 0
for (i = 0 to N)
{
sum = sum + arr[i] arr – N * 4 bytes
} sum – 4 bytes
print (sum) i – 4 bytes
} Aux (initializing for loop,
function call, return) – 4 bytes
𝑂(𝑛)
Total (estimated): 4N + 12 bytes
 Space complexity

Space Complexity = Auxiliary Space + Input Space

Algo3 () – Factorial of a number (iterative)

int fact = 1;
Constant Space (behavior)
for (int i = 1; i <= n; i++)
{
fact *= i;
}
return fact; fact – 4 bytes
n – 4 bytes
i – 4 bytes
Aux (initializing for loop,
𝑂(1) function call, return) – 4 bytes

Total (estimated): 16 bytes

 Space complexity

Space Complexity = Auxiliary Space + Input Space

Algo4 () – Factorial of a number (recursive)

factorial(n){
if (n <= 1) { 𝑂(𝑛)
return 1;
} else { Linear Space Complexity
return (n*factorial(n-1));
} 1
} assume n = 5 2*f(1)
𝑓𝑛(5) 3*f(2)
n – 4 bytes 4*f(3)
Aux (function 120 5*f(4)
call) – 5 * 4 bytes
Call Stack
Total (estimated): 4 bytes + 4*n bytes
 Space – Time Tradeoff and Efficiency

All efforts made by analyzing time and space complexity lead to the algorithm’s efficiency.

But, when we can say that an algorithm is efficient? The answer seems to be obvious: it should be
fast, and it should take the least amount of memory possible.

Unfortunately, in algorithmics, space and time are like two separate poles. Increasing speed will
most often lead to increased memory consumption and vice-versa.

On the one side, we have merge sort, which is extremely fast but requires a lot of memory. On the
other side, we have bubble sort, a slow algorithm but one that occupies minimal space. There are
also some balanced ones like in-place heap sort. Its speed and space usage are not the best, but
they’re acceptable.

Maximizing both the algorithm’s space and time complexity is impossible. We should adjust
those parameters according to our requirements and environment.
 5 Basic Sequences and Their Sums

Adding positive integers Summing up the squares

𝑛(𝑛 + 1) 𝑛(𝑛 + 1)(2𝑛 + 1)

𝑆= 𝑆=
2 6

Finding the sum of the cubes Adding up even numbers

Summing odd numbers 𝑆 = 𝑛(𝑛 + 1)

𝑛2(𝑛 + 1)2
𝑆= 𝑆 = 𝑛2
4
 What is the time and space complexity of the following codes:

a) b) c)
 What is the time and space complexity of the following codes:

a) b) c)

Time Complexity: O(n + m) Time Complexity: O(n²) Time Complexity: O(n²)

Space Complexity: O(1) Space Complexity: O(1) Space Complexity: O(1)
What is the time complexity of the following codes:

a)

b)

c)
What is the time complexity of the following codes:

a)
Time Complexity: O(log n)

b)
Time Complexity: O(log n)

c)

Time Complexity: O(nlog n)

What is the time complexity of the following code:
What is the time complexity of the following code:

Time Complexity: O(2^n)

What is the time complexity of the following code:
What is the time complexity of the following code:

Recursive Method:

Time Complexity: O(log n) Auxiliary Space: O(log n)

The iterative implementation of Binary Search:

Time Complexity: O (log n)

Auxiliary Space: O (1)
 Analysis of input size at each iteration of Binary Search:

At Iteration 1:
Also, we know that after k iterations, the length of the
Length of array = n array becomes 1 Therefore, the Length of the array:
At Iteration 2: 𝑛
=1
𝑛
2𝑘
Length of array = 2 => 𝑛 =2𝑘
At Iteration 3: Applying log function on both sides:
𝑛
𝑛
Length of array = = 2
=> log 2 𝑛 = log 2 2𝑘
2 22

Therefore, after Iteration k: => log 2 𝑛 = k ∗ log 2 2

𝑛 As (log 𝑎 𝑎 = 1) Therefore, 𝑘 = log 2 𝑛

Length of array =
2𝑘
 Self-study links

https://medium.com/@manishsakariya/time-complexity-examples-6a4877a1b923

https://www.youtube.com/watch?v=AWHi1-Xmd-Y

https://www.youtube.com/watch?v=yOb0BL-84h8&t=1107s

https://www.enjoyalgorithms.com/blog/time-complexity-analysis-of-loop-in-programming