Discrete Mathematics and

Its Applications
Lecture 3
Dr. Metwally Rashad
2020
Course Contents
# Chapter Title
1 The Foundations: Logic and Proofs
2 Basic Structures: Sets, Functions, Sequences, Sums, and Matrices
3 Algorithms
4 Number Theory and Cryptography
5 Induction and Recursion
6 Counting
7 Discrete Probability
8 Relations
9 Graphs
10 Trees
11 Boolean Algebra
12 Modeling Computation
 3.1 3.2
Concept of The Growth of
Algorithms Functions

Chapter 3
Algorithms

3.3
Complexity of
Algorithms
1/21
Concept of Algorithms
Definition 1. An algorithm is a finite sequence of precise instructions
for performing a computation or for solving a problem.
 Properties of an algorithm:
Input: an algorithm has input values from a specified set.
Output: from each set of input values an algorithm produces output values
from a specified set. The output values are the solution to the problem.
Definiteness: the steps of an algorithm must be defined precisely.
Correctness: an algorithm should produce the correct output values for each
set of input values.
Finiteness: an algorithm should produce the desired output after a finite
number of steps for input in the set.
Effectiveness: it must be possible to perform each step of an algorithm exactly
and in a finite amount of time.
Generality: the algorithm should be applicable for all problems of the desired
form not just for a particular set of input values. 2/21
Concept of Algorithms
Example Describe an algorithm for finding the largest integer in a finite
sequence of integers
Solution: We perform the following steps:

1. Set the temporary maximum equal to the first integer in the sequence
2. Compare the next integer in the sequence to the temporary maximum,
and if it is larger that the temporary maximum, set the temporary
maximum equal to this integer
3. Repeat the previous step (2) if there are more integers in the sequence
4. Stop when there are no integers left in the sequence. The temporary
maximum at this point is the largest integer in the sequence
3/21
Concept of Algorithms
Algorithm: Finding the maximum element in a finite
sequence

Procedure max(a1, a2, …, an: integer)

max := a1
For i := 2 to n
If max < ai then max := ai
{max is the largest element}
4/21
Concept of Algorithms: Searching Algorithms
Algorithm: The Linear Search Algorithm
Problem: “Locate an element x in a list of distinct elements a1, a2, …, an,

or determinelinear
Procedure that it search(x:
is not in the list.” a , a , …, a : distinct
integer, 1 2 n
integers)
i := 1
while (i  n and x  ai)
i := i + 1
if i  n then location := i
else location := 0
return location {location is the subscript of the term that
equals x, or is 0 if x is not found} 5/21
Concept of Algorithms: Searching Algorithms
 The Binary Search Algorithm
- This algorithm can be used when the list has terms occurring in order of
increasing size.
- The list is then split into two smaller sublists of the same size, or where one of

these smaller lists has one fewer term than the other.
- Is much more efficient than the linear search algorithm.
Example: To search for 19 in the list
1 2 3 5 6 7 8 10 12 13 15 16 18 19 20 22
 Split the list into 2 subsets with 8 terms each
1 2 3 5 6 7 8 10 12 13 15 16 18 19 20 22
 Compare 19 with the largest element of the first set 6/21
Concept of Algorithms: Searching Algorithms
 Split the second subset into 2 smaller subsets
12 13 15 16 18 19 20 22
 Compare 19 with 16

16 < 19  search 19 in the second set

 Split the second subset as:
18 19 20 22
 Compare 19 > 19 is false  search 19 in first set
 Split the subset as :
18 19
 Since 18 < 19  search restricted to the second list
 Finally 19 is located at the 14th element of the original list 7/21
Concept of Algorithms: Searching Algorithms
Algorithm: The Binary Search Algorithm

Procedure binary search (x: integer, a1, a2,…,an: increasing integers)

i := 1 {i is left endpoint of search interval}
j := n {j is right endpoint of search interval}
while i < j
m := (i + j)/2
If x > am then i := m + 1
else j := m
if x := ai then location := i
else location := 0
return location{location is the subscript of the term equal to x, or 0 if
x is not found}
8/21
Concept of Algorithms: Sorting Algorithms
 Sorting is putting the elements into a list in which the elements are in
increasing order.
 For instance,
- Sorting the list 7, 2, 1, 4, 5, 9 produces the list 1, 2, 4, 5, 7, 9.
- Sorting the list d, h, c, a, f produces the list a, c, d, f, h.
 More than 100 sorting algorithms have been devised, and it is surprising
how often new sorting algorithms are developed
 We will introduce two sorting algorithms, the bubble sort and the
insertion sort.

9/21
Concept of Algorithms: Sorting Algorithms
The Bubble Sort Algorithm
Problem : “Order the elements of a list”.
Example: Use the bubble sort to put 3, 2, 4, 1, 5 into increasing order.
Steps of the Bubble sort
3 2 2 2 2 2 2 2 1 1
2 3 3 3 3 3 1 1 2 2
4 4 4 1 1 1 3 3 3
3
1 1 1 4 4 4 4
4 4 4
5 5 5 5 5
5 5 5 5 5
1st pass 2nd pass 3rd pass 4th pass
= ordered = interchange 10/21
Concept of Algorithms: Sorting Algorithms
Algorithm: The Bubble Sort Algorithm

Procedure bubble sort (a1, …, an: real numbers with n ≥ 2)

for i := 1 to n-1 {count number of passes}
for j := 1 to n-i
if aj > aj+1 then interchange aj and aj+1
{a1, …, an is the increasing order}

11/21
Concept of Algorithms: Sorting Algorithms
The Insertion Sort Algorithm
Problem : “Order the elements of a list”.
Example: Use the insertion sort to put 3, 2, 4, 1, 5 in increasing order.

3 2 2 1 1 1
2 3 3 2 2 2
4 4 4 3 3 3
1 1 1 4 4 4 Sorted List
5 5 5 5
5 5

= ordered 12/21
Concept of Algorithms: Sorting Algorithms
Algorithm: The Insertion Sort Algorithm
Procedure insertion sort (a1, …, an: real numbers with n ≥ 2)
for j := 2 to n
i := 1
while aj > ai
i := i+1
m := aj
for k := 0 to j − i − 1
aj-k := aj-k-1
ai := m
{a1, …, an is the increasing order} 13/21
The Growth of Functions
 We will study the number of operations used in the algorithm
- Estimate the number of comparisons used by the linear and binary search
algorithms to find an element in a sequence of n elements.
- Estimate the number of comparisons used by the bubble sort and by the
insertion sort to sort a list of n elements
 The time required to solve a problem depends on
- The number of operations it uses (usually measured by a positive integer n).
- The hardware and software used to run the program that implements the
algorithm.
 We can estimate the growth of a function using big-O notation
 Using big-O notation, we can compare two algorithms to determine
which is more efficient. 14/21
Big-O Estimates

Definition 1. Let f and g be functions from the set of integers or the set
of real numbers to the set of real numbers. We say that the growth of
f (x) is O(g(x)) if there are constants C and k such that
|f (x)| ≤ C|g(x)|
whenever x > k. [This is read as “ f (x) is big-oh of g(x).”]

Note That:
- The constants C and k in the definition of big-O notation are called witnesses to the
relationship f (x) is O(g(x)).
- To establish that f (x) is O(g(x)) we need only one pair of witnesses to this relationship
- When there is one pair of witnesses to the relationship f (x) is O(g(x)), there are
infinitely many pairs of witnesses.
15/21
Big-O Estimates
Example1: Show that f (x) = x2 + 2x + 1 is O(x2).
Solution:
-We can estimate the size of f (x) when x > 1 because x<x2 and 1 < x2 when x > 1.
It follows that
x2+ 2x + 1 ≤ x2 + 2x2 + x2 = 4x2
whenever x > 1, we can take C = 4 and k = 1 as witnesses to show that f (x) is O(x2)

Example2: Show that f (x) = 7x2 is O(x3).

Solution:
- When x > 1, we have 7 x2 < 7x3, so that C = 7 and k = 1 are witnesses to show
that f (x) is O(x3)
- When x > 7, we have 7 x2 < x3, we can take C = 1 and k = 7 as witnesses to
establish the relationship 7 x2 is O(x3). 16/21
Big-O Estimates for Some Important Functions
Theorem 1:

Theorem 2:

Theorem 3:

17/21
Complexity of Algorithms
 Time Complexity: Determine the approximate number of operations
required to solve a problem of size n.
 Use the Big-O notation
 Count the expensive operations only
 Basic operations:
 Searching algorithms - key comparisons

 Sorting algorithms - list component comparisons

 Numerical algorithms - multiplications/divisions and/or additions/subtractions

 Space Complexity: Determine the approximate memory required to solve

a problem of size n.
 Worst Case: maximum number of operations
 Average Case: average number of operations used to solve the problem
18/21
Complexity of Algorithms
Example1: What is the worst-case complexity of the bubble sort in terms of
the number of comparisons made?
Solution:
- Bubble the largest element to the ’bottom’ by starting at the ‘top’ - swap elements until
the largest in the bottom position.
- Bubble the second largest to the position below the bottom.
- Continue until the list is sorted.
n-1 comparison on the first pass
n-2 comparisons on the second pass
.
.
1 comparison on the last pass
Worst-case complexity: (n - 1)+ (n - 2) + . . . . + 1 = (n-1)n/2 = O(n2)
19/21
Complexity of Algorithms
 Complexity of Matrix Multiplication

- We can determine the complexity of this algorithm in terms of the number

of
additions and multiplications used.
20/21
Complexity of Algorithms

Commonly Used Terminology for the Complexity of

Algorithms.
Complexity Terminology
O(1) Constant complexity
O(log n) Logarithmic complexity
O(n) Linear complexity
O(n log n) Linearithmic complexity
O(nb) Polynomial complexity
O(bn), where b > 1 Exponential complexity
O(n!)
Commonly Used Terminology forFactorial complexity
the Complexity of Algorithms.
21/21