Scribd
Upload
65 views

Design and Analysis of Algorithms: Muhammad Nasir

1. The document discusses asymptotic analysis and asymptotic notation used to analyze algorithms as input size grows to infinity. It focuses on analyzing worst-case running time. 2. Common asymptotic notations discussed include O, Ω, Θ, o, and ω notation. O notation provides an upper bound, Ω notation provides a lower bound, and Θ notation indicates both upper and lower bounds that are tight. 3. Examples are provided to demonstrate how to formally prove that a function belongs to a certain asymptotic class using these notations, such as proving 12n(n-1) is Ω(n2). Common asymptotic time complexities like constant, logarithmic, linear, and exponential are also summarized

Uploaded by

Mohammad Abid Shamkani
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
65 views

Design and Analysis of Algorithms: Muhammad Nasir

1. The document discusses asymptotic analysis and asymptotic notation used to analyze algorithms as input size grows to infinity. It focuses on analyzing worst-case running time. 2. Common asymptotic notations discussed include O, Ω, Θ, o, and ω notation. O notation provides an upper bound, Ω notation provides a lower bound, and Θ notation indicates both upper and lower bounds that are tight. 3. Examples are provided to demonstrate how to formally prove that a function belongs to a certain asymptotic class using these notations, such as proving 12n(n-1) is Ω(n2). Common asymptotic time complexities like constant, logarithmic, linear, and exponential are also summarized

Uploaded by

Mohammad Abid Shamkani
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 28

Design and Analysis of

Algorithms

Lecture # 4
Muhammad Nasir
Department of Computer Science
COMSATS University Islamabad,
Lahore Campus
[email protected]
Running Time Analysis (Revisited)
• Depends upon the input size e.g. n
• Depends upon the structure of input
Complexity with input size
Asymptotic Notation
Goal: to simplify analysis by getting rid of
unneeded information (like “rounding”
1,000,001≈1,000,000)
We want to say in a formal way 3n2 ≈ n2
Asymptotic Notation
Categorize algorithms based on asymptotic growth
rate e.g. linear, quadratic, polynomial, exponential.
Ignore small constant and small inputs.
Estimate upper bound and lower bound on growth
rate of time complexity function.
Describe running time of algorithm as n grows to
infinity.
Limitations:
Not always useful for analysis on fixed-size inputs.
All results are for sufficiently large inputs.
Asymptotic Notation
Concentrate on order of growth of an
algorithm’s basic operation as the input
size grows.

Asymptotic Notations O, Ω, Θ, o, 𝜔.
Θ-notation to mean “order exactly”.
O-notation to mean “order at most”.
Ω-notation to mean “order at least”.
Little o to mean “tight upper bound”.
Little 𝜔 to mean “tight lower bound”.
Classes of Functions
Let {g(n)} denote the class (set) of functions that are
O(g(n))
We have
{n}  {n2}  {n3}  {n4}  {n5}  …
where the containment is strict

{n3}
{n2}
{n}
Asymptotic analysis - terminology
Special classes of algorithms:
• Logarithmic: O(log n)
• Linear: O(n)
• Quadratic: O(n2)
• Polynomial: O(nk), k ≥ 1
• Exponential: O(an), n > 1
Polynomial vs. exponential ?
Logarithmic vs. polynomial ?
Asymptotic Running Times
Here is a list of common asymptotic running times:
Θ(1): Constant time; can’t beat it!
Θ(logn): Binary Search (Sorted array of length n).
Θ(n): About the fastest that an algorithm can run.
Θ(nlogn): Best sorting algorithms.
Θ(𝑛2 ), Θ(𝑛3 ): Polynomial time. Acceptable when n is not to
large (𝑛 ≤ 1000).
Θ(2𝑛 ), Θ(3𝑛 ): Exponential time. Acceptable only if n is small,
e.g., 𝑛 ≤ 50.
Θ(𝑛!), Θ(𝑛𝑛 ): Acceptable only for really small n, e.g. 𝑛 ≤ 20.
O-notation (Informal Introduction)
O(g(n)) is the set of all functions with a lower or
same order of growth as g(n), within a constant
multiple, as n goes to infinity
n ∈ O(n2) and ½ n(n−1) ∈ O(n2)
n3 ∉ O(n2) and n4+n+1 ∉ O(n2)
O-notation (Formal Introduction)
A function t(n) is said to be of O(g(n)),
denoted t(n)∈O(g(n)), if t(n) is bounded above
by some constant multiple of g(n) for all large
n,
i.e., if there exist some positive constant
c and some non-negative integer n0 such that
t(n) ≤ cg(n) for all n ≥ n0.
O-notation (Formal Introduction)
O-notation (Formal Introduction)

• Let t(n) = 5n + 25
• find the constants c and n0 to prove
formally that t(n) ∈ O(n2)
Ω-notation (Informal Introduction)

Ω(g(n)), stands for the set of all functions with a


higher or same order of growth as g(n), within a
constant multiple, as n goes to infinity
n3 ∈ Ω(n2)
12n(n−1) ∈ Ω(n2)
100n+5∉ Ω(n2)
Ω-notation (Formal Introduction)
A function t(n) is said to be in Ω(g(n)),
denoted t(n)∈Ω(g(n)), if t(n)is bounded below
by some positive constant multiple of g(n) for
all large n,
i.e., if there exist some positive constant
c and some non-negative integer n0 such that
t(n) ≥ cg(n) for all n ≥ n0.
Ω-notation (Formal Introduction)
Ω-notation (Formal Introduction)

Let t(n) = 12n(n−1)


Find the constants c and n0 to prove formally
that t(n) ∈ Ω(n2)
Θ-notation (Informal Introduction)

Θ(g(n)) is the set of all functions that have the


same order of growth as g(n)
Θ-notation (Formal Introduction)
A function t(n) is said to be in Θ(g(n)),
denoted t(n)∈Θ(g(n)), if t(n)is bounded both
above and below by some positive constant
multiples of g(n) for all large n,
i.e., if there exist some positive
constants c1 and c2 and some nonnegative
integer n0 such that
c1 g(n) ≤ t(n) ≤ c2g(n) for all n ≥ n0.
Θ-notation (Formal Introduction)
Θ-notation (Formal Introduction)
For example, functions like

are all asymptotically equivalent. As n


becomes large, the dominant (fastest
growing) term is some
constant times n2.
Θ-notation-Example

Consider the function:

𝒇 𝒏 = 𝟖𝒏𝟐 + 𝟐𝒏 − 𝟑

Lower bound 𝒇 𝒏 = 𝟖𝒏𝟐 + 𝟐𝒏 − 𝟑 grows


asymptotically at least as fast as 𝑛2 ,
Upper bound f(n) grows no faster asymptotically
than 𝑛2 ,
Θ-notation-Example

Lower bound: f(n) grows asymptotically at least


as fast as 𝒏𝟐 . For this, need to show that there
exist positive constants 𝑐1 and 𝑛0 , such that
f(n) ≥ 𝑐1 𝑛2 for all 𝑛 ≥ 𝑛0 . Consider the reasoning:

𝒇(𝒏) = 𝟖𝒏𝟐 + 𝟐𝒏 − 𝟑 ≥ 𝟖𝒏𝟐 − 𝟑 = 𝟕𝒏𝟐 + 𝒏𝟐 − 𝟑 ≥ 𝟕𝒏𝟐

Thus c1 = 7. We implicitly assumed that 2𝑛 ≥ 0


and 𝑛2 − 3 ≥ 0 These are not true for all 𝑛 but if
𝑛 ≥ 3, then both are true. So select 𝑛0 ≥ 3.
We then have 𝑓(𝑛) ≥ 𝑐1𝑛2 for all n ≥ 𝑛0 .
Θ-notation-Example

Upper bound: f(n) grows asymptotically no faster


than 𝒏𝟐 . For this, need to show that there exist
positive constants 𝑐2 and 𝑛0 , such that f(n) ≤ 𝑐2 𝑛2
for all 𝑛 ≥ 𝑛0 . Consider the reasoning:

𝒇 𝒏 = 𝟖𝒏𝟐 + 𝟐𝒏 − 𝟑 ≤ 𝟖𝒏𝟐 + 𝟐𝒏 ≤ 𝟖𝒏𝟐 + 𝟐𝒏𝟐 = 𝟏𝟎𝒏𝟐

Thus c2 = 10. We implicitly assumed that 2𝑛 ≤ 𝟐𝒏𝟐


This is not true for all 𝑛 but if 𝑛 ≥1. So select 𝑛0 ≥
1. We then have f(n) ≤ 𝑐2 𝑛2 for all n ≥ 𝑛0 .
Θ-notation-Example

From lower bound we have 𝑛0 ≥ 3


From upper bound we have 𝑛0 ≥ 1
But we need one….
Combining the two, we let 𝑛0 be the larger of the
two: 𝑛0 ≥ 3. In conclusion, if we let c1 = 7, c2 =
10 and 𝑛0 ≥ 3, we have:
𝟕𝒏𝟐 ≤ 𝟖𝒏𝟐 + 𝟐𝒏 − 𝟑 ≤ 𝟏𝟎𝒏𝟐 for all 𝑛 ≥ 3
We established,
0 ≤ 𝑐1 𝑔 𝑛 ≤ 𝑓 𝑛 ≤ 𝑐2 𝑔(𝑛) for all 𝑛 ≥ 𝑛0
Asymptotic Notation (Other Bounds)
We established that 𝑓 𝑛 ∈ 𝑛2 .
Let’s show why f(n) is not in some other asymptotic
class. First,
Show that 𝑓(𝑛) ∉ 𝜃(𝑛). If this were true, we would
have had to satisfy both the upper and lower
bounds.
Here, the lower bound is satisfied because
𝒇 𝒏 = 𝟖𝒏𝟐 + 𝟐𝒏 − 𝟑 does grow at least as fast
asymptotically as n. But the upper bound is false.
Upper bounds requires that there exist positive
constants c2 and n0 such that f(n) ≤ 𝑐2 𝑛 for all 𝑛 ≥
𝑛0 .
Asymptotic Notation (Other Bounds)
Informally we know that 𝒇 𝒏 = 𝟖𝒏𝟐 + 𝟐𝒏 − 𝟑 will
eventually exceed 𝑐2 𝑛 no matter how large we
make c2. Let check it…
Suppose we assume that constants c2 and n0 did
exist such that 𝒇 𝒏 = 𝟖𝒏𝟐 + 𝟐𝒏 − 𝟑 ≤ 𝑐2 𝑛 for all
𝑛 ≥ 𝑛0 Since this is true for all sufficiently large n
then it must be true in the limit as n tends to
infinity. If we divide both sides by n, we have:
3
lim 8𝑛 + 2 − ≤ 𝑐2
𝑛→∞ 𝑛
The left side tends to ∞. So, no matter how large c2
is, the statement is violated. Thus 𝑓(𝑛) ∉ 𝜃(𝑛).
Asymptotic Notation (Other Bounds)
Show that 𝑓(𝑛) ∉ 𝜃(𝑛3 ).
Here, the lower bound 𝑓(𝑛) ≥ 𝑐1 𝑛3 for all 𝑛 ≥ 𝑛0 is
violated.
If we divide both sides by 𝑛3 :
8 2 3
lim + 2 − 3 ≥ 𝑐1
𝑛→∞ 𝑛 𝑛 𝑛
The left side tends to 0. The only way to satisfy this is to set c1 = 0. But
by hypothesis, c1 is positive. This means that 𝑓(𝑛) ∉ 𝜃(𝑛3 ).
Sometimes we only interested in proving one bound
upper or lower bound. In that case, O-notation and
Ω-notation is used.

You might also like