0% found this document useful (0 votes)
32 views

Lecture 1 Introduction

This document introduces algorithms and their analysis. It defines an algorithm as a well-defined computational procedure that takes inputs and produces outputs. It also defines a program as an expression of an algorithm in a programming language. The document discusses analyzing algorithms to understand their behavior, complexity, correctness, and efficiency in terms of time and space. It provides examples of sorting algorithms and analyzing their running times to determine the best approach.
Copyright
© © All Rights Reserved
Available Formats
Download as PPT, PDF, TXT or read online on Scribd
0% found this document useful (0 votes)
32 views

Lecture 1 Introduction

This document introduces algorithms and their analysis. It defines an algorithm as a well-defined computational procedure that takes inputs and produces outputs. It also defines a program as an expression of an algorithm in a programming language. The document discusses analyzing algorithms to understand their behavior, complexity, correctness, and efficiency in terms of time and space. It provides examples of sorting algorithms and analyzing their running times to determine the best approach.
Copyright
© © All Rights Reserved
Available Formats
Download as PPT, PDF, TXT or read online on Scribd
You are on page 1/ 39

Lecture 1 Introduction

What is Algorithm?
• Algorithm
– is any well-defined computational procedure that
takes some value, or set of values, as input and
produces some value, or set of values, as output.
– is thus a sequence of computational steps that
transform the input into the output.
– is a tool for solving a well - specified
computational problem.
– Any special method of solving a certain kind of pro
blem (Webster Dictionary)

Tuesday, January 10, 2023 Lecture 1 Introduction 2


What is a program?
• A program is the expression of an algorithm in
a programming language
• a set of instructions which the computer will f
ollow to solve a problem

Tuesday, January 10, 2023 Lecture 1 Introduction 3


Where We're Going (1/2)

• Learn general approaches to algorithm design


– Divide and conquer
– Greedy method
– Dynamic Programming
– Basic Search and Traversal Technique
– Graph Theory
– Linear Programming
– Approximation Algorithm
– NP Problem

Tuesday, January 10, 2023 Lecture 1 Introduction 4


Where We're Going(2/2)
• Examine methods of analyzing algorithm correct
ness and efficiency
– Recursion equations
– Lower bound techniques
– O,Omega and Theta notations for best/worst/average case analysis
• Decide whether some problems have no solution
in reasonable time
– List all permutations of n objects (takes n! steps)
– Travelling salesman problem
• Investigate memory usage as a different measure
of efficiency

Tuesday, January 10, 2023 Lecture 1 Introduction 5


Some Application
• Study problems these techniques can be appli
ed to
– sorting
– data retrieval
– network routing
– Games
– etc

Tuesday, January 10, 2023 Lecture 1 Introduction 6


The study of Algorithm

• How to devise algorithms


• How to express algorithms
• How to validate algorithms
• How to analyze algorithms
• How to test a program

Tuesday, January 10, 2023 Lecture 1 Introduction 7


Importance of Analyze Algorithm
• Need to recognize limitations of various algorithms for so
lving a problem
• Need to understand relationship between problem size a
nd running time
– When is a running program not good enough?

• Need to learn how to analyze an algorithm's running tim


e without coding it
• Need to learn techniques for writing more efficient code
• Need to recognize bottlenecks in code as well as which p
arts of code are easiest to optimize

Tuesday, January 10, 2023 Lecture 1 Introduction 8


Why do we analyze about them?
• understand their behavior, and (Job -- Selection, perfor
mance, modify)
• improve them. (Research)

Tuesday, January 10, 2023 Lecture 1 Introduction 9


What do we analyze about them?

• Correctness
– Does the input/output relation match algorithm re
quirement?
• Amount of work done (aka complexity)
– Basic operations to do task
• Amount of space used
– Memory used

Tuesday, January 10, 2023 Lecture 1 Introduction 10


What do we analyze about them?
• Simplicity, clarity
– Verification and implementation.
• Optimality
– Is it impossible to do better?

Tuesday, January 10, 2023 Lecture 1 Introduction 11


Complexity
• The complexity of an algorithm is simply the amount of
work the algorithm performs to complete its task.

Tuesday, January 10, 2023 Lecture 1 Introduction 12


What’s more to performance?

Tuesday, January 10, 2023 Lecture 1 Introduction 13


Why study algorithms and performance?

• Algorithms help us to understand scalability.


• Performance often draws the line between what is
feasible and what is impossible.
• Algorithmic mathematics provides a language for
talking about program behavior.
• Performance is the currency of computing.
• The lessons of program performance generalize to
other computing resources.
• Speed is fun!

Tuesday, January 10, 2023 Lecture 1 Introduction 14


What is the running time of this algorithm?

PUZZLE(x)
while x != 1
    if x is even
     then  x = x / 2
     else x = 3x + 1

Sample run:  7, 22, 11, 34, 17, 52, 26, 13, 40, 20,
10, 5, 16, 8, 4, 2, 1

Tuesday, January 10, 2023 Lecture 1 Introduction pp 15


The Selection Problem (1/2)
• Problem: given a group of n numbers, determi
ne the kth largest
• Algorithm 1
– Store numbers in an array
– Sort the array in descending order
– Return the number in position k

Tuesday, January 10, 2023 Lecture 1 Introduction 16


The Selection Problem(2/2)
• Algorithm 2
– Store first k numbers in an array
– Sort the array in descending order
– For each remaining number, if the number is larger than the kth numb
er, insert the number in the correct position of the array
– Return the number in position k

Which algorithm is better?

Tuesday, January 10, 2023 Lecture 1 Introduction 17


Example: What is an Algorithm?
Problem: Input is a sequence of integers stored in an array.
Output the minimum.
INPUT Algorithm

OUTPUT
instance
m:= a[1];
25, 90, 53, 23, 11, 34 for I:=2 to size of input 11
if m > a[I] then
m:=a[I];
return s

m
Data-Structure

Tuesday, January 10, 2023 Lecture 1 Introduction 18


Define Problem
• Problem:
– Description of Input-Output relationship
• Algorithm:
– A sequence of computational step that transform the
input into the output.
• Data Structure:
– An organized method of storing and retrieving data.
• Our task:
– Given a problem, design a correct and good algorithm
that solves it.

Tuesday, January 10, 2023 Lecture 1 Introduction 19


Example Algorithm A

Problem: The input is a sequence of integers stored in array.


Output the minimum.

Algorithm A

Tuesday, January 10, 2023 Lecture 1 Introduction 20


Example Algorithm B
This algorithm uses two temporary arrays.

1. copy the input a to array t1;


assign n  size of input;

2. While n > 1
For i  1 to n /2
t2[ i ]  min (t1 [ 2*i ], t1[ 2*i + 1] );
copy array t2 to t1;
n n/2;

3. Output t2[1];

Tuesday, January 10, 2023 Lecture 1 Introduction 21


Visualize Algorithm B
34 6 5 9 20 8 11 7

Loop 1

6 5 8 7

Loop 2

5 7

Loop 3

5
Tuesday, January 10, 2023 Lecture 1 Introduction 22
Example Algorithm C
Sort the input in increasing order. Return the
first element of the sorted data.

34 6 5 9 20 8 11 7

black
Sorting box

5 6 7 8 9 11 20 34

Tuesday, January 10, 2023 Lecture 1 Introduction 23


Which algorithm is better?
The algorithms are correct, but which is the best?
• Measure the running time (number of operations
needed).
• Measure the amount of memory used.
• Note that the running time of the algorithms increase
as the size of the input increases.

Tuesday, January 10, 2023 Lecture 1 Introduction 24


What do we need?

Correctness: Whether the algorithm computes


the correct solution for all instances

Efficiency: Resources needed by the algorithm

1. Time: Number of steps.


2. Space: amount of memory used.

Measurement “model”: Worst case, Average case


and Best case.

Tuesday, January 10, 2023 Lecture 1 Introduction 25


Time vs. Size of Input
Measurement
parameterized by the
size of the input.
4
Td(n)
The algorihtms A,B,C Tc (n)
are implemented and run
Running time
(second)
in a PC. 2
Tb (n)

Algorithms D is
Ta (n)
implemented and run in
a supercomputer.
0
Let Tk( n ) be the 500 1000
amount of time taken by Input Size

the Algorithm
Tuesday, January 10, 2023 Lecture 1 Introduction 26
What is Algorithm Analysis?
• How to estimate the time required for an algo
rithm
• Techniques that drastically reduce the running
time of an algorithm
• A mathemactical framwork that more rigorous
ly describes the running time of an algorithm

Tuesday, January 10, 2023 Lecture 1 Introduction pp 27


Important Question
• Is it always important to be on the most prefer
red curve?
• How much better is one curve than another?
• How do we decide which curve a particular alg
orithm lies on?
• How do we design algorithms that avoid being
on the bad curves?

Tuesday, January 10, 2023 Lecture 1 Introduction 28


Algorithm Analysis(1/5)
• Measures the efficiency of an algorithm or its implem
entation as a program as the input size becomes very
large
• We evaluate a new algorithm by comparing its perfor
mance with that of previous approaches
– Comparisons are asymtotic analyses of classes of algorith
ms
• We usually analyze the time required for an algorith
m and the space required for a datastructure

Tuesday, January 10, 2023 Lecture 1 Introduction 29


Algorithm Analysis (2/5)
• Many criteria affect the running time of an algorithm
, including
– speed of CPU, bus and peripheral hardware
– design think time, programming time and debuggi
ng time
– language used and coding efficiency of the progra
mmer
– quality of input (good, bad or average)

Tuesday, January 10, 2023 Lecture 1 Introduction 30


Algorithm Analysis (3/5)
• Programs derived from two algorithms for solving th
e same problem should both be
– Machine independent
– Language independent
– Environment independent (load on the system,...)
– Amenable to mathematical study
– Realistic

Tuesday, January 10, 2023 Lecture 1 Introduction 31


Algorithm Analysis (4/5)
• In lieu of some standard benchmark conditions unde
r which two programs can be run, we estimate the al
gorithm's performance based on the number of key a
nd basic operations it requires to process an input of
a given size
• For a given input size n we express the time T to run
the algorithm as a function T(n)
• Concept of growth rate allows us to compare running
time of two algorithms without writing two programs
and running them on the same computer

Tuesday, January 10, 2023 Lecture 1 Introduction 32


Asymptotic Performance
• In this course, we care most about asymptotic
performance
– How does the algorithm behave as the problem
size gets very large?
• Running time
• Memory/storage requirements
• Bandwidth/power requirements/logic gates/etc.

Tuesday, January 10, 2023 Lecture 1 Introduction 33


Asymptotic Notation
• By now you should have an intuitive feel for
asymptotic (big-O) notation:
– What does O(n) running time mean? O(n2)?
O(n lg n)?
– How does asymptotic running time relate to
asymptotic memory usage?
• Our first task is to define this notation more
formally and completely

Tuesday, January 10, 2023 Lecture 1 Introduction 34


Analysis of Algorithms
• Analysis is performed with respect to a
computational model
• We will usually use a generic uniprocessor
random-access machine (RAM)
– All memory equally expensive to access
– No concurrent operations
– All reasonable instructions take unit time
• Except, of course, function calls
– Constant word size
• Unless we are explicitly manipulating bits

Tuesday, January 10, 2023 Lecture 1 Introduction 35


Input Size
• Time and space complexity
– This is generally a function of the input size
• E.g., sorting, multiplication
– How we characterize input size depends:
• Sorting: number of input items
• Multiplication: total number of bits
• Graph algorithms: number of nodes & edges
• Etc

Tuesday, January 10, 2023 Lecture 1 Introduction 36


Running Time
• Number of primitive steps that are executed
– Except for time of executing a function call most
statements roughly require the same amount of
time
• y=m*x+b
• c = 5 / 9 * (t - 32 )
• z = f(x) + g(y)
• We can be more exact if need be

Tuesday, January 10, 2023 Lecture 1 Introduction 37


Analysis
• Worst case
– Provides an upper bound on running time
– An absolute guarantee
• Average case
– Provides the expected running time
– Very useful, but treat with care: what is
“average”?
• Random (equally likely) inputs
• Real-life inputs

Tuesday, January 10, 2023 Lecture 1 Introduction 38


Function of Growth rate

Tuesday, January 10, 2023 Lecture 1 Introduction 39

You might also like