01 Lecture One
01 Lecture One
Introduction
2
3
4
Book Title:
First Course in Algorithms Through
Puzzles
Authors:
Ryuhei Uehara
Publisher:
Springer
Edition:
2019
5
Algorithms
• Informally, an algorithm is ….
A well-defined computational procedure that takes
some value, or set of values, as input and produces
some value, or set of values, as output.
• Empirically, an algorithm is …
A tool for solving a well-specified computational
problem.
• Some definitions …
An algorithm is said to be correct if, for every input
instance, it halts with the correct output.
10
Gallery of Problems
11
Gallery of Problems
12
Gallery of Problems
13
Some Algorithms
• Matrix multiplication
algorithm
Given a sequence of
conformable matrices,
compute the most
efficient way of forming
the product of the
matrix sequence
15
Some Algorithms
16
Some Algorithms
• String matching
algorithm
Given a sequence of
characters, compute
where (if at all) a
second sequence of
characters occurs in
the first
17
Hard problems
• Usual measure of efficiency is speed
How long does an algorithm take to produce its result?
Define formally measures of efficiency
• Problems exist that, in all probability, will take a long time to
solve
Exponential complexity
NP-complete problems (nondeterministic polynomial time)
• Problems exist that are unsolvable
18
Hard problems
• NP-complete problems are interesting in and of
themselves
Some of them arise in real applications
Some of them look very similar to problems for which
efficient solutions do exist
• Not known whether NP-complete problems really are as
hard as they seem, or, perhaps, the machinery for solving
them efficiently has not been developed just yet
19
Algorithms as a
technology
• Even if computers were infinitely fast and memory was
plentiful and free
Study of algorithms still important – still need to
establish algorithm correctness
Since time and space resources are infinite, any
correct algorithm would do
• Real-world computers are fast but not infinitely so
• Memory is cheap but not unlimited
20
Efficiency
• Time and space efficiency are the goal
• Algorithms often differ dramatically in their efficiency
• Example: Two sorting algorithms
– INSERTION-SORT – time efficiency is c1n2
– MERGE-SORT – time efficiency is c1nlogn
• For which problem instances would one algorithm be
preferable to the other?
21
Efficiency
• Answer depends on several factors:
– Speed of machine performing the computation
• Internal clock speed
• Shared environment
• I/O needed by algorithm
– Quality of implementation (coding)
• Compiler optimization
• Implementation details (e.g., data structures)
– Size of problem instance
• Most stable parameter – used as independent variable
22
Efficiency
• INSERTION-SORT
Implemented by an ace programmer and run on a machine A
that performs 109 instructions per second such that time
efficiency is given by:
tA(n) = 2n2 instructions (i.e., c1=2)
• MERGE-SORT
Implemented by a novice programmer and run on a machine B
that performs 107 instructions per second such that time
efficiency is given by:
tB(n) = 50nlogn instructions (i.e., c1=50)
23
Efficiency
Machine A Machine B
Problem Size Insertion- Sort Merge- Sort
n 2n2/109 50nlogn/107
10,000 0.20 0.66
50,000 5.00 3.90
100,000 20.00 8.30
500,000 500.00 47.33
1,000,000 2,000.00 99.66
5,000,000 50,000.00 556.34
10,000,000 200,000.00 1,162.67
24
50,000,000 5,000,000.00 6,393.86
Efficiency
• Graphical comparison
Time Efficiency Comparison
10.00
8.00
Seconds
2.00
0.00
1 9 17 25 33 41 49 57 65
Size of Problem (in 1000s)
25
Algorithms vis-à-vis
other technologies
• Are algorithms really that important in the face of
dramatic advances in other technologies?
Hardware: super-fast clock speeds, parallelism,
pipelining
Graphical User Interfaces (GUI)
Object Oriented Systems
LANs and WANs
• YES! Algorithms are at the core of most technologies
used in contemporary computation
26
27