Scribd
Upload
2 views87 pages

CS1010S-Lec-03+Recursion%2C+Iteration+%26+Order+of+Growth

The document is a lecture on recursion, iteration, and order of growth for a programming methodology course. It covers key concepts such as divide and conquer techniques, the differences between recursion and iteration, and the importance of understanding time and space complexity. Additionally, it emphasizes ethical behavior regarding course materials and includes examples of recursive functions like factorial and Fibonacci.

Uploaded by

gnnbbmmwrr
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
2 views87 pages

CS1010S-Lec-03+Recursion%2C+Iteration+%26+Order+of+Growth

The document is a lecture on recursion, iteration, and order of growth for a programming methodology course. It covers key concepts such as divide and conquer techniques, the differences between recursion and iteration, and the importance of understanding time and space complexity. Additionally, it emphasizes ethical behavior regarding course materials and includes examples of recursive functions like factorial and Fibonacci.

Uploaded by

gnnbbmmwrr
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 87

CS1010S Programming Methodology

Lecture 3
Recursion, Iteration &
Order of Growth
25 Jan 2023
Expected Level Progression
Difficulty Level
Don’t pay for
notes/exam
papers!
NUS Course Material
Ethical Behaviour and Respecting Copyright

All course participants (including permitted guest students) who have access to the course
materials on LumiNUS or any approved platforms by NUS for delivery of NUS modules are not
allowed to re-distribute the contents in any forms to third parties without the explicit consent
from the module instructors or authorized NUS officials
Examples of what is disallowed
• No posting on any websites (except otherwise explicitly allowed)
• No selling of material
• No sharing of questions/answers which could lead to cheating/plagiarism
Done with all the
missions?
Got a lot of time to burn?
Optional Trainings
Contests
Due 5 Feb 2023
Winning: 400 EXP
Participation: 50 EXP
Recap

Inputs Function Output


Learning Outcomes
After this lesson, you should be able to
• know how apply divide and conquer technique to solve a
problem
• differentiate what is recursion and iteration
• state the order of growth in terms of time and space for
small programs
Divide and Conquer
Problem

Sub- Sub-
problem 1 problem 2

Key Insight:
Sub-sub Sub-sub
problem 3 problem 4
Smaller problems
are easier to solve
f calling
itself!

When to use recursion?

Smaller sub-problem(s) has same


structure as the parent
Example
Consider the factorial function:
𝑛! = 𝑛 × 𝑛 − 1 × (𝑛 − 2) ⋯ × 1

Rewrite:
𝑛 × 𝑛−1 !, 𝑛 >1
𝑛! = ቊ
1, 𝑛=1
Factorial
𝑛 ×𝑓 𝑛−1 , 𝑛>1
𝑓(𝑛) = ቊ
1, 𝑛=1

def factorial(n):
if n == 1:
return 1
else:
return n * factorial(n – 1)
Recursion
def factorial(n):
if n == 1: terminating condition
return 1 (base case)
else:
return n * factorial(n – 1)

recursive call
Function that calls itself is called a recursive function
Recursive Process
factorial(5)
5 * factorial(4)
5 * (4 * factorial(3))
5 * (4 * (3 * factorial(2))
5 * (4 * (3 * (2 * factorial(1)))
5 * (4 * (3 * (2 * 1)))
5 * (4 * (3 * 2))
5 * (4 * 6)
5 * 24
120
n computed with

n-1 computed with

n-2 computed with

2 computed with

Terminating
1
condition
(trivial answer!)
Ingredients for recursion
1. Base Case
Does involve a recursive call!
Has trivial answer

2. Recursive Step
Connects solution at to its subproblem!
Fibonacci Numbers
Leonardo Pisano Fibonacci (12th century) is credited for
the sequence:

0, 1, 1, 2, 3, 5, 8, 13, 21, …

Note: each number is the sum of the previous two.


Fibonacci in math
0, 𝑛=0
𝑓(𝑛) = ቐ 1, 𝑛=1
𝑓 𝑛 − 1 + 𝑓(𝑛 − 2) 𝑛 > 1
Fibonacci in Python
0, 𝑛=0
𝑓(𝑛) = ቐ 1, 𝑛=1
𝑓 𝑛 − 1 + 𝑓(𝑛 − 2) 𝑛 > 1
def fib(n):
if (n == 0):
return 0
elif (n == 1):
return 1
else:
return fib(n – 1) + fib(n – 2)
Recursion tree
fib(5)
root

fib(4) fib(3)

fib(3) fib(2) fib(2) fib(1)

fib(1) fib(1) fib(0) fib(1) fib(0) 1


fib(2)

1 1 0 1 0
fib(1) fib(0)

1 0

leaves
Mutual Recursion
def ping(n): ping(6)
if (n == 0):
return n Ping! pong(5)
else: Pong! ping(4)
print("Ping!") Ping! pong(3)
pong(n - 1) Pong! ping(2)
Ping! pong(1)
def pong(n): Pong! ping(0)
if (n == 0):
return n
else:
print("Pong!")
ping(n - 1)
Factorial: Another way! res = 1
counter = 1
res = 1
res = 1 * 2
res = 1 * 2 * 3
res = 1 * 2 * 3 * 4
F
res = 1 * 2 * 3 * 4 * 5 counter <= n
…

T
def factorial(n):
res, counter = 1, 1
res = res * counter
while counter <= n:
res = res * counter
counter = counter + 1
return res counter = counter + 1
while loop while <expression>:
<body>

expression
Predicate (condition) to stay within the loop
body
Statement(s) that will be evaluated if predicate is True
Yet another way! non-inclusive.
Up to n.
def factorial(n):
res = 1
for counter in range(2, n+1):
res = res * counter
return res

def factorial(n):
res, counter = 1, 1
while counter <= n:
res = res * counter
counter = counter + 1
return res
for loop for <var> in <sequence>:
<body>

sequence
a sequence of values
var
variable that take each value in the sequence
body
statement(s) that will be evaluated for each value in the sequence
range function Examples

range([start,] stop[, step]) for i in range(10):


print(i)
• creates a sequence of integers
for i in range(3, 10):
• from start (inclusive) to stop
(non-inclusive) print(i)
• incremented by step
for i in range(3, 10, 4):
print(i)
break and continue
for j in range(10): 0
print(j) 1
Break out
if j == 3: 2 of loop
break 3
print("done") done

1
for j in range(10): 3 Continue with the
if j % 2 == 0: 5 next value!
continue 7
print(j) 9
print("done") done
Recursion
vs
Iteration
Recursive process occurs when there
are deferred operations.

Iterative process does not have


deferred operations.
Recursive Process
factorial(5)
5 * factorial(4)
5 * (4 * factorial(3))
5 * (4 * (3 * factorial(2))
5 * (4 * (3 * (2 * factorial(1)))
5 * (4 * (3 * (2 * 1)))
5 * (4 * (3 * 2))
5 * (4 * 6)
5 * 24
120 deferred
operations
Iterative Process
res counter
def factorial(n):
1 1
res, counter = 1, 1
while counter <= n: 1 2
res = (res * counter) 2 3
counter = counter + 1 6 4
return res
24 5
factorial(4)
Order of Growth
Rough measure of resources
used by a computational process
Time: how long it takes to run a
program

Space: how much memory do we


need to run the program
Which primitives require time?
All operations! (=, +, -, *, …)

Which primitives require space?


Variables!
Time
Complexity
def factorial(n): Iterative Factorial
res, counter = 1, 1
while counter <= n: c units

res = res * counter (a + b) units n times


counter = counter + 1 (a + b) units
return res

Let T(n) denote the time needed for


factorial w.r.t. the input n.
Operations Time needed (in ms)

Assignment a
𝑇 𝑛 = 2𝑎 + 𝑛 ∗ 𝑐 + 2𝑎 + 2𝑏
Arithmetic b
= 𝐴′ + 𝑛𝐵′
Relational c
def factorial(n): Recursive Factorial
if n == 1:
return 1
else: factorial(5)
return n * fact(n – 1) 5 * factorial(4)
5 * (4 * factorial(3))
5 * (4 * (3 * factorial(2))
5 * (4 * (3 * (2 * factorial(1)))
Operations Time needed (in ms) 5 * (4 * (3 * (2 * 1)))
Assignment a
5 * (4 * (3 * 2))
5 * (4 * 6)
Arithmetic b
5 * 24
Relational c 120
𝑇 𝑛 =𝑐+𝑏 +𝑇 𝑛−1 def factorial(n):
if n == 1: c units
= 𝑐 ′ + 𝑇(𝑛 − 1) return 1
else:
= 𝑐′ + 𝑐′ + 𝑇 𝑛 − 2
return n * fact(n – 1)
= 2𝑐 ′ + 𝑇 𝑛 − 2
= 2𝑐 ′ + 𝑐 ′ + 𝑇(𝑛 − 3) b units

= 3𝑐 ′ + 𝑇(𝑛 − 3)
…
= 𝑛 − 1 𝑐′ + 𝑇 𝑛 − 𝑛 − 1
= 𝑛 − 1 𝑐 ′ + 𝑇(1)
= 𝑛𝑐 ′ − 𝑐 ′ = 𝐴′ + 𝑛𝐵′
Space Complexity
def factorial(n):
res, counter = 1, 1
while counter <= n:
Iterative Factorial
res = res * counter
counter = counter + 1
return res res counter n
factorial(4)

res counter
1 1 Let S(n) denote the space
1 2
needed for factorial w.r.t.
the input n.
2 3
6 4
S(n) = 3 * const
24 5
def factorial(n):
if n == 1:
return 1
Recursive Factorial
else:
return n * fact(n – 1)
1
factorial(4)
1 the space
Let S(n) denote
needed for factorial w.r.t.
factorial(4) the input n.2
= 4 * factorial(3)
= 4 * 3 * factorial(2) 2
= 4 * 3 * 2 * factorial(1) S(n) = n * const
= 4 * 3 * 2 * (1) 3
= 4 * 3 * (2)
= 4 * (6) 6
= 24 24
4
But we are interested in –
- How time requirement grows with input!
- How space requirement grows with input!

All constants depend on CPU!

𝑇𝑖𝑡𝑒𝑟𝑎𝑡𝑖𝑣𝑒 (𝑛) = 𝐴′ + 𝑛𝐵′ 𝑆𝑖𝑡𝑒𝑟𝑎𝑡𝑖𝑣𝑒 𝑛 = 3 ∗ 𝑐𝑜𝑛𝑠𝑡


𝑇𝑟𝑒𝑐𝑢𝑟𝑠𝑖𝑣𝑒 (𝑛) = 𝐴′ + 𝑛𝐵′ 𝑆𝑟𝑒𝑐𝑢𝑟𝑠𝑖𝑣𝑒 𝑛 = 𝑛 ∗ 𝑐𝑜𝑛𝑠𝑡
Order of Growth 𝑓 𝑛
Let n denote size of the problem.
Let 𝑅 𝑛 denote the resources
needed. 𝑅 𝑛

Big-O Notation
𝑅 𝑛 has order of growth
O(𝑓 𝑛 ) written
𝑅 𝑛 = O (𝑓 𝑛 )
If there are positive constants 𝑘
and 𝑛0 such that
𝑅 𝑛 ≤𝑘𝑓 𝑛 ∀𝑛 ≥ 𝑛0 𝑛0 𝑛
Suppose a computation 𝐶 takes 3𝑛 + 5 steps
to complete, what is the order of growth?

O(3𝑛 + 5) = O(𝑛)

Since, 3𝑛 + 5 ≤ 100𝑛, ∀𝑛 ≥ 1
Let’s assume that:
𝑛 is doubled (i.e. increased to 2𝑛)

If 𝑅 𝑛 ∈ 𝑂 𝑛 then, resources double.


2
If 𝑅 𝑛 ∈ 𝑂 𝑛 then, resources quadruple.
Some common functions
• 1
• log 𝑛
• 𝑛
• 𝑛2
• 𝑛3
• 𝑛 log 𝑛
• 2 𝑛
𝑛 𝑙𝑜𝑔 𝑛 𝑛 𝑙𝑜𝑔 𝑛 𝑛2 𝑛3 2𝑛
1 0 0 1 1 2
2 0.69 1.38 4 8 4
3 1.098 3.29 9 27 8
10 2.3 23.0 100 1000 1024
20 2.99 59.9 400 8000 106
30 3.4 109 900 27000 109
100 4.6 461 10000 106 1.2 × 1030
200 5.29 1060 40000 8 × 106 1.6 × 1060
300 5.7 1710 90000 27 × 106 2.03 × 1090
1000 6.9 6910 106 109 1.07 × 10301
2000 7.6 15200 4 × 106 8 × 109 ?
3000 8 24019 9 × 106 27 × 109 ?
106 13.8 13.8 × 106 1012 1018 ?

13.7 billion years ≈ 259 seconds


Take the largest term.
Drop the constants.
Another example
How about 3𝑛 + 4𝑛2 + 4?

Order of growth
= O 3𝑛 + 4𝑛2 + 4
= O(3𝑛)
Tips
• Identify dominant terms, ignore smaller terms
• Ignore additive or multiplicative constants
- 4𝑛2 – 1000𝑛 + 300000 = O(𝑛2)
𝑛
- + 200𝑛 log 𝑛 = O(𝑛 log 𝑛)
7

log𝑐 𝑏
Note. log 𝑎 𝑏 =
log𝑐 𝑎
Base is not important!
More tricks in CS1231,
CS3230
Some involve sophisticated proofs
For now…
Count the number of “basic computational
steps”.
- Identify the basic computation steps
- Try a few small values of n
- Extrapolate for really large n
- Look for “worst case” scenario
Moral of the story
Different ways of performing a computation
(algorithms) can consume dramatically
different amounts of resources.
More Examples
Greatest Common Divisor

The GCD of two numbers 𝑎 and 𝑏, is the


largest positive integer that divides both 𝑎
and 𝑏 without remainder.
Greatest Common Divisor

Naïve Solution.
1. Set GCD to 1.
2. Start with 1.
3. Check for every number if it divides and b. If it
does, set it as GCD.
4. Continue until you reach a or b.
Greatest Common Divisor
Euclid’s Algorithm.

Given two numbers a and b, where


𝑎 = 𝑏𝑄 + 𝑟 (the remainder of the division).
Then we have,

𝐺𝐶𝐷 𝑎, 𝑏 = 𝐺𝐶𝐷 𝑏, 𝑟 , ∀𝑎, 𝑏 > 0


𝐺𝐶𝐷(𝑎, 0) = 𝑎
Greatest Common Divisor
def gcd(a, b): 𝐺𝐶𝐷 𝑎, 𝑏 = 𝐺𝐶𝐷 𝑏, 𝑟 , ∀ 𝑎, 𝑏
if (b == 0): 𝐺𝐶𝐷(𝑎, 0) = 𝑎
return a
else:
return gcd(b, a % b)

GCD(206,40) = GCD(40,6)
= GCD(6,4)
= GCD(4,2)
= GCD(2,0)
= 2
Another example
Towers of Hanoi

Rules.
1. Can only put one disc at a time
2. Cannot put larger disc over a smaller disc
Towers of Hanoi

Rules.
1. Can only put one disc at a time
2. Cannot put larger disc over a smaller disc
Towers of Hanoi
Towers of Hanoi
Towers of Hanoi
Towers of Hanoi
Towers of Hanoi
Towers of Hanoi
Towers of Hanoi
Towers of Hanoi
A B C

Suppose we know how to move 3


discs from A to C
Towers of Hanoi
A B C

Suppose we know how to move 3


discs from A to C
Towers of Hanoi
A B C

Claim: we can move 3 discs from


A to B. Why?
Towers of Hanoi
A B C

Claim: we can move 3 discs from


A to B. Why?
Towers of Hanoi
A B C

How to move 4 discs?


1. Move 3 discs from A to B
2. Move 1 disc from A to C
Recursion
• To move n discs from A to C (via B):
- Move n-1 discs from A to B (via C)
- Move the disc from A to C
- Move n-1 discs from B to C (via A)
Tower of Hanoi
def move_tower(size, src, dest, aux):
if size == 0:
return True
else:
move_tower(size-1, src, aux, dest)
print_move(src, dest)
move_tower(size-1, aux, dest, src)

def print_move(src, dest):


print("move top disk from ", src," to ", dest)
Another Example
Exponentiation
def power(b, e): def factorial(n):
if e == 0: if n == 0:
return 1 return 1
else: else:
return b * power(e – 1) return n * factorial(n – 1)

• Time requirement?
• Space requirement? Notice any
similarity?
Exponentiation – another angle!
power(a, 13)
1 𝑒=0
𝑒
𝑒
𝑏 =൞ (𝑏 2 )2 𝑒 is even = a * power(a, 12)
𝑏 ⋅ 𝑏 𝑒−1 𝑒 is odd = a * power(b, 6)
= a * power(c, 3)
= a * c * power(c, 2)
= a * c * power(d, 1)
= a * c * d * power(d, 0)
Fast Exponentiation
def fast_expt(b, e):
if e == 0:
return 1
elif e % 2 == 0:
return fast_expt(b*b, e//2)
else:
return b * fast_expt(b, e-1)

• Time requirement? O(log 𝑒)


• Space requirement? O(log 𝑒)
Can we do this iteratively?
Summary
• Recursion
- Solve the problem for a simple (base) case
- Express (divide) a problem into one or more
smaller similar problems

• Iteration:
- while and for loops
Summary
• Order of growth:
- Time and space requirements for
computations w.r.t the input

- Different ways of performing a computation


(algorithms) can consume dramatically
different amounts of resources.
Something to think about….
• Can you write a recursive function
sum_of_digits that will return the sum of digits of
an arbitrary positive integer?

• How about a recursive function


product_of_digits that will return the product of
the digits?
Notice a pattern?
How would you write a function that
computed the sum of square roots of the
digits of a number?

You might also like