Scribd
Upload
19 views

Cos323 s06 Lecture02 Rootfinding

1) Root finding algorithms aim to find the value where a function equals zero. 2) Bisection is a robust method that uses function values to iteratively narrow the range that brackets the root. 3) Newton's method uses the function's derivative to rapidly converge quadratically near the root when it is smooth. 4) Root finding in more than one dimension uses similar techniques but requires calculating Jacobians and inverting matrices, making the methods more complex and fragile.

Uploaded by

missionsk81
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPT, PDF, TXT or read online on Scribd
19 views

Cos323 s06 Lecture02 Rootfinding

1) Root finding algorithms aim to find the value where a function equals zero. 2) Bisection is a robust method that uses function values to iteratively narrow the range that brackets the root. 3) Newton's method uses the function's derivative to rapidly converge quadratically near the root when it is smooth. 4) Root finding in more than one dimension uses similar techniques but requires calculating Jacobians and inverting matrices, making the methods more complex and fragile.

Uploaded by

missionsk81
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPT, PDF, TXT or read online on Scribd
You are on page 1/ 23

Root Finding

COS 323
1-D Root Finding

• Given some function, find location where f(x)=0


• Need:
– Starting position x0, hopefully close to solution
– Ideally, points that bracket the root

f(x+) > 0

f ( x –) < 0
1-D Root Finding

• Given some function, find location where f(x)=0


• Need:
– Starting position x0, hopefully close to solution
– Ideally, points that bracket the root
– Well-behaved function
What Goes Wrong?

Tangent point: Singularity: Pathological case:


very difficult brackets don’t infinite number of
to find surround root roots – e.g. sin(1/x)
Example: Press et al., Numerical Recipes in C
Equation (3.0.1), p. 105
2
2 ln((Pi - x) )
f := x -> 3 x + ------------- + 1
4
Pi
> evalf(f(Pi-1e-10));
30.1360472472915692...

> evalf(f(Pi-1e-100));
25.8811536623525653...

> evalf(f(Pi-1e-600));
2.24285595777501258...

> evalf(f(Pi-1e-700));
-2.4848035831404979...

> evalf(f(Pi+1e-700));
-2.4848035831404979...

> evalf(f(Pi+1e-600));
2.24285595777501258...

> evalf(f(Pi+1e-100));
25.8811536623525653...

> evalf(f(Pi+1e-10));
30.1360472510614803...
Bisection Method

• Given points x+ and x– that bracket a root, find


xhalf = ½ (x++ x–)
and evaluate f(xhalf)
• If positive, x+  xhalf else x–  xhalf
• Stop when x+ and x– close enough
• If function is continuous, this will succeed
in finding some root
Bisection

• Very robust method


• Convergence rate:
– Error bounded by size of [x+… x–] interval
– Interval shrinks in half at each iteration
– Therefore, error cut in half at each iteration:
|n+1| = ½ |n|
– This is called “linear convergence”
– One extra bit of accuracy in x at each iteration
Faster Root-Finding

• Fancier methods get super-linear convergence


– Typical approach: model function locally by something
whose root you can find exactly
– Model didn’t match function exactly, so iterate
– In many cases, these are less safe than bisection
Secant Method

• Simple extension to bisection: interpolate or


extrapolate through two most recent points

2
3

4
1
Secant Method

• Faster than bisection:


|n+1| = const. |n|1.6
• Faster than linear: number of correct bits
multiplied by 1.6
• Drawback: the above only true if sufficiently close
to a root of a sufficiently smooth function
– Does not guarantee that root remains bracketed
False Position Method

• Similar to secant, but guarantee bracketing

2
3
4

• Stable, but linear in bad cases


Other Interpolation Strategies

• Ridders’s method: fit exponential to


f(x+), f(x–), and f(xhalf)
• Van Wijngaarden-Dekker-Brent method:
inverse quadratic fit to 3 most recent points
if within bracket, else bisection
• Both of these safe if function is nasty, but
fast (super-linear) if function is nice
Newton-Raphson

• Best-known algorithm for getting quadratic


convergence when derivative is easy to evaluate
• Another variant on the extrapolation theme

1
2
3 Slope = derivative at 1
f ( xn )
xn 1  xn 
4 f ( xn )
Newton-Raphson

• Begin with Taylor series


f ( xn ) want
f ( xn   )  f ( xn )   f ( xn )  
2
 ...  0
2
• Divide by derivative (can’t be zero!)
f ( xn ) 2 f ( xn )
  0
f ( xn ) 2 f ( xn )
2 f ( xn )
  Newton     0
2 f ( xn )
f ( xn ) 2
 Newton       n1 ~  n
2

2 f ( xn )
Newton-Raphson

• Method fragile: can easily get confused

• Good starting point critical


– Newton popular for “polishing off” a root found
approximately using a more robust method
Newton-Raphson Convergence

• Can talk about “basin of convergence”:


range of x0 for which method finds a root
• Can be extremely complex:
here’s an example
in 2-D with 4 roots
Yale site
D.W. Hyatt
Popular Example of Newton: Square Root

• Let f(x) = x2 – a: zero of this is square root of a


• f’(x) = 2x, so Newton iteration is

 
2
xn 2  a 1
xn 1  xn   2 xn  xan
2 xn

• “Divide and average” method


Reciprocal via Newton

• Division is slowest of basic operations


• On some computers, hardware divide not available
(!): simulate in software
a
b  a * b1
f ( x)  1x  b  0
f ( x)   x12 x
1
b
xn 1  xn  1  xn 2  bxn 
x
 x2

• Need only subtract and multiply


Rootfinding in >1D

• Behavior can be complex: e.g. in 2D


want
f ( x, y )  0
want
g ( x, y )  0
Rootfinding in >1D

• Can’t bracket and bisect

• Result: few general methods


Newton in Higher Dimensions

• Start with
want
f ( x, y )  0
want
g ( x, y )  0

• Write as vector-valued function


 f ( x, y ) 
f (x n )   

 g ( x , y ) 
Newton in Higher Dimensions

• Expand in terms of Taylor series


want
f (x n  δ)  f (x n )  f (x n ) δ  ...  0

• f’ is a Jacobian
f (x n )  J  
f
x
f
y

Newton in Higher Dimensions

• Solve for 
δ   J 1 (x n ) f (x n )

• Requires matrix inversion (we’ll see this later)


• Often fragile, must be careful
– Keep track of whether error decreases
– If not, try a smaller step in direction 

You might also like