Round-off Error due to Arithmetic Operations

Addition – due to equaling the exponent.

Examples:

Summation:

Chopped:

1
Round-off Error due to Arithmetic Operations

Subtractive Cancellation (subtracting numbers of

almost equal size) – too few significant figures
left.

Examples:

2
Round-off Error due to Arithmetic Operations
Smearing
Occurs when individual terms are larger than summation
itself. Consider the exponential series with x = -10

Consider formulas such as:

2 3 4 5
x x x x
ex = 1 + x + + + + +
2! 3! 4! 5!
With 7-decimal-digit accuracy:
exact answer = 4.54 10-05
computed answer = – 6.26 10-05
(45 terms) wrong sign !
Largest intermediate terms are:
9th = –2,755.732 & 10th = 2,755.732
3
 Approximations and Rounding Errors
• Precautions:
– Sums of large and small numbers: due to equaling the
exponent. They are common in sums of infinite series
where the individual terms are very small when
compared with the accumulated sum. This error can be
reduced by summing first the small terms and using
double precision.
– Cancellation of the subtraction: The subtraction of very
similar numbers.
– Smearing: The individual terms are larger than the total
sum.
– Inner products: They are prone to rounding errors. Thus,
it is convenient to use double precision in this type of
calculations. n
 xi y i = x1y1 + x2y 2 +  + xn yn
i =1
4
 Chapter 4

Truncation Errors and the

Taylor Series

5
 Truncation Error
Error caused by the nature of the numerical technique
employed to approximate the solution.
Example:
Maclaurin series expansion of ex

2 3 4 5
x x x x
ex = 1 + x + + + + +
2! 3! 4! 5!
x2
If we use a truncated version of the series: e = 1 + x +
x
2!
x3 x 4 x5
Then the Truncation Error is: + + +
3! 4! 5!

6
 Taylor Series Expansion

Basic Idea:
Predict the value of a function, ƒ, at a point xi+1
based on the value of the function and all of its
derivatives, ƒ, ƒ', ƒ",… at a neighboring point xi

Given xi, ƒ(xi), ƒ'(xi), ƒ"(xi), ... ƒn+1(xi),

we can predict or approximate ƒ(xi+1)

7
 Taylor Series Expansion
General Form:

h2 h3 hn n
f (x i+1 ) = f (x i ) +hf (x i ) + f (x i ) + f (x i ) + + f (x i ) + R n
2! 3! n!

h = "step size" = xi+1 – xi

Rn = remainder to account for all other terms
h n +1 n +1
= f ( ) with xi    xi+1
(n + 1)!

= O (hn+1) with x not exactly known "on the order of hn+1

"
Note: f(x) must be a function with n+1 continuous derivatives

8
 Taylor Series Expansion

h2 hn n
f (x i+1 ) = f (x i ) + h f (x i ) + f (x i ) + + f (x i ) + O(h n +1)
2! n!

• 0th order T.S. approx. (n = 0): f(xi+1) = f(xi) + O (h1)

• 1st order T.S. approx. (n = 1): f(xi+1) = f(xi) + hf '(xi) + O (h2)

2
h
• 2nd order T.S. approx. (n = 2):f (x i+1 ) = f (x i ) + h f (x i ) + f (x i ) + O (h n +1 )
2!

• nth order T.S. approximation will be exact for an nth order

polynomial

9
 Taylor Series Expansion
f(x )
f(xi ) Zero order
f(xi+1 ) f(xi )

f(xi+1 ) f(xi )+f '(xi )h

f(xi+1 ) f(xi )+f '(xi )h+ )+f "(xi )h2/2!

f(xi+1 )

x
xi xi+1
h
10
 Taylor Series Expansion
General Form:

h2 h3 hn n
f (x i+1 ) = f (x i ) +hf (x i ) + f (x i ) + f (x i ) + + f (x i ) + R n
2! 3! n!

h = "step size" = xi+1 – xi

Rn = remainder to account for all other terms
h n +1 n +1
= f ( ) with xi    xi+1
(n + 1)!

= O (hn+1) with x not exactly known "on the order of hn+1

"
Note: f(x) must be a function with n+1 continuous derivatives

11
 Taylor Series Expansion
Remainder Term: What is ξ ?
If Zero- order approximation: f ( x i +1 ) = f ( xi ) + Ro Ro
f ' ( ) =
h

12
 Taylor Series Example
Use zero-order to fourth-order Taylor series expansions to
approximate the function.
f(x)= -0.1x4 – 0.15x3 – 0.5x2 – 0.25x +1.2
From xi = 0 with h =1. Predict the function’s value at xi+1 =1.

Solution
▪ f(xi)= f(0)= 1.2 , f(xi+1)= f(1) = 0.2 ………exact solution
• Zero- order approx. (n=0) ➔ f(xi+1)=1.2 f ( xi +1 ) = f ( xi )
Et = 0.2 – 1.2 = -1.0
• First- order approx. (n=1) ➔ f(xi+1)= 0.95 f ( x i +1 ) = f ( x i ) + f ' ( x i )h
f’(x)= -0.4x3 – 0.45x2 – x – 0.25, f’(0)= -0.25
f( xi+1)= 1.2- 0.25h = 0.95
Et = 0.2 - 0.95 = -0.75
13
 Taylor Series Example
• Second- order approximation (n=2) ➔ f(xi+1)= 0.45
f ' ' ( x i )h 2
f ( x i +1 ) = f ( x i ) + f ( x i )h +
'

2!
f’’(x) = -1.2 x2 – 0.9x -1 , f’’(0)= -1
f( xi+1)= 1.2 - 0.25h - 0.5 h2 = 0.45
Et = 0.2 – 0.45 = -0.25
• Third-order approximation (n=3) ➔ f(xi+1)= 0.3
f ' ' ( x i ) h 2 f ( 3) ( x i ) h 3
f ( x ) = f ( x i ) + f ( x i )h +
'
+
2! 3!
i +1

f( xi+1)= 1.2 - 0.25h - 0.5 h2 – 0.15h3 = 0.3

Et = 0.2 – 0.3 = -0.1

14
Numerical Differentiation from Taylor Series Expansion
Objective:
Evaluate the derivatives of function, ƒ(xi), without
doing it analytically.
When would we want to do this?
1. function is too complicated to differentiate
analytically:
2 + cos(1 + x ) 0.5x
e
1 + 0.5x
2. function is not defined by an equation,
i.e., given a set of data points (xi, ƒ(xi)), i=1,…,n
i 0 1 2 3 4
xi 1.0 3.0 5.0 7.0 9.0
ƒ(xi) 2.3 4.1 5.5 5.7 5.9
15
Forward Finite Divided Difference

16
Backward Finite Divided Difference

17
Centered Finite Divided Difference

18
 Taylor Series and Truncation errors
Questions:
• Which is a better approximation?
Forward, Centered, or Backward?

• Why?

• When would you use which?

Note:
We also can get higher order forward, centered, and
backward difference derivative approximations
[C&C Chapter 23, tabulated in Figs. 23.1-3]

19
 Example Problem 4.4

Jika kita asumsikan bahwa persamaan f(x) tidak diketahui, sedangkan yang
diketahui adalah tabel berikut:

𝑥1 = -0.5 𝑥2 = -0.25 𝑥3 = 0 𝑥4 = 0.25 𝑥5 = 0.5

𝑓 𝑥1 = 1.2125 𝑓 𝑥1 𝑓 𝑥1 = 1.2 𝑓 𝑥1 𝑓 𝑥1 = 0.925
= 1.2332 = 1.1035

Misal menggunakan metode centered finite divided difference terpendek

−0.925 + 8 × 1.1035 − 8 × 1.2332 + 1.2125

1.1035 − 1.2332 𝑓′ 𝑥 =
𝑓′ 𝑥 = 12 × 0.25
2 × 0.25 =-0.25
=-0.25

20
 Error Propagation
Error Propagation
Errors which appear because we are basing current
calculations on previous calculations which also incurred
some form of error.
Example:

21
 Example Combining Roundoff and Truncation Error

Determine h to minimize the total error of a forward finite-

divided difference approximation for:
f (x i+1 ) − f (x i )
f '(x) 
• Truncation Error: h
f (x i+1 ) − f (x i ) h xi    xi+1
f '(x)  − f "()
h 2
• Round-off Error:
f ( x ) − fˆ ( x )   f ( x ) with  = machine epsilon.

ˆ  (x + h).
(1   ) −  (x i (1  ) h
).
As a result: f ' = i
−  "()
h 2
Roundoff   (x i + h)  +  (x i )   2  (x i )
Error h h
22
Example Combining Roundoff and Truncation Error

 Total error  = truncation error + roundoff error 

h 2 f (x i )
E = | Total Error |  f "() +
2 h

NOTE: Truncation error decreases as h decreases

Round-off error increases as h decreases

23
Example Combining Roundoff and Truncation Error

24
29