1

MECE 390-A1
Numerical Methods of Mechanical Engineering

Chapter 1. Errors & Taylor series expansion

Lecture 2. 2018/09/06
Fall 2018
Department of Mechanical Engineering, University of Alberta

MEC E 390 -A1

 2

Objective

1. Decimal system and Binary system: Conversion

2. IEEE Floating Point representation: Precision & conversion
3. Storage limitations & sources of round-off errors
4. Taylor series expansion: truncation error & order of errors
5. Effect of step size

MEC E 390 -A1

Round-off errors 3

To understand Round-off errors, we need to firs look how computers store numbers

 Floating point numbers

Computers use only a fixed number of digits to represent numbers: Finite precision
While we use a Decimal System of digits (base 10): 0,1,2,3…9, computers use a Binary System
(base 2):0,1 (bit)

 Bits are grouped in bytes: 1byte = 8bits

Example: binary to decimal

MEC E 390 -A1

Binary system 4

Example: Conversion decimal to binary: 156

Method

Principle

MEC E 390 -A1

Binary system cont… 5

Example: Conversion decimal to binary: 0.8125

Example: Conversion decimal to binary: 100.25

Frist write 100.25 as 100+0.25  100
 0.25

MEC E 390 -A1

Binary system cont… 6

Integers
All Integers can be exactly represented by using binary system. However, only up to a certain limit.
 Limit for n bits is 2𝑛𝑛 − 1.  4-bit integers

Example: find maximum integer for 5 bits

25 − 1 = 32 − 1 = 31

Real numbers (Floating points)

The standard for the representation of these numbers is the IEEE Floating Point Standard.

MEC E 390 -A1

IEEE Floating representation 7

 The IEEE Floating Point representation

Single Precision system: 32 bits

Sign (S): + or - (1bit)
1
Exponent (exp): 10𝑛𝑛 (8bits)
8
Mantissa (fraction): 23 (bits)
Double Precision system: 1+11+52 = 64 bits
1 Extended Precision system: 1+15+64 = 80 bits

 Ranges of numbers that can be represented

If we pass the limit for decimal precision

Single Precision
𝑥𝑥 ≥ 10308  inf (∞) Overflow
𝑥𝑥 ≤ 10−308  0 Underflow
MEC E 390 -A1
IEEE Floating representation cont… 8

 The IEEE Floating Point conversion

Sign: positive 0, negative 1
Exponent (power): Read non zero binary number
Then apply -15
1001= 𝟏𝟏 × 20 + 𝟎𝟎 × 21 + 𝟎𝟎 × 22 +𝟏𝟏 × 23 = 9-15 = -6
10101= 𝟏𝟏 × 20 + 𝟎𝟎 × 21 + 𝟏𝟏 × 22 +𝟎𝟎 × 23 +𝟏𝟏 × 24
= 21-15 = 6
Fraction: write as 1.*****, drop zeros
101000000 = 1.101
1001000100 = 1.10010001
Mantissa

Example: covert 100.25 to IEEE Floating Points

+6
100.2510 = 1100100.012 (𝑠𝑠𝑠𝑠𝑠𝑠, 𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑝 4 𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒) = 26 × 1.100100012 = +221−15 × 1.100100012
−0.0253901 = 0.0000011012 = 2−6 × 1.100100012 = +221−15 × 1.100100012
-6
MEC E 390 -A1
Storage limitations 9

 Errors due to real number storage

Floating point storage has a number of limitations:
1. Limited range of numbers that can be stored-dynamic range.
2. Only a finite number of real numbers can be stored within a given range (e.g. between -1 ~ 1)
3. The intervals between numbers, ∆𝑥𝑥, increases as the numbers grow in magnitude.

The net effect of this is that all computers involving floating point numbers are subjected to
computer-induced errors. Collectively, these errors fall under the umbrella of round-off error.

10𝑛𝑛 ∆𝑥𝑥 10−𝑛𝑛 10−𝑛𝑛 10𝑛𝑛

infiinty zero infiinty

over flow Dynamic range Under flow Dynamic range over flow

MEC E 390 -A1

Source of Round-off errors 10

Example: 51/17=3 (exact)

4/3=1.3333333…(rounded off to: 8 digits for single precision, 16 digits for double precision)
computer prefers 2 -1=1 than 1000000002 – 1000000001=1 (large storage required)
The loss of significant digits during the subtraction of nearly equal numbers is perhaps the greatest
source of round-off error.

 Machine/computer precision 𝜺𝜺𝒎𝒎 : Minimum relative step that can be represented in Finite
Precision Arithmetic.

Example: if 𝜹𝜹 < 𝜺𝜺𝒎𝒎𝟏𝟏 then 1.0 + 𝜹𝜹 =1.0, : if 𝜹𝜹 < 𝜺𝜺𝒎𝒎𝟎𝟎.𝟏𝟏 then 1.0 + 𝜹𝜹 =1.1
Value of 𝜺𝜺𝒎𝒎 : double precision ~ 10−16
single precision ~ 10−7

MEC E 390 -A1

Tayler series 11

Many numerical procedure are approximations that are derived from Tayler series. We often can
use Tayler series to estimate the expected error.
 Tayler series expansion
ℎ2 ℎ3 𝑑𝑑𝑑𝑑
𝑓𝑓 𝑥𝑥0 + ℎ = 𝑓𝑓 𝑥𝑥0 + 𝑓𝑓′ 𝑥𝑥0 ℎ + 𝑓𝑓𝑓(𝑥𝑥𝑜𝑜 ) + 𝑓𝑓𝑓(𝑥𝑥𝑜𝑜 ) + ⋯ ,where 𝑓𝑓 𝑥𝑥0 = 𝑑𝑑𝑑𝑑�𝑥𝑥 and ℎ = 𝑥𝑥 − 𝑥𝑥𝑜𝑜 = ∆𝑥𝑥
2! 3! 𝑜𝑜

If we truncate this series after the n derivative term, we can show the source of the error
e.g. n=1: 𝑓𝑓 𝑥𝑥0 + ℎ = 𝑓𝑓 𝑥𝑥0 + 𝑓𝑓′ 𝑥𝑥0 ℎ + truncation error (~𝑂𝑂(ℎ2 ))

Truncation error: n=1

Truncation error: n=2

MEC E 390 -A1

Truncation error 12

If we take only up to the first order approximation as:

Truncation error

′ 𝑥𝑥0
ℎ2 ′
ℎ3
𝑓𝑓 𝑥𝑥0 + ℎ = 𝑓𝑓 𝑥𝑥0 + 𝑓𝑓 ℎ + 𝑓𝑓 𝑥𝑥𝑜𝑜 + 𝑓𝑓𝑓(𝑥𝑥𝑜𝑜 ) + ⋯
3! 3!
Dominant error (~∝ 𝑂𝑂(ℎ2 ))
 The largest contributing term in the error is the second
derivatives term.
The corresponding error is proportional to ℎ2 .
Error = 𝑂𝑂(ℎ2 ),
For, n order approximation, the dominant error is : Error = 𝑂𝑂(ℎ𝑛𝑛 )
This mean that reducing can dramatically increase the accuracy of Taylor’s formula.
For example, for the first order approximation, when h is halved the error should go down by
factor 4.
∆𝑥𝑥 ∆𝑥𝑥 1 1
i.e. ℎ = ∆𝑥𝑥  ; thus 𝜀𝜀  𝜀𝜀( )2 = 𝜀𝜀 ,
2 2 2 4

MEC E 390 -A1

Example of Taylor series 13

Example 1. Approximate the following function at 𝑥𝑥 = 0 using Taylor series

𝑓𝑓 𝑥𝑥 = 𝑥𝑥 3 + 2𝑥𝑥 2 + 3𝑥𝑥 + 2
Taylor expansion at 𝑥𝑥 = 0 is given by
𝑥𝑥 2 𝑥𝑥 3
𝑓𝑓(𝑥𝑥) = 𝑓𝑓 0 + 𝑓𝑓𝑓 0 𝑥𝑥 + 𝑓𝑓𝑓𝑓 0 + 𝑓𝑓𝑓𝑓 0 +⋯
2! 3!
0 approximation 𝑓𝑓0 𝑥𝑥 =𝑓𝑓 0 + ⋯ = 2
1st approximation: 𝑓𝑓1 𝑥𝑥 = 𝑓𝑓 0 + 𝑓𝑓𝑓 0 𝑥𝑥 = 2 + 𝑥𝑥(3𝑥𝑥 2 + 3𝑥𝑥 + 3)𝑥𝑥=0 + ⋯ = 3𝑥𝑥 + 2
𝑥𝑥 2 𝑥𝑥 2
2nd approximation: 𝑓𝑓2 𝑥𝑥 = 𝑓𝑓 0 + 𝑓𝑓𝑓(0)𝑥𝑥 + 𝑓𝑓𝑓(0) 2! + ⋯ = 2 + 𝑥𝑥(3𝑥𝑥 2 + 3𝑥𝑥 + 3)𝑥𝑥=0 + (6𝑥𝑥 + 4)𝑥𝑥=0
2
= 2𝑥𝑥 2 + 3𝑥𝑥 + 2
𝑥𝑥 2 𝑥𝑥 3
3rd approximation: 𝑓𝑓 𝑥𝑥 = 𝑓𝑓 0 + 𝑓𝑓𝑓(0)𝑥𝑥 + 𝑓𝑓𝑓(0) 2! + 𝑓𝑓𝑓(0) 3! +⋯
𝑥𝑥 2 𝑥𝑥 3
=2+ 𝑥𝑥(3𝑥𝑥 2
+ 3𝑥𝑥 + 3)𝑥𝑥=0 + (6𝑥𝑥 + 4)𝑥𝑥=0 + (6𝑥𝑥 + 4)𝑥𝑥=0 = 𝑥𝑥 3 + 2𝑥𝑥 2 + 3𝑥𝑥 + 2
2 6
Note: 3rd approximation yields an exact function.
Discussion: fifth order polynomial function, fifth order approximation

MEC E 390 -A1

Example of Taylor series 14

MEC E 390 -A1

Example of Taylor series 15

Example 2. Approximate the following function using Taylor series

1
𝑓𝑓 𝑥𝑥 =
1 − 𝑥𝑥
The 1st, 2nd and 3rd order approximations about 𝑥𝑥 = 𝑥𝑥0 are given by, respectively:
1 𝑥𝑥−𝑥𝑥
1st approximation: 𝑓𝑓1 𝑥𝑥 = + (1−𝑥𝑥 𝑜𝑜)2 + 𝑂𝑂((∆𝑥𝑥)2 ) (error is 2nd order)
1−𝑥𝑥𝑜𝑜 𝑜𝑜
1 𝑥𝑥−𝑥𝑥𝑜𝑜 (𝑥𝑥−𝑥𝑥𝑜𝑜 )2
2nd approximation: 𝑓𝑓2 𝑥𝑥 = + + (1−𝑥𝑥 )3 + 𝑂𝑂((∆𝑥𝑥)3 ) (error is 3rd order)
1−𝑥𝑥𝑜𝑜 (1−𝑥𝑥𝑜𝑜 )2 𝑜𝑜
1 𝑥𝑥−𝑥𝑥 (𝑥𝑥−𝑥𝑥 )2 (𝑥𝑥−𝑥𝑥 )3
3rd approximation: 𝑓𝑓3 𝑥𝑥 = + (1−𝑥𝑥 𝑜𝑜)2 + (1−𝑥𝑥𝑜𝑜)3 + (1−𝑥𝑥𝑜𝑜)4 + 𝑂𝑂((∆𝑥𝑥)4 ) (error is 4th order)
1−𝑥𝑥𝑜𝑜 𝑜𝑜 𝑜𝑜 𝑜𝑜

Figure: Tayler series

approximations and exact
function (left). Ture
(absolute) errors (right)

MEC E 390 -A1

Effects of step size 16

There are, in general two ways of reducing truncation error

1. Choose a smaller step size
2. Include more terms in series expansion

However, in either case, we are increasing the number of operations that must be performed by
the computer and this has the effect of increasing round-off error. Because the total error is the
sum of the round-off and truncation errors, there is trade-off that generally favors step sizes that
are small but not too small.

 When step size decreases

Round-off error: We will require a larger number of numerical computations. The effects of round-
off will accumulate possibly causing a larger global error.
Truncation error will decrease because the effect of eliminating extra terms in an approximation is
less important

MEC E 390 -A1

 Effects of step size cont… 17

Total error
Step size Round-off Error Truncation Error
Error

hoptimal Truncation error

Decrease Increase Decrease
Increase Decrease Increase

Round-off error
Sept size, h

In general, we do no have the patience to find the exact value of hoptimal. Use instead your
intuitive (Remember, we are smarter than machine! )

MEC E 390 -A1

Error measurement 18

Numerical error is a signed quantity defined by the amount that needs to be added to the
numerical or approximate value to be equal to the exact or true value.
Exact Value = Numerical Value + Error
True Value = Approximate Value + Error

1. Ture error absolute error

 𝑬𝑬𝒕𝒕 = Ture Value – Approximated Value
- The Absolute Error tells us how big the error is, but not how important it is.
- The Absolute Error is a signed quantity
- The order of the two terms in the error calculation is important!

2. Relative error
Ture Value – Approximated Value
 𝜺𝜺𝒕𝒕 = and used as:
Ture Value
Percent Relative Error: 𝜺𝜺𝒕𝒕 % = 100𝜺𝜺𝑻𝑻

MEC E 390 -A1

Error measurement cont… 19

Relative error is preferred for setting a convergence or stopping criterion, because it give the
relative importance of the error.
 In engineering problems often 0.1% or 0.01% accuracy is acceptable, but in numerical
calculations it is often easy and advisable to require higher accuracy of the results.

3. Practical difficulties
 When true value tends to zero (True value  0), Relative Error tends to infinity (𝜺𝜺𝒕𝒕  ∞ ).
Solution: when calculating the Relative Error, add a very small value to the denominator, to
Ture Value – Approximated Value 𝜹𝜹~10−6 (single precision)
prevent division by zero. 𝜺𝜺𝒕𝒕 =
Ture Value+𝜹𝜹 𝜹𝜹~10−12 (double precision)

MEC E 390 -A1