INTRODUCTION TO

NUMERICAL ANALYSIS
Two Methods in Solving a Problem

 Direct or Analytic Method

- uses basic algebra and calculus
- results are mathematical functions that
is used to evaluate specific problems
 Indirect or Numerical Method

- solution is always numerical and only

an approximation
The Need for Numerical
Computation
Exact vs. Approximate Solutions
 x y 2
Linear system 
2 x  y  4
4

2
Definite Integral 3 x dx
0

Roots of p pz   c0  c1 z  c2 z  c3 z  c5 z
2 3 4
The Need for Numerical
Computation
Problems that can be solved in principle by
a finite sequence of elementary operations:
(1) Solve a system of n linear equations in n
unknowns.
(2) Minimize a linear function of n variables
subject to m linear constraints.
(3) Find the shortest tour between n cities.
The Need for Numerical
Computation

(4) Find an eigenvalue of an n x n matrix.

(5) Minimize a function of several variables.

(6) Evaluate an integral.

(7) Solve an ordinary differential equation

(8) Solve partial differential equation

The Need for Numerical
Computation
The day has long since passed when an
advance in the physical sciences could be
achieved, or a significant engineering product
developed, without numerical mathematics.

“Numerical analysis is about

constructing and analyzing quantitative
methods for the computation of solutions to
mathematical problems.”
What is Numerical Analysis?

concerns with the development and evaluation

of systematic methods and techniques known
as algorithms for approximating solutions to
problems where an exact solution is difficult
to obtain or is non-existent
What is Numerical Analysis?
The discipline of computing approximate
solutions of mathematical problems with main
goal of developing efficient algorithms for
computing precise numerical values of
mathematical quantities.

With objects of interest in scientific

applications known as Scientific Computing which
seek not only the qualitative properties but also the
quantitative numerical data of the problem.
Sources of error in applied numerical
computation

(i) Inexactness of the mathematical model

for the underlying physical phenomenon.
(ii) Errors in measurements of parameters
entering the model.
(iii) Round-off errors in computer arithmetic.
(iv) Approximations used to solve the full
mathematical system.
History of Numerical Analysis

 least-squares data fitting(1795)

 systems of linear equations (1809)
 Numerical quadrature (1814)
 Fast fourier transform (1805)
rediscovered by Cooley and Tukey
in 1965
Carl Friedrich Gauss
History of Numerical Analysis
Built on a strong foundation: the
mathematical subject of approximation theory.

Hermite, Chebyshev, Legendre, Lagrange,

Charles Pafnuty Lvovich Adrien-Marie Joseph
(1822-1901) (1821-1894) (1752-1833) (1736-1813)
 Numerical Methods

A Babylonian tablet
(approx 1000 – 2000 BC)
which is an approximation to

2
(accurate to 6 decimal places)
Examples
1. The Martini glass filled to a depth of 6cm
216
contains about of wine. How deep do
3
you need to fill the red wine glass below in
order to ensure that it contain the same
amount of wine?
Examples
2. Finding solution to the general form of the
Navier-Stokes equations for the conservation
Dv
of momentum which is    P  f
Dt

3. Google uses numerical methods; they are

prepared to sacrifice some accuracy in order
to get maximum speed…
Representation of Numbers
 Representation of Integers
A nonnegative integer N will be represented
in the binary system as

N  an an 1    a1a0 2
n 1
 an 2  an 1 2
n
     a1 2  a0 2
1 0

where the coefficients ak are either 0 or 1.

 Representation of Numbers
 Representation of Fractions
If x is a positive real number, then its integral
part xI is the largest integer less than or equal to x,
while xF  x  xl

is its fractional part. Equivalently,

where each bk is either zero or one.

 Representation of Numbers

Each decimal number x can be represented in its

binary form.

n 1 1 2
x  an 2  an 1 2
n
     a1 2  a0 2  b1 2  b2 2    
1 0
 Representation of Numbers

Base 10
1 2
257.76  2  10  5  10  7  10  7 10  6  10
2 1 0

Base 2
 (1 23  0  2 2  1 21  1 20 ) 
(1011.0011) 2   1 2 3 4 

  ( 0  2  0  2  1  2  1  2 ) 10
 11.1875
Numerical Error
Defined as the difference between the
true value in a calculation and the approximate
value found using a numerical method

True Error = True Value – Approximate Value

 Ways of Measuring Error
- Absolute Error
- Relative Error
Some definitions
 If p* is an approximation to p, the absolute error is
lp-p*l and the relative error is
, provided that

 The number p* is said to approximate p to t

significant digits if t is the largest nonnegative
integer for which
Why measure errors?

1) To determine the accuracy of numerical

results.

2) To develop stopping criteria for iterative

algorithms.
Types of Errors
 Truncation error: occurs when an exact formula
is replaced by another to make it easier (or
possible) to solve numerically.
 Round-off error: occurs because computers
cannot represent all real numbers exactly, as
computer numbers have a limited number of digits.
 Human error: is the error in data resulting from
arithmetic or computer error.
Representation of Numbers in Digital
Computers
For a word of length n, a number can be stored
in several different fashions:

 (a) Fixed-point representation where the position of the

decimal (binary)point is fixed.

 (b) Floating-point representation where the position of

the decimal (binary) is not fixed at the outset; rather its
position with respect to the first digit is indicated for
each number separately.
Floating Point Format for Binary
Numbers

y    m  2e
  sign of number 0 for  ve, 1 for - ve
m  mantissa 12  m  102 

1 is not stored as it is always given to be 1.

e  integer exponent
 A normalized floating-point number is a number
of the form
±.d1 . . . dt × βe
where
β = integer = fixed base
e = integer = exponent;
t = number of digits
1 ≤ d1 ≤ β − 1; 0 ≤ di ≤ β − 1 for 2 ≤ i ≤ t.
d1 d2 dt
The fractional part .d1 ...d t    2  ...  t
 

t is called the mantissa of the floating-point number.

Rounding Error in Machine
Number
1. chopping, or rounding toward zero, is the simplest
strategy, in which the base expansion of
a number is truncated after the first p digits

2. rounding to nearest, or rounding to even sets to be

the machine number that is closest to x in absolute
value; if two numbers satisfy this property, then it
is set to be the choice whose last digit is even.
Some definitions
A numerical method is said to be unstable if
the roundoff errors introduced at one stage of the
computation propagate with increasing magnitude
in later stages.

A mathematical problem is said to be well-

conditioned if small changes in the data of a
problem result in small changes in the solution.
Loss of Significance
 Subtraction of numbers that are nearly equal
can create unwanted inaccuracies

x  2  0.001
Example
y  2.003  0.001

Let z  x  y
z 0.001  0.001
 = 0.6667 = 66.67%
z | 2  2.003 |
 Horner’s method

Pn x   an x  an 1 x
n n 1
 ...  a1 x  a0
1

Use nested multiplication to evaluate P(c)


Pn x   an x n 3
 an 1 x n4
  
 ...  a3 x  a2 x  a1 x  a0
P  x   3x  4 x  2 x  2 x  1
5 4 3

# of addition: 4
# of multiplication: 13
P  x   3 x 4  4 x 3  2 x 2  2  x  1

 
P  x   3 x 3  4 x 2  2 x  x  2 x  1

P x  3 x 2
 
 4x  2 x x  2 x  1

P x  3 x  4  x  2  x  x  2  x  1
Nested Multiplication
# of addition: 4
# of multiplication: 5
P  x   3x5  4 x 4  2 x3  2 x  1 P x  3 x  4  x  2  x  x  2  x  1

Evaluate P(0.12) using 3-digit rounding off and 3-

digit truncation.

P  x   3x5  4 x 4  2 x3  2 x  1

P  0.12   1.23578921  true value 

P  0.12   1.24 3 significant digits 
P  x   3x5  4 x 4  2 x3  2 x  1 P x  3 x  4  x  2  x  x  2  x  1
Evaluate P(0.12) using 3-digit rounding off.

P  x   3x5  4 x 4  2 x3  2 x  1
Every operation you performed, you round-off to 3 significant digits

P  0.12   3  0.12   4  0.12   2  0.12   2  0.12   1

5 4 3

 0.0000750 0.000832 0.00346 0.240 1.00

 0.00422 0.240 1.00
 0.234 1.00
 1.23

 0.12   0.00173  0.12   0.000208

4
 0.12   0.0144
2

 0.12    0.0144  0.12   0.00173  0.12   0.000208 0.12   0.0000250

3 5
Evaluate P(0.12) using 3-digit rounding off.

P x  3 x  4  x  2  x  x  2  x  1
Every operation you performed, you round-off to 3 significant digits

P  0.12   3 0.12  4 0.12  2 0.12 0.12  2 0.12  1

 3.64 0.12  2 0.12  0.12  2 0.12  1
  0.437  2  0.12  0.12  2  0.12  1
  2.44  0.12  0.12  2  0.12  1
  0.293 0.12  2  0.12  1
  0.0352  2  0.12  1
 1.96  0.12  1
 0.235  1
 1.24
P  x   3x5  4 x 4  2 x3  2 x  1 P x  3 x  4  x  2  x  x  2  x  1
Evaluate P(0.12) using 3-digit truncation.

P  x   3x5  4 x 4  2 x3  2 x  1
Every operation you performed, you truncate to 3 significant digits

P  0.12   3  0.12   4  0.12   2  0.12   2  0.12   1

5 4 3

 0.0000741 0.000824 0.00344 0.240 1.00

 0.000749 0.00344 0.240 1.00
 0.00418 0.240 1.00
 0.235 1.00
 1.23
 0.12   0.00172  0.12   0.000206
4
 0.12   0.0144
2

 0.12    0.0144  0.12   0.00172  0.12   0.000206 0.12   0.0000247

3 5
Evaluate P(0.12) using 3-digit rounding off.

P x  3 x  4  x  2  x  x  2  x  1
Every operation you performed, you round-off to 3 significant digits

P  0.12   3 0.12  4 0.12  2 0.12 0.12  2 0.12  1

 3.64 0.12  2 0.12  0.12  2 0.12  1
  0.436  2  0.12  0.12  2  0.12  1
  2.43 0.12  0.12  2  0.12  1
  0.291 0.12  2  0.12  1
  0.0349  2  0.12  1
 1.96  0.12  1
 0.235  1
 1.23