Numerical Analysis Ch V: Interpolation and Curve-Fitting

CH V

INTERPOLATION AND CURVE-FITTING

5.1 Introduction

This chapter is concerned with finding

intermediate values and fitting
mathematical functions to a given
discrete data. Such data may come from
measurements made during an
experiment or from numerical results
obtained from an analysis of a problem.

Figure 5.1: Interpolation

If the function passes exactly through
every point of the given values, the
process is called interpolation (figure 5.1).
If, on other hand, the functions which are a
“good fit” to the points but don’t necessary
pass through them, the process is called
curve-fitting, which tries to average out
some of the errors (figure 5.2).

Figure 5.2: Curve-fitting

5.2 Linear Interpolation

This is the simplest kind of interpolations. Suppose we are given a table of values:

x f (x)
----------------------------------
x0 f (x0)
x1 f (x1)
… ……
… ……
xn f (xn)
Numerical Analysis Ch V: Interpolation and Curve-Fitting

If we wish to find the value of f ( x̄ ) where x̄ is somewhere between x3 and x4. Using
linear interpolation, we draw a line between the two tabulated points f(x 3) and f(x4) as
shown in figure (5.3). We are approximating the curve in this region by a straight line.
Using the similar triangles, we form the proportion:

BC DE AC
= BC = DE
AC AE AE

x̄ − x 3
x4 − x3 [ 4
= f ( x )−f ( x 3 )]

x̄ − x 3
P ( x̄ ) = f ( x )int =f ( x3 ) + [ f ( x 4 ) − f ( x3 ) ]
x 4 − x3

In general, to find the value of f(x) for some x located between x i and xi+1 in the table
of given values, then the interpolated value p(x), which is only an approximation for
f(x), is given by:

x − xi
P ( x ) = f ( xi ) + [ f ( x i+1 ) − f ( x i ) ]
x i+1 − x i
Note that if x = xi then p(xi) = f(xi) and if x = xi+1 then p(xi+1) = f(xi+1)

Example 5.1

Given the value of sin x from a book of tables as following:

Sin 00 = 0.0
Sin 100 = 0.17365

Find the sin of 6.50 using linear interpolation.

Solution

6.5 − 0
P (6.5) = f (0 ) + [ f (10 ) − f (0 ) ]
10 −0

6.5
P (6.5) = 0.0+ [ 0.17365 − 0.0 ]
10
= 0.11287
Numerical Analysis Ch V: Interpolation and Curve-Fitting

This has a substantial error and if we want the best possible accuracy from such an
interpolation, we should pick two points xi and xi+1 which are close to the needed value
of x̄
For example, if we pick the two values at 60 and 70, we get:

Sin 60 = 0.10453
Sin 70 = 0.12187

6.5 − 6
P (6.5) = f (6 ) + [ f (7 ) − f (6 ) ]
7−6

0.5
P (6.5) = 0.10453 + [ 0.12187 − 0.10453 ]
1
= 0.1132
This is more close to the correct answer actually.

Let us return now to the original interpolation formula

x − xi
P ( x ) = f ( xi ) + [ f ( x i+1 ) − f ( x i ) ]
x i+1 − x i

and rearrange the right hand side so that the formula becomes:

P( x )=
{ x i +1−x i } {
f ( x i +1 )−f ( x i )
x + f ( x i )−
f ( x i+1 )−f ( x i )
x i +1−x i
xi
}
Once we chose the two tabular points x i and xi+1 , the two quantities in brackets are
constants, which we could call a1 and a0. In this case we can express P(x) in the form
of a first degree polynomial as:

P(x) = a1 x + a0
Numerical Analysis Ch V: Interpolation and Curve-Fitting

5.3 Lagrange Interpolating Polynomial

In previous section we discussed linear interpolation as being the process where we

approximate a small portion of a curve with a straight line which passes through two
known points of the curve

As a better approximation, we might choose to approximate the function f(x) with

some higher order curve. This could be done with the help of Lagrange Interpolating
Polynomial.

In this polynomial, for any set of n+1 data points ( x i , yi ) for i = 0,1,2,3,….,n the
interpolating polynomial is given by:

Qn (x) = L0 (x) y0 + L1 (x) y1 + L2 (x) y2 + ……….. + Ln (x) yn

Where:

Li (x), i = 0,1,2,……n are “ Lagrangian” polynomials of degree n defined as:

( x-x 0 ) ( x-x1 ) .. . ..( x-x i-1 )( x-x i+1 ) . .. .. .( x-x n-1 ) ( x-x n )
Li ( x )=
( xi -x 0 ) ( xi -x 1 ) .. . ..( x i -x i-1 )( x i -x i+1 ) . .. .. .( x i -x n-1 ) ( x i -x n )

Example 5.2

Use Lagrangian polynomials to derive a polynomial passing through the following

points:

x0 = 1 y0 = 1
x1 = 3 y1 = 5
x2 = 6 y2 = 10

and estimate the value of y when x = 4.5

Solution:

There are 3 data points, hence there will be a second order interpolating polynomial
given by:

Q2 (x) = L0 (x) y0 + L1 (x) y1 + L2 (x) y2

Where:
Numerical Analysis Ch V: Interpolation and Curve-Fitting

( x-x 1 ) ( x-x 2 )
L0 ( x )=
( x 0 -x 1 ) ( x 0 -x 2 )

( x-3) ( x-6 ) 1
¿ = ( x 2 − 9 x + 18 )
( 1-3 ) ( 1-6 ) 10

( x-x 0 ) ( x-x 2 )
L1 ( x )=
( x 1 -x 0 ) ( x 1 -x 2 )

( x-1) ( x-6 ) 1
¿ =− ( x2 − 7 x + 6 )
( 3-1 ) ( 3-6 ) 6

( x-x 0 ) ( x-x 1 )
L2 ( x )=
( x 2 -x 0 ) ( x 2 -x 1 )

( x-1) ( x-3 ) 1
¿ = ( x2 − 4 x + 3 )
( 6-1 ) ( 6-3 ) 15

Then
1 1 1
Q2 ( x ) = [ ( x 2 − 9 x +18 )] × 1 + [− ( x 2 − 7x +6 ) × 5] + [ ( x2 − 4 x +3 )] × 10
10 6 15

1
¿− ( x 2 − 34 x + 18 )
15
As a check, the three values of x should be substituted into the polynomial to give:

Q2 (1) = 1 , Q2 (3) = 5 and Q2 (6) = 10

Then Q2 (4.5) = 7.65

Disadvantages of Lagrange Interpolating Polynomial:

1. Lagrange approach has a high number of arithmetic operations that must be

carried out to compute an interpolation.

2. If new data points are added to a set of points that has already been operated on,
the whole process must be started again from the beginning.
Numerical Analysis Ch V: Interpolation and Curve-Fitting

5.4 Difference methods

Newton Forward-Difference Formula

As alternative approach to find the interpolating polynomials Q n(x) that pass exactly
through n+1 data points given as ( xi, yi ) i = 0,1,2,…..,n is based on polynomial
representation given by:

Qn (x) = C0 + C1 (x-x0) + C2 (x-x0) (x-x1) +……. + Cn (x-x0) (x-x1) ….. (x –xn-1)

Where C0 , C1, C2 , ……… Cn can be determined for equally spaced values of x as:

C0 = y 0
Δy Δ y0 = y1 − y0
C1 = 0
h Δ 2 y 0 = Δ ( Δ y 0 ) = Δy 1 − Δy 0
Δ2 y 0 and
C2 = 2
2h Where h = x i+1 − x i

In general
j
Δ y0
C j= j
j! h
j j−1 j−1
Where: Δ yi = Δ y i+1 − Δ yi which can be taken from the following table.

x y ∆y ∆2y ∆3y ∆4y ∆5y

--------------------------------------------------------------------------------------------------------
x0 y0
∆y0
x1 y1 ∆2y0
∆y1 ∆3y0
x2 y2 ∆2y1 ∆4y0
∆y2 ∆3y1 ∆5y0
x3 y3 ∆ y2
2
∆ y1
4

∆y3 ∆3y2
x4 y4 ∆2y3
∆y4
x5 y5
Numerical Analysis Ch V: Interpolation and Curve-Fitting

Example 5.3
Given the following data based on the function y = cos x where x is in degrees:

x 20 25 30 35 40

y 0.93969 0.90631 0.86603 0.81915 0.76604

use the forward difference scheme to estimate cos (27).

Solution: ‫ والحسابات‬6‫تاكد من نقل البيانات‬

Arrange the data as a table of forward differences:

x y ∆y ∆2y ∆3y ∆4y

---------------------------------------------------------------------------------------------
20 0.93969
-0.3338
25 0.90631 -0.0069
-0.04028 0.00031
30 0.86603 -0.00651 0.00005
-0.04687 0.00036
35 0.81915 -0.00623
-0.05311
40 0.76604

h = 5, x0 = 20
Δ j y0
C j= j
j! h
C0 = 0.93969
Δ y0
C1=
h = -0.006676
2
Δ y0
C2 = 2
2! h = -0.000138
Δ3 y0
C3 = 3
3! h = 0.0000004
Δ4 y 0
C 4=
4 ! h 4 = 0.000000003

Q4(x) = 0.93696-0.00667 (x-20) – 0.000138 (x-20)(x-25)+0.0000004 (x-20)(x-25)(x-

30)
+0.0000000003 (x-20)(x-25)(x-30)(x-35)
Numerical Analysis Ch V: Interpolation and Curve-Fitting

Then Q4(27) = (Exact solution = )

5.5 Further topics:

 Hermit Interpolation

Problems:
5.- Use Lagrangian polynomials to obtain an interpolating polynomial for the
data
x 0.0 0.1 0.2 0.3
y 0.1 0.1005 0.102 0.1046
and use it to estimate the value of y when x = 0.4 [ Answer y(0.4) = 0.1084]

2- For the following set of data, express the third interpolating polynomial in its
Newton’s forward difference form.

X 0 2 4 6
y 10 7 0 -11

5.6 Real life exercises:

 A metal ball was heated to 90 oC, and then it was dropped into water at 20 oC.
the relation between time and the temperature change of the ball was according
to the following table:

t (min) 0 5 10 15 20 25
T (oC) 90 62.8 45.8 35.6 29.5 25.8

Use Linear, Lagrange and Newton forward difference interpolation

techniques to estimate the ball temperature after 12 minutes.
Numerical Analysis Ch V: Interpolation and Curve-Fitting

Least-Squares Curve-Fitting -5.5

If we are seeking a function to follow closely a large number of data points measured
in an experiment, it is often more practical to seek a function which represents a “best
fit” to the data rather than one which passes through all points exactly.

Given n data points (xi, yi), i=0,1,2,….,n, the required “best fit” polynomial can be
written in the form:
p (x) = a0 + a1 x + a2 x2 + ….. + an xn
At each of the n tabulated points, the resulting p(xi) will be approximately equal to the
tabulated yi with an error which we shall call Ei

We could then write the following n equations, one for each tabulated point:

y0 – p(x0) = E0
y1 – p(x1) = E1
y2 – p(x2) = E2
⋮
yn – p(xn) = En

Or, in general,
yi – p(xi) = Ei

Can we make the sum of the errors equals to zero? We can but it is not desirable as the
data may be fitted to a completely wrong curve.

Can we make the sum of the absolute errors equal to zero? No, since then each of the
Ei would in turn have to be zero.

Instead, we can minimize the sum of the squares of Ei. We used the squares to preserve
the plus sign in order to keep the Ei from canceling each other out.

The sum of the squares of the difference between p(x) and the actual values of y is
given by:
n
S = ∑ [ y i −p ( x i ) ]2
i=1

which can be minimized by taking the partial first derivatives of S with respect to each
of the constants, a0, a1, a2,…… an and putting the result equals to zero, thus

∂S ∂S ∂S ∂S
= = = ..... = =0
∂ a 0 ∂ a1 ∂ a2 ∂ an
Numerical Analysis Ch V: Interpolation and Curve-Fitting

∂S
= 2 ∑ ( y i − a0 - a 1 x i - a 2 x 2 …. . - a n x n ) (−1 ) = 0
∂ a0 i i

∂S
= 2 ∑ ( y i − a0 - a 1 xi - a 2 x 2 …. . - a n x n ) (−xi ) = 0
∂ a1 i i

∂S
= 2 ∑ ( yi − a0 - a 1 xi - a 2 x 2 …. . - a n x n ) (−x 2 ) = 0
∂ a2 i i i

... .
∂S
= 2 ∑ ( y i − a0 - a 1 x i - a 2 x 2 …. . - a n x n ) (−x n ) = 0
∂ an i i

This linear system of equations can be written in matrix form and be solved using any
of the learned techniques. The matrix form is:

[ ] [ ][ ]
nn n nn n n n n n n
n ∑ x i ∑ x 2 . . . . ∑ x n ¿ ∑ x i ∑ x 2∑ x 3 . . . . ∑ x n+ 1 ¿ ∑ x 2 ∑ x 3 ∑ x 4 . . . ∑ x n+ 2 ¿ [ . . . . . . . . . . . . ¿ ] ¿ ¿
i=1 i=1 i i=1 i i=1 i=1 i i=1 i i=1 i i=1 i i=1 i i=1 i i=1 i
¿
Example 5.4
Suppose in an experiment, a heavy object was dropped from a height of 1100 feet. Let
x stands for the time, in seconds, after the object was dropped. If the height was
measured to the nearest 10 feet for the first 4 seconds and was as in the next table:

xi 0 1 2 3 4
yi 1100 1080 1040 96 840
0

Use least squares curve fitting method to fit the measured data to a first, second and
third order polynomial and use these to estimate the distance at 2.5 seconds.
Numerical Analysis Ch V: Interpolation and Curve-Fitting

Solution:
First order polynomial:
The polynomial will be in the form: y = a0 + a1 x

[ ]
n
n∑ xi ¿ ¿ ¿ ¿
i= 1
The set of equations are: ¿
Therefore we need to sum the values of x, y, x2, and xy. This is best done in table
form.

N x Y X2 Xy
1 0 1100 0 0
2 1 1080 1 1080
3 2 1040 4 2080
4 3 960 9 2880
5 4 840 16 3360
∑ 10 5020 30 9400

Using the matrix form and the table to construct the equations:

5a0 + 10a1 = 5020

10a0 + 30a1 = 9400

Solving the two equations for the two unknowns, we get:

a0 = 1132
a1 = -64

Then the first degree polynomial to represent the data is

Y = 1132 – 64 x

At 2.5 seconds, the distance is:

Y = 1132 – 64 (2.5) = 972 feet

Numerical Analysis Ch V: Interpolation and Curve-Fitting

5.6- Curve-Fitting with pre specified function:

Sometimes the given data is known to follow a specific function and the aim is to find values
of the parameters specifying the function. For example suppose that a given set of data is to
follow the function
y=a+b sin ( )
πx
10
Then, for this function to usable, values of a and b have to determined.
Applying the same procedure of least squares curve fitting,

[ ( ( ))] =minimum
2
π xi
S= ∑ n y i− a+b sin
i=¿ ¿ 10

∂S ∂ S
Solving for = =0 one would arrive to the following two equations in the two
∂a ∂b
unknowns, a and b

( )
n n
πx
na+ ∑ sin b=¿ ∑ y i ¿
i=1 10 i=1

( ) ( ) ( )
n n n
πx 2 πx πx
∑ sin 10
a+ ∑ sin
10
b=¿ ∑ y i sin
10
¿
i=1 i =1 i=1

Solving the two equations and obtaining values of a and b, one can use the function
y=a+b sin
πx
10 ( )
to calculate the value of y at any value of x.

In case the data is to be fitted to an exponential function, which represents the behavior of
many real life systems, which has the form of

In dealing with this relation to find values of A and B, the easiest approach is to linearise the
function by taking the natural logarithm of both sides:

Let (lnA = a) and (B=b)

Now the function to be minimized is

Applying least squares fitting gives

where (B = b) and (A = exp(a)).

Numerical Analysis Ch V: Interpolation and Curve-Fitting

5.7- Data fitting using cubic spline:

Suppose that the data is not subject to an error, so that the lease squares approach is not
appropriate to apply, but still we wish to fit the data to a smooth curve. Polynomials are a
common choice because of their ease of evaluation, differentiation, and integration compared
to other functions such as trigonometric and exponential.

When n (order of polynomial) becomes large, in many cases, one may get oscillatory
behavior in the resulting polynomial. This was shown by Runge when he interpolated data
based on a simple function of
1
y=
1+25 x2
on an interval of [–1, 1]. For example, take six equidistantly spaced points in [–1, 1] and find
y at these points as given in Table 1.

.Table 1 Six equidistantly spaced points in [–1, 1]

1
y= x
1+25 x2
0.038461 1.0–
0.1 0.6–
0.5 0.2–
0.5 0.2
0.1 0.6
0.038461 1.0

Now through these six points, one can pass a fifth order polynomial
−11 5 4 −11 3 2 −11 −1
f 5 ( x )=3 . 1378×10 x +1 .2019 x −3 . 3651×10 x −1 .7308 x +1 .0004×10 x+5 . 6731×10 ,
−1≤x≤1

through the six data points. On plotting the fifth order polynomial (figure 5.4) and the
original function, one can see that the two do not match well. One may consider choosing
more points in the interval [–1, 1] to get a better match, but it diverges even more (see Figure
5.5), where 20 equidistant points were chosen in the interval [–1, 1] to draw a 19th order
polynomial. In fact, Runge found that as the order of the polynomial becomes infinite, the
polynomial diverges in the interval of −1<x<−0 .726 and 0 .726< x<1 .

So what is the answer to using information from more data points, but at the same time
keeping the function true to the data behavior? The answer is to use the technique known as
spline interpolation. The most common spline interpolations used are linear, quadratic, and
cubic splines.
Numerical Analysis Ch V: Interpolation and Curve-Fitting

1.2

0.8

0.4
y

0
-1 -0.5 0 0.5 1

-0.4
x
5th Order Polynomial Function 1/(1+25*x^2)

.Figure 5.4 5th order polynomial interpolation with six equidistant points

2.1

8.0

4.0
y

0
1- 5.0- 0 5.0 1

4.0-
x
laimonyloP redrO ht5 )2^x*52+1(/1 noitcnuF

laimonyloP redrO ht91 laimonyloP redrO ht21

.Figure 5.5 Higher order polynomial interpolation is a bad idea

Numerical Analysis Ch V: Interpolation and Curve-Fitting

The condition for a cubic spline is to pass a set of cubics through the given data points.

For the cubic spline to be continues, it requires that both slope and curvature be the same for
the pair of cubics that join at each point.

The cubic for the ith interval, which lies between point (xi , yi) and (xi+1 , yi+1) is in the form:

y = ai(x-xi)3 + bi(x-xi)2 + ci(x-xi) + di

The mathematical formulation continues using the two conditions of equating the 1st and 2nd
derivatives (slope and curvature) of the each adjacent cubics. This ends up with the values of
a, , bi , ci , di are found in terms of the values of the 2nd derivatives as:

Values of the second derivatives are calculated from the following system of equations:

Here there are n-2 equations but n unknowns. 2 additional equations involving S 1 and Sn can
be obtained by specifying the conditions at the end intervals of the whole curve.

Three alternative choices are often used:

1- Natural spline S1 = 0 Sn = 0
2- Parabolic run out spline S1 = S2 Sn = Sn-1
3- Cubic run out spline S1 linearly extrapolated from S2 and S3
Sn linearly extrapolated from Sn-1 and Sn-2

S 2−S1 S 3−S 2 ( h1+ h2 ) S 2−h1 S 3

= ⇒ S1 =
h1 h2 h2

S n−Sn−1 S n−1−Sn−2 ( hn−2 +hn−1 ) S n−1−h n−1 S n−2

= ⇒ S n=
hn−1 h n−2 hn−2

For each end condition, the matrix of coefficients becomes:

Condition 1: S1 = 0 Sn = 0
Numerical Analysis Ch V: Interpolation and Curve-Fitting

[ ]
2 ( h1+ h2 ) h2 ¿ ⋯ ¿
h2 2 ( h2+ h3 ) h3 ¿ ¿ ¿ ¿ ¿ ⋱ ¿ ¿ ¿ ⋯ ¿ h n−2¿ 2 ( h n−2 +h n−1) ¿
h3 2 ( h 3 + h4 ) h4 ¿ ¿

Condition 2: S1 = S2 Sn = Sn-1

[ ]
( 3 h1 +2 h2 ) h2 ¿ ⋯ ¿
h2 2 ( h 2+ h3 ) h3 ¿ ¿ ¿ ¿ ¿ ⋱ ¿ ¿ ¿ ⋯ ¿ hn−2 ¿ ( 2 hn−2 +3 hn−1 ) ¿
h3 2 ( h3 +h 4 ) h4 ¿ ¿

Condition 3: S1 and Sn are linear extrapolations

[
2 2
(h ¿ ¿ 1+2 h2 ) h2−h1
( h1 +h2 ) ¿ ¿ ⋯ ¿ h 2n−2−h2n−1
h2 h2 ¿ ¿ h3 ¿ 2 ( h3 +h 4 ) ¿ h 4 ¿ ¿ ¿ ¿ ⋮ ¿ ¿ ¿ ¿ ⋱ ¿ ¿ ¿ ⋯ ¿ ¿ ( hn−
hn−2
h2 2 ( h2 +h3 ) h3 ¿

In each case the S vector is reduced to n-2 elements and the coefficient matrix becomes
square of size (n-2) (n-2). For each of the cases, the right hand side vector is the same.

Example:
Fit the following set of data using cubic spline curve fitting technique. Use the three end
conditions; natural spline, parabolic runs out spline, and cubic run out spline.

f(x) x
-8 0
-7 1
0 2
19 3
56 4

Note that there are 4 data points meaning there will be 3 cubic polynomials of the form
y = ai(x-xi)3 + bi(x-xi)2 + ci(x-xi) + di

For the 1st polynomial we need to know values of a1 , b1 , c1 , and d1

For the 2nd polynomial we need to know values of a2 , b2 , c2 , and d2
For the 3rd polynomial we need to know values of a3 , b3 , c3 , and d3
The calculation of these coefficients requires the knowledge of the values of the second
derivatives at each data point. This can be achieved by the matrices form represented earlier.

Condition 1:
¿
So that
S1 = 0 S2 = 6.4285 S3 = 10.2857 S4 = 24.4285 S5 = 0
Numerical Analysis Ch V: Interpolation and Curve-Fitting

Condition 2:
¿
So that
S1 = 4.8 S2 = 4.8 S3 = 12.0 S4 = 19.2 S5 = 19.2

Condition 3:
¿
So that
S1 = 0 S2 = 6.0 S3 = 12.0 ` S4 = 18.0 S5 = 24.0

Now calculating the coefficients using the relations

gives;
Condition 1:
cubic No. 1 a1 = 1.0714 b1 = 0 c1 = -0.0714 d1 = -8
Then the interpolating polynomial for the 1 interval would be;
st

y = 1.0714(x-xi)3 - 0.0714(x-xi) - 8

cubic No. 2 a2 = b2 = c2 = d2 =
cubic No. 3 a3 = b3 = c3 = d3 =

Condition 2:
cubic No. 1 a1 = b1 = c1 = d1 =
cubic No. 2 a2 = b2 = c2 = d2 =
cubic No. 3 a3 = b3 = c3 = d3 =

Condition 3:
cubic No. 1 a1 = b1 = c1 = d1 =
cubic No. 2 a2 = b2 = c2 = d2 =
cubic No. 3 a3 = b3 = c3 = d3 =

blanks are left as an exercise.

Closing remark
Real world numerical data is usually difficult to analyze. Any function which would
effectively correlate the data would be difficult to obtain and highly unwieldy. To this end,
the idea of the cubic spline was developed. Using this process, a series of unique cubic
polynomials are fitted between each of the data points, with the stipulation that the curve
obtained be continuous and appear smooth. These cubic splines can then be used to determine
rates of change and cumulative change over an interval.

The splines strength can be demonstrated by the extreme extent, in its ability to correlate data
which doesn’t follow any specific pattern without a single polynomial’s extreme behavior.

Take, for instance, the 100 random data points below:

Numerical Analysis Ch V: Interpolation and Curve-Fitting

Random data points

Clearly there is no relation between these data points. A spline, however, can interpolate all
100 points without the drastic behavior that the necessary 99th degree polynomial would
exhibit.

Cubic curve through the random data

Examples of applying cubic spline

Numerical Analysis Ch V: Interpolation and Curve-Fitting

Exponential test case

Bump Test case