How to put a polynomial through points

Ed Bueler

MATH 310 Numerical Analysis

September 2012

Ed Bueler (MATH 310 Numerical Analysis) How to put a polynomial through points September 2012 1 / 29
purpose

These notes are an online replacement for the 14 September class

of Math 310 (Fall 2012), while Bueler is away.

The topics here are also covered in Chapter 8 of the text

(Greenbaum & Chartier). The emphasis here is on how to put a
polynomial through points. When we get to Chapter 8 we will
address the “how good” question.

Ed Bueler (MATH 310 Numerical Analysis) How to put a polynomial through points September 2012 2 / 29
an example of the problem

suppose you have a function y = f (x) which goes through these points:

(−1, 2), (0, 3), (3, 4), (5, 0)

the x-coordinates of these points are not equally-spaced!

◦ in these notes I will never assume the x-coordinates are equally-spaced
let us name these points (xi , yi ), for i = 1, 2, 3, 4
there is a polynomial P(x) of degree 3 which goes through these points
we will build it concretely
we will show later that there is only one such polynomial

Ed Bueler (MATH 310 Numerical Analysis) How to put a polynomial through points September 2012 3 / 29
a picture of the problem

figure below shows the points

as stated, we suppose that they are values of a function f (x)
but we don’t see that function
6

2
y

-1

-2
-2 -1 0 1 2 3 4 5 6
x

Ed Bueler (MATH 310 Numerical Analysis) How to put a polynomial through points September 2012 4 / 29
how to find P(x)

P(x) is the degree 3 polynomial through the 4 points

a standard way to write it is:

P(x) = c0 + c1 x + c2 x 2 + c3 x 3

note: there are 4 unknown coefficients and 4 points

◦ degree n − 1 polynomials have the right length for n points
the facts “P(x) = y ” for the given points gives 4 equations:

c0 + c1 (−1) + c2 (−1)2 + c3 (−1)3 = 2

c0 + c1 (0) + c2 (0)2 + c3 (0)3 = 3
c0 + c1 (3) + c2 (3)2 + c3 (3)3 = 4
c0 + c1 (5) + c2 (5)2 + c3 (5)3 = 0

MAKE SURE that you are clear on how I got these equations, and that
you can do the same thing in an example with different points or different
polynomial degree

Ed Bueler (MATH 310 Numerical Analysis) How to put a polynomial through points September 2012 5 / 29
a linear system

you can solve the equations by hand . . . that would be tedious

we want to automate the process
we have a great matrix-vector tool, namely M ATLAB
I recognize the system has a matrix form “A v = b”:

1 −1 (−1)2 (−1)3
    
c0 2
1 0 2 3  
 0 0 c1
  =  
3
1 3 32 33  c2  4
1 5 52 53 c3 0

(a known square matrix A) × (an unknown vector v) = (a known vector b)

I am not simplifying the numbers in the matrix . . . because:
◦ a machine can do that, and
◦ the pattern in the matrix entries is clear if they are unsimplified
MAKE SURE you can convert from the original “fit a polynomial through
these points” question into the matrix form “A v = b”

Ed Bueler (MATH 310 Numerical Analysis) How to put a polynomial through points September 2012 6 / 29
how to easily find P(x)

M ATLAB is designed to solve linear systems . . . easily!

enter the matrix and the known vector into M ATLAB:
>> A = [1 -1 (-1)^2 (-1)^3; 1 0 0^2 0^3; 1 3 3^2 3^3; 1 5 5^2 5^3]
A =
1 -1 1 -1
1 0 0 0
1 3 9 27
1 5 25 125
>> b = [2; 3; 4; 0]
b =
2
3
4
0

solve the linear system to get v = [c0 c1 c2 c3 ]:

>> v = A \ b
v =
3.000000
0.983333
-0.066667
-0.050000

so the polynomial is P(x) = 3 + 0.983333x − 0.066667x 2 − 0.05x 3

Ed Bueler (MATH 310 Numerical Analysis) How to put a polynomial through points September 2012 7 / 29
notes on matrices and vectors in M ATLAB

you enter matrices like A by rows

◦ spaces separate entries
◦ semicolons separate rows
column vectors like b are just matrices with one column
◦ to quickly enter column vectors use the transpose operation:
>> b = [2 3 4 0]’
b =
2
3
4
0

to solve the system A v = b we “divide by” the matrix: v = A−1 b

. . . but this is left division, so M ATLAB makes it into a single-character
operation, the backslash operation:
>> v = A \ b

the forward slash does not work because of the sizes of the matrix and the vector are not right:
>> v = b / A % NOT CORRECT for our A and b; wrong sizes

Ed Bueler (MATH 310 Numerical Analysis) How to put a polynomial through points September 2012 8 / 29
did we solve the problem?
the polynomial we found had better go through the points:
>> 3.000000 + 0.983333*(-1) - 0.066667*(-1)^2 -0.050000*(-1)^3
ans = 2
>> 3.000000 + 0.983333*(0) - 0.066667*(0)^2 -0.050000*(0)^3
ans = 3
>> 3.000000 + 0.983333*(3) - 0.066667*(3)^2 -0.050000*(3)^3
ans = 4.0000
>> 3.000000 + 0.983333*(5) - 0.066667*(5)^2 -0.050000*(5)^3
ans = -1.0000e-05

a graph is convincing, too:

6

2
y

-1

-2
-2 -1 0 1 2 3 4 5 6
x

Ed Bueler (MATH 310 Numerical Analysis) How to put a polynomial through points September 2012 9 / 29
the general case

suppose we have n points (xi , yi ) with distinct x-coordinates

◦ for example, if n = 4 we have points (x1 , y1 ), (x2 , y2 ), (x3 , y3 ), (x4 , y4 )
then the polynomial has degree one less: the polynomial P(x) which
goes through the n points has degree n − 1
the polynomial has this form:

P(x) = c0 + c1 x + c2 x 2 + · · · + cn−1 x n−1

the equations which determine P(x) say that the polynomial goes
through the points:

P(xi ) = yi for i = 1, 2, . . . , n

written out there are n equations of this form:

c0 + c1 xi + c2 xi2 + · · · + cn−1 xin−1 = yi for i = 1, 2, . . . , n

the n coefficients ci are unknown, while the xi and yi are known

Ed Bueler (MATH 310 Numerical Analysis) How to put a polynomial through points September 2012 10 / 29
the pattern in the matrix, for the general case

as a matrix:
x12 x1n−1
 
1 x1 ...
1 x2 x22 ... x2n−1 
A=
 
.. .. 
 . . 
1 xn xn2 ... xnn−1
and b is a column vector with entries yi : b = [y1 y2 . . . yn ]0
as before, this gives a system of n equations, Av = b
the matrix A is called a Vandermonde matrix, from about 1772

Ed Bueler (MATH 310 Numerical Analysis) How to put a polynomial through points September 2012 11 / 29
Vandermonde matrix, built-in

actually, Vandermonde matrices are already built-in to M ATLAB

for example, the Vandermonde matrix A for our original four points
(−1, 2), (0, 3), (3, 4), (5, 0) is
>> vander([-1 0 3 5])
ans =
-1 1 -1 1
0 0 0 1
27 9 3 1
125 25 5 1

two comments:
◦ oops! the columns are in reversed order, compared to our choice
◦ note that only the x-coordinates are needed to build A, and not the
y -coordinates
we easily fix the column order to agree with our earlier ordering using
“fliplr”, which stands for “flip left-to-right”:
>> A = fliplr(vander([-1 0 3 5]))
A =
1 -1 1 -1
1 0 0 0
1 3 9 27
1 5 25 125

Ed Bueler (MATH 310 Numerical Analysis) How to put a polynomial through points September 2012 12 / 29
Vandermonde matrix method for polynomial interpolation

thus a complete code to solve our 4 point problem earlier is:

A = fliplr(vander([-1 0 3 5]));
b = [2 3 4 0]’;
v = A \ b

after the coefficents v are computed, they form P(x) this way:

P(x) = v(1) + v(2) x + v(3) x 2 + · · · + v(n) x n−1

thus we can plot the 4 points and the polynomial this way:

plot([-1 0 3 5],[2 3 4 0],’o’,’markersize’,12)

x = -2:0.01:6; P = v(1) + v(2)*x + v(3)*x.^2 + v(4)*x.^3;
hold on, plot(x,P,’r’), hold off
xlabel x, ylabel y

this was the “convincing” graph, shown earlier

Ed Bueler (MATH 310 Numerical Analysis) How to put a polynomial through points September 2012 13 / 29
on the cost of solving Vandermonde matrix problems

now, often we want to do a polynomial fit problem like this many times for
different data
so, is it quick? here are some facts to know about solving these systems:
◦ if there are n points then the matrix A has n rows and n columns
◦ internally in M ATLAB, the linear system A v = b is solved by Gaussian
elimination
◦ Gaussian elimination does about 23 n3 arithmetic operations (i.e. additions,
subtractions, multiplications, divisions) to solve such a linear system
so finding the coefficients of the polynomial P(x) through n points takes
about n3 operations
but then you need more operations to evaluate that polynomial, which is
what you usually do with it

Ed Bueler (MATH 310 Numerical Analysis) How to put a polynomial through points September 2012 14 / 29
“new” idea: Newton’s form

before Vandermonde there was already a good, practical idea

◦ an old idea of Newton, perhaps about 1690
the idea is to write the polynomial through the data P(x) not using the
“monomials” 1, x, x 2 , x 3 , . . . , x n−1 ,
. . . but instead to use a form of the polynomial which includes the
x-coordinates of the data points:

P(x) = c0 + c1 (x − x1 ) + c2 (x − x1 )(x − x2 )+
· · · + cn−1 (x − x1 )(x − x2 ) . . . (x − xn−1 )

do you see why this helps?

Ed Bueler (MATH 310 Numerical Analysis) How to put a polynomial through points September 2012 15 / 29
Newton’s form example: 4 points

with the n = 4 points (−1, 2), (0, 3), (3, 4), (5, 0) we can write
P(x) = c0 + c1 (x + 1) + c2 (x + 1)(x) + c3 (x + 1)(x)(x − 3)
this polynomial must go through the four points, so:
c0 =2
c0 + c1 (0 + 1) =3
c0 + c1 (3 + 1) + c2 (3 + 1)(3) =4
c0 + c1 (5 + 1) + c2 (5 + 1)(5) + c3 (5 + 1)(5)(5 − 3) =0
note that lots of terms are zero!
the system of equations has the form
Mw=b
where M is a triangular matrix, b is the same as in the Vandermonde
form, and w has the unknown coefficients:
w = [c0 c1 c2 c3 ]0

Ed Bueler (MATH 310 Numerical Analysis) How to put a polynomial through points September 2012 16 / 29
Newton’s form example, cont.

can you solve this by hand?

yes: find c0 from first equation, then c1 from second equation, etc.
I get c0 = 2, c1 = 1, c2 = −1/6, c3 = −1/20, so

1 1
P(x) = 2 + (x + 1) − (x + 1)(x) − (x + 1)(x)(x − 3)
6 20
MAKE SURE you can do this yourself, on a similar example

Ed Bueler (MATH 310 Numerical Analysis) How to put a polynomial through points September 2012 17 / 29
Newton’s form example, cont.2

so we have a concrete polynomial, but not in standard form:

1 1
P(x) = 2 + (x + 1) − (x + 1)(x) − (x + 1)(x)(x − 3)
6 20
an uninteresting calculation puts it in standard form:

59 1 2 1 3
P(x) = 3 + x− x − x
60 15 20
= 3 + 0.983333x − 0.066667x 2 − 0.05x 3

which is exactly the same polynomial we found earlier

Ed Bueler (MATH 310 Numerical Analysis) How to put a polynomial through points September 2012 18 / 29
Newton’s form for polynomial interpolation: example code

the advantage of the Newton form is that a triangular matrix M is created

◦ which makes it easier to solve the system by hand
◦ only O(n2 ) operations are needed to solve the system
◦ the polynomial comes out in a non-standard form but it is just as easy to
evaluate at a point
for now here is a short code to solve the 4 point problem:
newt4.m
% NEWT4 Compute P(x) using the Newton form, for 4 points.

n = 4; x = [-1 0 3 5]’; y = [2 3 4 0]’; % the points

M = zeros(n,n); % makes M the right size

% form M by columns
M(:,1) = ones(n,1);
for j=2:n
M(j:n,j) = M(j:n,j-1) .* (x(j:n) - x(j-1));
end
b = y;

w = M \ b % w has the coefficients of the polynomial:

% P(x) = w1 + w2 (x-x1) + w3 (x-x1) (x-x2) + w4 (x-x1) (x-x2) (x-x3)

Ed Bueler (MATH 310 Numerical Analysis) How to put a polynomial through points September 2012 19 / 29
Newton’s form shows there is a unique interpolating polynomial

for both Vandermonde and Newton matrix approximations we build an invertible

matrix, so in each case there is exactly one solution
this is easiest to see from the general Newton form matrix:
1
 
 1 (x2 − x1 ) 
 1 (x3 − x1 ) (x3 − x1 )(x3 − x2 ) 
M =
 
. . .


. . . .. 
 . . . . 
1 (xn − x1 ) (xn − x1 )(xn − x2 ) ... (xn − x1 )(xn − x2 ) . . . (xn − xn−1 )

the diagonal entries are all nonzero as long as the x-coordinates are distinct
we can calculate the determinant of this Newton form matrix M
because the matrix is triangular, the determinant is the product of the diagonal:
Y
det M = (xi − xj ) 6= 0
i>j

so the polynomial P(x) always exists and is unique

Ed Bueler (MATH 310 Numerical Analysis) How to put a polynomial through points September 2012 20 / 29
Lagrange’s idea: no systems at all!

another new idea

given the same n = 4 points (−1, 2), (0, 3), (3, 4), (5, 0)
Lagrange and others, by about 1800, knew how to write down four
polynomials, now called the Lagrange polynomials, corresponding to the
x-coordinates x1 , . . . , x4 :
(x − x2 )(x − x3 )(x − x4 ) x(x − 3)(x − 5)
`1 (x) = =
(x1 − x2 )(x1 − x3 )(x1 − x4 ) (−1)(−4)(−6)
(x − x1 )(x − x3 )(x − x4 ) (x + 1)(x − 3)(x − 5)
`2 (x) = =
(x2 − x1 )(x2 − x3 )(x2 − x4 ) (1)(−3)(−5)
(x − x1 )(x − x2 )(x − x4 ) (x + 1)(x)(x − 5)
`3 (x) = =
(x3 − x1 )(x3 − x2 )(x3 − x4 ) (4)(3)(−2)
(x − x1 )(x − x2 )(x − x3 ) (x + 1)(x)(x − 3)
`4 (x) = =
(x4 − x1 )(x4 − x2 )(x4 − x3 ) (6)(5)(2)

the pattern needs attention: I. the numerator and denominator have the
same pattern, but the denominator is a constant with no variable “x”; II.
`i (x) has no “(x − xi )” factor in the numerator, nor “(xi − xi )” factor in the
denominator; III. as long as the xi are distinct, we never divide by zero
Ed Bueler (MATH 310 Numerical Analysis) How to put a polynomial through points September 2012 21 / 29
Lagrange’s idea: polynomials which “hit one point”

why is this helpful?

consider a plot of `1 (x), `2 (x), `3 (x), `4 (x):
3
l1(x)
l2(x)
l3(x)
2 l4(x)

-1

-2

-3
-2 -1 0 1 2 3 4 5 6

a crucial pattern emerges:

the polynomial `i (x) has value 0 at all of the x-values of the
points, except that it is 1 at xi

MAKE SURE make sure you can find the Lagrange polynomials if I give
you the x-values of n points
Ed Bueler (MATH 310 Numerical Analysis) How to put a polynomial through points September 2012 22 / 29
Lagrange’s idea, cont.

the picture on the last page illustrates what is generally true of the
Lagrange polynomials:
(
1, j = i,
`i (xj ) =
0, otherwise.

also, the Lagrange polynomials for the 4 points are each of degree 3
so why does this help find P(x)?
recall that we have values yi which we want the polynomial P(x) to “hit”
that is, we want this to be true for each i:

P(xi ) = yi

thus the answer is:

P(x) = y1 `1 (x) + y2 `2 (x) + y3 `3 (x) + y4 `4 (x)

Ed Bueler (MATH 310 Numerical Analysis) How to put a polynomial through points September 2012 23 / 29
Lagrange’s idea, cont.2

wait, why is this the answer?:

∗
P(x) = y1 `1 (x) + y2 `2 (x) + y3 `3 (x) + y4 `4 (x)

because P(x) is of degree three, as a linear combination of degree 3

polynomials, and
because:
P(x1 ) = y1 `1 (x1 ) + y2 `2 (x1 ) + y3 `3 (x1 ) + y4 `4 (x1 )
= y1 · 1 + y2 · 0 + y3 · 0 + y4 · 0
= y1 ,

and
P(x2 ) = y1 `1 (x2 ) + y2 `2 (x2 ) + y3 `3 (x2 ) + y4 `4 (x2 )
= y1 · 0 + y2 · 1 + y3 · 0 + y4 · 0
= y2 ,

and so on
Ed Bueler (MATH 310 Numerical Analysis) How to put a polynomial through points September 2012 24 / 29
Lagrange’s idea, cont.3

on the last slide we saw that P(xi ) = yi because the polynomials `i (x)
help “pick out” the point xi in the general expression ∗ on the last slide
we can say this more clearly using summation notation:
◦ the polynomial is a sum of the Lagrange polynomials with coefficients yi :
4
X
P(x) = yi `i (x)
i=1

◦ when we plug in one of the x-coordinates of the points, we get only one
“surviving” term in the sum:
4
X X
P(xj ) = yi `i (xj ) = yj · 1 + yi · 0 = yj
i=1 i6=j

Ed Bueler (MATH 310 Numerical Analysis) How to put a polynomial through points September 2012 25 / 29
returning to our 4-point example
for our 4 concrete points (−1, 2), (0, 3), (3, 4), (5, 0), we can
slightly-simplify the Lagrange polynomials we have computed already:
1
`1 (x) = − x(x − 3)(x − 5)
24
1
`2 (x) = + (x + 1)(x − 3)(x − 5)
15
1
`3 (x) = − (x + 1)(x)(x − 5)
24
1
`4 (x) = + (x + 1)(x)(x − 3)
60
so the polynomial which goes through our points is
1 1
P(x) = −(2) x(x − 3)(x − 5) + (3) (x + 1)(x − 3)(x − 5)
24 15
1 1
− (4) (x + 1)(x)(x − 5) + (0) (x + 1)(x)(x − 3)
24 60
a tedious calculation simplifies this to
59 1 2 1 3
x− x −
P(x) = 3 + x ,
60 15 20
which is exactly what we found earlier
Ed Bueler (MATH 310 Numerical Analysis) How to put a polynomial through points September 2012 26 / 29
so, is the Lagrange scheme a good idea?

for n points { (xi , yi ) } we have the following nice formulas which

“completely answer” the polynomial interpolation problem:
Y x − xj
`i (x) =
xi − xj
j6=i

n
X
P(x) = yi `i (x)
i=1
Q P
note “ ” is a symbol for a product, just like “ ” is a symbol for sum
we solve no linear systems and we just write down the answer! yeah!
is this scheme a good idea in practice?
NOT REALLY!

Ed Bueler (MATH 310 Numerical Analysis) How to put a polynomial through points September 2012 27 / 29
so, is the Lagrange scheme a good idea? cont.

we have seen that actually using the formulas to find a familiar form for
P(x) is . . . awkward
the problem with the Lagrange form is that even when we write down the
correct linear combination of Lagrange polynomials `i (x) to give P(x), we
do not have quick ways of getting:
◦ either the coefficients ai in the standard form,

P(x) = a0 + a1 x + a2 x 2 + · · · + an−1 x n−1

◦ or the values of the polynomial P(x) at locations x̄ in between the xi :

P(x̄) = ȳ

generally-speaking, the output values of a polynomial are the desired

numbers; this is the purpose of polynomial interpolation
moral: sometimes a formula for the answer is less useful than an
algorithm that leads to the numbers you actually want
. . . and we’ll get back to that!

Ed Bueler (MATH 310 Numerical Analysis) How to put a polynomial through points September 2012 28 / 29
conclusion: how to do polynomial interpolation

the problem is to find the degree n − 1 polynomial P(x) which goes

through n given points (xi , yi )
we have three methods, all of which do the job:
◦ the Vandermonde matrix method,
◦ the Newton polynomial form, and its triangular matrix method,
◦ and Lagrange’s direct formula for the polynomial
the first two require solving linear systems, while the last does not
◦ Lagrange’s direct formula requires us to simplify like crazy
◦ Newton gives easier linear systems (triangular) than does Vandermonde
◦ M ATLAB makes solving linear systems easy anyway
later in the course:
◦ we will address how accurate polynomial interpolation is
◦ we will get to one more algorithm, Neville’s algorithm, which gets the
polynomial values but skips finding any coefficients

Ed Bueler (MATH 310 Numerical Analysis) How to put a polynomial through points September 2012 29 / 29