Week 9: Polynomial Interpolation

and Least Squares Fitting

1 Interpolation
Given some x-y data (x1 , y1 ) , (x2 , y2 ) , . . . , (xn , yn ), the goal of polynomial interpolation is to
find a polynomial that passes through all of the data points. For example, suppose you have
three data points: (0, 1), (0.5, −2), and (1, 3). With three points, you can find a quadratic
polynomial that passes through all of them. First, assume the polynomial looks like

p(x) = c1 + c2 x + c3 x2 .

Next, force the polynomial to match the data at each of the x-coordinates:

p(0) =c1 + c2 · 0 + c3 · 02 = 1,
p(0.5) =c1 + c2 (0.5) + c3 (0.5)2 = −2,
p(1) =c1 + c2 · 1 + c3 · 12 = 3.

This gives a system of three equations and three unknowns, A~c = ~y, which can be solved in
Matlab using the backslash \ operator. In this example, the linear system ends up looking
like this:     
1 0 02 c1 1
 1 0.5 (0.5)2   c2  =  −2  .
1 1 12 c3 3
In general, whenever you have n data points, you can always find a polynomial of degree
n − 1 which passes through all of them:
    
1 x1 x21 . . . xn−1
1 c1 y1
 1 x2 x2 . . . xn−1   c2   y2 
 2 2    
 1 x3 x2 . . . xn−1   c3 y3
=
  
 3 3  
 .. .. .. ..   ..   .. 
 . . . ... .  .   . 
2 n−1
1 xn xn . . . x n cn yn

On the other hand, polynomials of very high degree can have undesirable properties, so next
we will consider an alternative which, in Matlab, is basically the same process.

2 Least Squares Fitting

Details on the mathematical derivation of the least-squares method can be found
in your book, on pages 162-163 and in Problem 45 of Section 3.4.
If the number of points is larger than the degree of polynomial that you want to use, then
the linear system for determining the coefficients will be over-determined (more rows than
columns). To find the least-squares polynomial of a given degree, you carry out the same
 Figure 1: Example of least squares fitting with polynomials of degrees 1, 2, and 3.

process as we did for interpolation, but the resulting polynomial will not interpolate the
data, it will just be “close”. Here are some examples of what the linear system will look like
for determining the least-squares polynomial coefficients:
   
1 x1 y1
 1 x2      y2 

 c1
 
Linear:  1 x3  =  y3 
  
 .. ..  c2  .. 
 . .   . 
1 xn yn
   
1 x1 x21 y1
 1 x2 x2  
 y2 
 
c1

 2 
2
Quadratic:  1 x3 x3   c2  =  y3 
   
 .. .. ..  c  .. 
 . . .  3  . 
2
1 xn xn yn
   
1 x1 x21 x31   y1
 1 x 2 x 2 x 3  c1  y2 
2 2 
c
   
 1 x3 x2 x3   2  y3 
Cubic:  3 3    =
..   c3   .. 

 .. .. ..
 . . . .  c  . 
2 3 4
1 xn xn xn yn

If you were solving one of these over-determined linear systems by hand, you would multiply
both sides by AT , which would give you a square linear system:

AT A~c = AT ~y.
This method will still work in Matlab. You can set

c = (A.’*A)\(A.’*b)

However, this is unnecessary, because Matlab can recognize an overdetermined system and
find the least-squares solution automatically. In other words, you can type c=A\b and it will
give you the same solution.

3 Homework #8
Make a function called leastSquares.m, which, given some data points defined by the
vectors x and y, will find and evaluate the least-squares polynomial. The user should be able
to choose the degree d of polynomial, and the vector X of evaluation points. Hints:

1. The first line should declare that the m-file is a function with four inputs and one
output:

function Y = leastSquares( x, y, d, X )

• The column-vectors x and y specify the data that you know.

• The number d is the degree of the polynomial that you want to use.
• The column-vector X has all of the x-coordinates where you want to evaluate the
least-squares polynomial.
• The output Y will be a column-vector which contains the values of the least-squares
polynomial at the x-coordinates given in X.

2. Here are some guidelines for what your function should do:

(a) Initialize the matrix A to be all zeros with length(x) rows and d+1 columns.
(b) Initialize the matrix B to be all zeros with length(X) rows and d+1 columns.
(c) Define the entries of the matrix A and the matrix B using a for-loop over the
columns:
for j = 1 : d+1
-set the jth column of A equal to x.^(j-1)
-set the jth column of B equal to X.^(j-1)
end
(d) Find the least-squares solution to the over-determined linear system A*c=y. Even
though the linear system may not be square, you can still use the backslash
operator to solve for c. Matlab will automatically find the least-squares solution
if you type c=A\y. c contains the coefficients for the least-squares polynomial.
(e) Evaluate the least-squares polynomial at the coordinates given in X, by setting
Y=B*c.
Once your function leastSquares.m is complete, test it by picking a function, sampling it
at a few points, and using these as your x and y vectors. For example, you could sample the
2
function f (x) = e−x at six equally spaced points from 0 to 1 by typing these lines:

f = @(x) exp(-x.^2);
x = (0:.2:1).’;
y = f(x);

Then, you can choose a polynomial degree, pick which x-coordinates you want to evaluate
at, and call leastSquares.m:

d = 2;
X = (0:.01:1).’;
Y = leastSquares( x, y, d, X );
2
Finally, view the results of your labor by plotting the original function e−x and your least
squares fit (represented in your program by the x-values X and the y-values Y ) on the same
plot. Place a legend on the plot (try help legend in the command line if you don’t know
how to use this MATLAB function) to label the original function and the least squares
fit. What happens if you change d from 2 to 3? Does it look like the cubic least-squares
polynomial provides a better fit?