Function Approximation

Pafnuty Chebyshev, 1821-1894

 Administration
 Assignment 3 is out; due on April 23 midnight
 Chapter 3 Excises 30 (a)(b)(c)(d), 33, 48, 49,
69 (20% each)
 Extra credit (10%): App 8

 Mid-term exam: Apr 30

Numerical Methods © Wen-Chieh Lin 2

 Function Approximation
 How does a computer approximate cos(x), exp(x),
and other non-polynomial functions?
 Efficient and accurate approximation is desired

 Chebyshev polynomials
 Orthogonal polynomials
 Converge faster than a Taylor series

 Rational functions
 Ratio of two polynomials
 Fourier series
 Seriesof sine and cosine terms
 Can approximate periodic or discontinuous functions
Numerical Methods © Wen-Chieh Lin 3
 Truncation Error in Taylor Series
 Recall that Taylor series
f " (a )
f ( x ) f (a ) f ' (a )( x a )  ( x a )2 
2!
f ( n ) (a ) f ( n 1)
()
f (a )   ( x a ) 
n
( x a ) n 1
n! ( n 1)!
 If approximating a function using first n terms

f ( n 1) ()
Error  ( x a )n 1
(n 1)!

 Error grows rapidly as x-value departs from x = a!

Numerical Methods © Wen-Chieh Lin 4
 Chebyshev Polynomials
 Recursive definition
Tn 1 ( x ) 2 xTn ( x ) Tn 1 ( x ), T0 ( x ) 1, T1 ( x ) x
T2 ( x ) 2 x 2 1
T3 ( x ) 4 x 3 3 x
T4 ( x ) 8 x 4 8 x 2 1

Numerical Methods © Wen-Chieh Lin 5

 Chebyshev Polynomials (cont.)
 Equivalent form
Tn ( x ) cos(n 
acos( x )) x cos()

 Proof

cos(n 1)cos(n 1)2 cos ncos

Tn 1 ( x ) Tn 1 ( x ) 2 xTn ( x )

Tn 1 ( x ) 2 xTn ( x ) Tn 1 ( x ), T0 ( x ) 1, T1 ( x ) x

Numerical Methods © Wen-Chieh Lin 6
 Chebyshev Series

T0 ( x ) 1 1 T0

T1 ( x ) x x T1
1
T2 ( x ) 2 x 1
2
x  (T2 T0 )
2

2
1
T3 ( x ) 4 x 3 3x x  (T3 3T1 )
3

4
1
T4 ( x ) 8 x 4 8 x 2 1 x  (T4 4T2 3T0 )
4

Numerical Methods © Wen-Chieh Lin 7

Chebyshev Series is More Economical
 On the interval [-1, 1], approximating a
function with a series of Chebyshev
polynomials is better than with a Tayor series
as it has a smaller maximum error

 Onan interval [a, b], we have to transform the

function to translate the x-value into [-1, 1]

~ x a
f ( x)  f ( ~
x) x 2 1
b a
Numerical Methods © Wen-Chieh Lin 8
 Example: Economizing a Power Series
 Maclaurin series is Taylor series expansion of
a function about 0 1
x  (T2 T0 )
2

x 2
 Maclaurin series of e
1
2 3 4 5 6 x 3
 (T3 3T1 )
x x x x x 4
e 1 x    
x
 
2 6 24 120 720 1
x  (T4 4T2 3T0 )
4

8
 Approximated by a Chebyshev series
1 1 1 1 1 1
e T0 T1   (T2 T0 )   (T3 3T1 )   (T4 4T2 3T0 )
x

2 2 6 4 24 8
1 1 1
 (10T1 5T3 )  (10T0 15T2 ) 
120 16 23040
Numerical Methods © Wen-Chieh Lin 9
 Example (cont.)
 Maclaurin series of ex truncated after 3rd
degree 2 3 4 5 6
x x x x x
e x 1 x      
2 6 24 120 720
x2 x3
1 x  
2 6
 Chebyshev series of ex truncated after 3rd
degree
e x 1.2661T0 1.1302T1 0.2715T2 0.0443T3 

0.9946 0.9973 x 0.5430 x 2 0.1772 x 3

Numerical Methods © Wen-Chieh Lin 10
 Example (cont.)
 Error of Maclaurin series grows rapidly when
x-value departs from zero
Error of Maclaurin series

Error of Chebyshev series

x
Numerical Methods © Wen-Chieh Lin 11
 Orthogonaliy of Chebyshev Polynomials
 Two polynomials p(x), q(x) are orthogonal on
an interval [a, b] if their inner product
b
p, q p( x ) q( x ) w( x )dx 0, w( x ) 0
a

 Setof polynomials is orthogonal if each

polynomial is orthogonal to each other

 Chebyshev Polynomials are orthogonal on

interval [-1, 1] with w( x ) 1 1 x 2

Numerical Methods © Wen-Chieh Lin 12

 Verify Orthogonaliy of
Chebyshev Polynomials
 Example: p(x) = T0(x), q(x) = T3(x)
1 1
p( x ), q( x )  1
(4 x 3x )
3
dx 0
1
1 x 2
 Matlab code
 int('(4*x^3-x)/sqrt(1-x^2)', -1 ,1)

Numerical Methods © Wen-Chieh Lin 13

 Least Squares Approximation using
Orthogonal Polynomials
 System of normal equations for a high-degree
polynomial fit is ill-conditioned
 Fitting data with orthogonal polynomials such
as Chebyshev polynomials can reduce the
condition number to about 5
f ( x ) a0T0 ( x ) a1T1 ( x )  anTn ( x )
 T0 ( x1 ) T1 ( x1 )  Tn ( x1 )  a0  y1 
 T0 ( x2 ) T1 ( x2 )  Tn ( x2 )  a1  y2 
 
  
      
 
  
 T0 ( xm ) T1 ( xm )  Tn ( xm ) an  ym 
Numerical Methods © Wen-Chieh Lin 14
 Rational Function Approximation
 Approximating a function with a Chebyshev
series has smaller maximum error than with a
Taylor series
 We can further improve the maximum error
using rational function approximation
a0 a1 x a2 x  an x
2 n
f ( x ) RN ( x ) 
1 b1 x b2 x  bm x
2 m

N n m 1
 Padé approximation: n ≥m
Numerical Methods © Wen-Chieh Lin 15
 Padé Approximation
 Represent a function in a Taylor series
expanded at x = 0
 Coefficients are determined by setting

f ( x ) RN ( x ) f ( i ) (0)
ci 
i!
a  a x a x 2
 a x n
( c0 c1 x c2 x 2  cN x N )  0 1 2 n
1 b1 x b2 x 2  bm x m
and equating coefficients
a0 c0 cn 1 b1cn  bm cn m 1 0
a1 b1c0 c1 
 cN b1cN 1  bm cN m 0
an bm cn m  b1cn 1Numerical
cn Methods © Wen-Chieh Lin 16
 Example: Padé Approximation
 Findarctan(x) ≈R10(x)
 Maclaurin series of arctan(x) through x10 is

1 3 1 5 1 7 1 9
arctan( x ) x  x  x  x  x
3 5 7 9
 f(x) = R10(x)

1 3 1 5 1 7 1 9 a0 a1 x a2 x 2 a3 x 3 a4 x 4 a5 x 5

x x  x  x  x 
3 5 7 9 1 b1 x b2 x 2 b3 x 3 b4 x 4 b5 x 5

Numerical Methods © Wen-Chieh Lin 17

 Example: Padé Approximation (cont.)
 Equating coefficients
1 3 1 5 1 7 1 9 a0 a1 x a2 x 2 a3 x 3 a4 x 4 a5 x 5
x x  x  x  x 
3 5 7 9 1 b1 x b2 x 2 b3 x 3 b4 x 4 b5 x 5
 x0 through x5  x6 through x10
a0 0 1
b 13 b3 0
5 1

a1 1 17 15 b2 13 b4 0

a2 b1 17 b1 15 b3 13 b5 0
a3 13 b2 1
9 17 b2 15 b4 0
a4 13 b1 b3
1
b1 17 b3 15 b5 0
a5 15 13 b2 b4 9
Numerical Methods © Wen-Chieh Lin 18
Padé Approximation vs. Maclaurin Series
7 3 64 5
 Padé Approximation x x  x
arctan( x )  9 945
10 2 5 4
1 x  x
 Maclaurin series 9 21
1 3 1 5 1 7 1 9
arctan( x ) x  x  x  x  x
3 5 7 9

Numerical Methods © Wen-Chieh Lin 19

 Padé Approximation vs. Maclaurin Series
1 3 1 5 1 7 1 9 7 3 64 5
arctan( x ) x  x  x  x  x x x  x
3 5 7 9 arctan( x )  9 945
10 2 5 4
1 x  x
9 21

Pade approximation

True value

Pade approximation
Maclaurin series
Numerical Methods © Wen-Chieh Lin 20
 Chebyshev-Padé Rational Function
 We can get a rational function approximation
from a Chebyshev series expansion of a
function as well
 Example:
e x 1.2661T0 1.1302T1 0.2715T2 0.0443T3 
 We first form
a0 a1T1 a2T2
1.2661T0 1.1302T1 0.2715T2 0.0443T3 
1 b1T1

 Then expand the numerator and set the

coefficients of each degree of the T’
s to zero
Numerical Methods © Wen-Chieh Lin 21
 Recall that Tn ( x ) cos(n)
1
cos(n) cos(m)  [cos(n m)cos(n m )]
2
1
Tn ( x )Tm ( x )  [Tn m ( x ) T n m ( x )]
2
 Apply the last equation to
a0 a1T1 a2T2
1.2661T0 1.1302T1 0.2715T2 0.0443T3 
1 b1T1
 The numerator becomes
1.2661b1
1.2661T0 1.1302T1 0.2715T2 0.0443T3  (T1 T1 )
2
1.1302b1 0.2715b1 0.0443b1
 (T2 T0 )  (T3 T1 )  (T4 T2 )
2 2 2
a0 a1T1 a2T2
Numerical Methods © Wen-Chieh Lin 22
 Equating the Coefficients of Each Degree
of T’
s in Numerator
a0 a1T1 a2T2 
1.2661b1
1.2661T0 1.1302T1 0.2715T2 0.0443T3  (T1 T1 )
2
1.1302b1 0.2715b1 0.0443b1
 (T2 T0 )  (T3 T1 )  (T4 T2 )
2 2 2
1.1302b1 0.2715b1
a0 1.2661  a1 1.1302 1.2661b1 
2 2
1.1302b1 0.0443b1 0.2715b1
a2 0.2715   0 0.0443 
2 2 2
1. 0817 0 . 6727 T 0 .0799 T 1 .0018 0 .6727 x  0 .1598 x 2
ex  1 2

1 0.3263T1 1 0.3263 x
Numerical Methods © Wen-Chieh Lin 23
Chebyshev vs. Chebyshev-Padé Rational

Numerical Methods © Wen-Chieh Lin 24

 Error Plot

Chebyshev serires

Pade-Chebyshev
rational function

Numerical Methods © Wen-Chieh Lin 25

 Fourier Series
 Representing a function as a trigonometric
series
A0 
f ( x )   An cos(nx ) Bn sin( nx )
2 n 1

 f(x)need be integrable so that the coefficients

can be computed

Numerical Methods © Wen-Chieh Lin 26

 Properties of Orthogonality

  0, n 0


sin( nx )dx 0



cos(nx )dx 
2, n 0




sin( nx ) cos( mx )dx 0


 0, n m


sin(nx ) sin( mx )dx 
, n m
 0, n m


cos(nx ) cos(mx )dx 
, n m
Numerical Methods © Wen-Chieh Lin 27
 Computing Fourier Coefficients
using Orthogonality
A0 
f ( x )   An cos(nx ) Bn sin( nx )
2 n 1
     
A0
 f ( x )dx  dx An cos(nx )dx Bn sin(nx )dx
  2  
n1 n 1

A00 0
  A  

 cos( mx ) f ( x ) dx  0
cos(mx )dx An cos(nx ) cos(mx )dx
  2 
n1
 
Bn sin(nx ) cos(mx )dx

n
1

0 Am0
1 
Am  cos(mx ) f ( x )dx, m 1,2....
 
Numerical Methods © Wen-Chieh Lin 28
 Computing Fourier Coefficients
using Orthogonality (cont.)
A0 
f ( x )   An cos(nx ) Bn sin( nx )
2 n 1
  A  

 sin( mx ) f ( x ) dx  0
sin(mx )dx An cos(nx ) sin(mx )dx
  2 
n 1
 
Bn sin(nx ) sin(mx )dx

n
1

0 Bm0

1 
Bm  f ( x ) sin(mx )dx, m 1,2....
 
Numerical Methods © Wen-Chieh Lin 29
 Fourier Series for Periods other than 2π
 Letthe period of f(x) be P
 Consider that f(x) is periodic in [-P/2, P/2]

 Change of variable y 
x
P2
1 
Am  cos(mx ) f ( x )dx, m 1,2....
 
2 P2 2mx
Am   f ( x ) cos( )dx, m 1,2....
P P 2 P
1 
Bm  f ( x ) sin(mx )dx, m 1,2....
 
2 P2 2mx
Bm   f ( x ) sin( )dx, m 1,2....
P P 2 P
Numerical Methods © Wen-Chieh Lin 30
Fourier Series for Periods other than 2π

A0 
 2nx 2nx 
f ( x )   An cos( ) Bn sin( )
2 n 1  P P 

2 P2 2nx
An   f ( x ) cos( )dx, n 1,2....
P P 2 P

2 P2 2nx
Bn   f ( x ) sin( )dx, n 1,2....
P P 2 P
Numerical Methods © Wen-Chieh Lin 31
 Example: f(x) = x, x in [-π,π]
1 
Am  cos(mx ) f ( x )dx, m 1,2....
 

1  1  sin(mx ) sin(mx ) 
 cos(mx ) xdx  x  dx  0
   m m 


1 
Bm  f ( x ) sin(mx )dx, m 1,2....
 

1  1  cos(mx ) cos(mx ) 
 sin(mx ) xdx  x  dx 
   m m 


2 cos(m) 2( 1) m 1
  
2( 1) m 1
m m x  sin( mx )
m
1 m
Numerical Methods © Wen-Chieh Lin 32
 Fourier Series for Nonperiodic Functions
 Chop the interval of interest [0, L]

Numerical Methods © Wen-Chieh Lin 33

 Fourier Series for Nonperiodic Functions
 Solution 1: make an even function by
reflecting the segment about y-axis, f(x)=f(-x)

Numerical Methods © Wen-Chieh Lin 34

 Fourier Series for Nonperiodic Functions
 Solution
2: make an odd function by reflecting
the segment about the origin, f(x) = -f(x)

Numerical Methods © Wen-Chieh Lin 35

Properties of Even and Odd Functions

 Product of two even functions is an even function

 Product of two odd functions is an even function

 Product of an even function and an odd

function is an odd function
L L
if f ( x ) is even f ( x )dx 2f ( x )dx
L 0

L
if f ( x ) is odd f ( x)dx 0
L
Numerical Methods © Wen-Chieh Lin 36
Fourier Series for Even Functions
2 P2 2mx
Am   f ( x ) cos( )dx, m 1,2....
P P 2 P
2 L 2mx
 f ( x ) cos( )dx even function
2 L L 2L
1 L mx
 2 f ( x ) cos( )dx
L 0 L
2 P2 2mx
Bm   f ( x ) sin( )dx, m 1,2....
P P 2 P
0 odd function
Numerical Methods © Wen-Chieh Lin 37
Fourier Series for Odd Functions
2 P2 2mx
Am   f ( x ) cos( )dx, m 1,2....
P P 2 P
0 odd function
2 P2 2mx
Bm   f ( x ) sin( )dx, m 1,2....
P P 2 P

2 L 2mx
 f ( x ) sin( )dx even function
2L L 2L
1 L mx
 2 f ( x ) sin( )dx
L 0 L
Numerical Methods © Wen-Chieh Lin 38
Example: Fourier Series of Nonperiodic
Functions—even extension y

0, 0 x 1

f ( x )  1
1, 1 x 2

2 L mx x
Am  f ( x ) cos( )dx -2 -1 1 2
L 0 L
mx  0, m even
2 2 
 f ( x ) cos( )dx  ( 1)( m 1) 2
2 0 2 2 , m odd
 m
2 2
A0   f ( x )dx 1
2 0
A0   2nx 2nx 
f ( x )   An cos( ) Bn sin( )
2 n 1  P P 
Numerical Methods © Wen-Chieh Lin 39
Example: Fourier Series of Nonperiodic
Functions—odd extension y

0, 0 x 1
 1
f ( x ) 
1, 1 x 2
 x
2 L mx -2 -1 1 2
Bm  f ( x ) sin( )dx -1
L 0 L
2 2 mx 2  m 
 f ( x ) sin( )dx  cos(m) cos( )
2 0 2 m 2 
A0   2nx 2nx 
f ( x )   An cos( ) Bn sin( )
2 n 1  P P 
n
2  cos(n) cos( 2 ) nx
  sin( )
n 1 n 2
Numerical Methods © Wen-Chieh Lin 40