Scribd
Upload
57 views

CE 205: Numerical Methods: Curve-Fitting (Linear and Non-Linear Regression)

This document discusses numerical methods for curve fitting and regression analysis. It covers linear and nonlinear regression techniques, including linear interpolation, polynomial interpolation, and linear regression. Linear regression finds the best-fit linear relationship between a dependent variable and independent variables by minimizing the residual errors. The document also discusses assessing the goodness of fit, extending linear regression to polynomials and multiple variables, and linearizing nonlinear relationships for regression analysis.

Uploaded by

Mishkat Ahmed Naoshad
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
57 views

CE 205: Numerical Methods: Curve-Fitting (Linear and Non-Linear Regression)

This document discusses numerical methods for curve fitting and regression analysis. It covers linear and nonlinear regression techniques, including linear interpolation, polynomial interpolation, and linear regression. Linear regression finds the best-fit linear relationship between a dependent variable and independent variables by minimizing the residual errors. The document also discusses assessing the goodness of fit, extending linear regression to polynomials and multiple variables, and linearizing nonlinear relationships for regression analysis.

Uploaded by

Mishkat Ahmed Naoshad
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 15

CE 205: Numerical Methods

Curve-fitting (Linear and


Non-linear Regression)

CE 205: Numerical Methods Dr. Tanvir Ahmed


Motivation
sampling data are acquired at
discrete points.
- but the values at undefined points are
wanted in many engineering applications

Interpolation
-for very precise data
- Ex: density of water at different temp.

Least-squares Regression
-for imprecise/noisy data
- Purpose is trend analysis

CE 205: Numerical Methods Dr. Tanvir Ahmed


Interpolation and linear regression

CE 205: Numerical Methods Dr. Tanvir Ahmed


Interpolation and linear regression

Least-square regression
Visually sketch a line that conforms to the
data

Linear interpolation
Connect the data points consecutively by
lines segments

Polynomial interpolation
Connect the data points consecutively by
simple curves

CE 205: Numerical Methods Dr. Tanvir Ahmed


Linear Least Squares Regression
A method to determine the “best” coefficients in a linear
model for given data set [(x1,y1), (x2,y2)……(xn,yn)].

A straight line model: y  a0  a1 x  e

Coefficients Residual Error

“Best” coefficients can be obtained by minimizing the sum


of the squares of the estimate residuals.
n n
Sr   e  yi  a0  a1 xi 
2 2
i
i1 i1

CE 205: Numerical Methods Dr. Tanvir Ahmed


Estimating the best-fit parameters

n xi yi   xi  yi
a1 
n x   x 
2
2
i i

a0  y  a1 x

Sr
Standard error of the estimate: s y/ x 
n2

CE 205: Numerical Methods Dr. Tanvir Ahmed


Improvement by linear regression

(a) the spread of data around the (b) the spread of the data around
mean of the dependent data the best fit line:

Sum of squares of residuals Sum of squares of residuals between


between data points and the mean data points and the regression line
n
S r    yi  a0  a1 xi 
n
St    yi  y 
2 2

i 1 i 1

CE 205: Numerical Methods Dr. Tanvir Ahmed


Assessing the “Goodness” of fit

St-Sr quantifies the improvement or error reduction due to


describing data in terms of a straight line rather than as an
average value

St  Sr
r 
2

St

r2 represents the percentage of the original uncertainty


explained by the model (Coefficient of Determination)
- For 
a perfect fit, Sr=0 and r2=1.
2
- If r =0, there is no improvement over simply picking the mean.
- If r2<0, the model is worse than simply picking the mean!

CE 205: Numerical Methods Dr. Tanvir Ahmed


Polynomial regression

Some engineering data is


poorly represented by a
straight line.

A higher order polynomial


(e.g. parabola) may be well-
suited

CE 205: Numerical Methods Dr. Tanvir Ahmed


Polynomial regression
For a second order polynomial, the least-square regression
can be extended to
n n
Sr   e  yi  a0  a1 xi  a x
2
i
2 2
2 i 
i1 i1

The coefficients can be determined using Gaussian


elimination:


CE 205: Numerical Methods Dr. Tanvir Ahmed


Polynomial regression
The two-dimensional case can easily be extended to m-th
order polynomial
y  a0  a1 x  a2 x 2    am x m  e

The sum of squares of residuals:


n n
Sr   e  yi  a0  a1 xi  a x  a x
2
i
2
2 i m i 
m 2

 i1 i1

Can you formulate the matrix to determine the



coefficients?

CE 205: Numerical Methods Dr. Tanvir Ahmed


Multiple linear regression

y is a linear function of two or


more independent variables:

 a0  a1x1  a2 x2 am xm
y

Again, the best fit is obtained


by minimizing the sum of the
squares of the estimate
residuals:
n n
Sr   ei2  yi  a0  a1 x1,i  a2 x2,i am xm,i 
2

 i1 i1

CE 205: Numerical Methods Dr. Tanvir Ahmed


Nonlinear relationship
Linear regression is predicated on the fact that the
relationship between the dependent and independent
variables is linear - this is not always the case.

Three common examples are:

exponential : y  1e 1x

power : y   2 x 2
x
saturation - growth - rate : y   3
3  x

CE 205: Numerical Methods Dr. Tanvir Ahmed


Linearizing nonlinear relationship

this may involve taking logarithms or inversion:

M odel Nonlinear Linearized

exponential : y  1e1 x ln y  ln 1  1 x

power : y  2 x 2 log y  log  2  2 log x


x 1 1 3 1
saturation- growth - rate : y   3  
3  x y 3 3 x

CE 205: Numerical Methods Dr. Tanvir Ahmed


Linearizing nonlinear relationship

CE 205: Numerical Methods Dr. Tanvir Ahmed

You might also like