CURVE FITTING

TECHNIQUES
NUMERICAL METHODS AND ANALYSIS
LESSON 5

Presented by: Engr. Romulo C. Cruz, Jr.

Thursday, 10
November 2022
 LEARNING OUTCOMES
At the end of the lesson, the students should be able to:
• apply the least squares method to find the best fit curve for a
given set of data

• use technology tools to generate regression models and data

Presented by Engr. Romulo C. Cruz, Jr. Electrical Engineering Department College of Engineering& Architecture
 CURVE FITTING
• the process of finding the best fit curve for a given set of data

• the representation of the relationship between two variables by

means of an algebraic equation

• can be used as an aid for data visualization, to infer values of a

function where no data are available, and to summarize the
relationships among two or more variables

https://en.wikipedia.org/wiki/Curve_fitting#:~:text=Fitted%20curves%20can%20be%20used,among%20two%20or%20more%20variables.

Presented by Engr. Romulo C. Cruz, Jr. Electrical Engineering Department College of Engineering& Architecture
 BEST FIT CURVE

In approximate models, the line of best fit does not

necessarily pass through any of the points in a scatterplot.
Nevertheless, the line should pass as close as possible
to all the points! That means that the sum of the errors (also
called residue) of all the points to be as small as possible.
 Presented by Engr. Romulo C. Cruz, Jr. Electrical Engineering Department College of Engineering& Architecture

LINEAR AND NON-LINEAR

REGRESSION
Linear regression – Curve Fitting for Non-linear regression – Curve Fitting for
linear relationships non-linear relationships

http://www.notespoint.com/curvefitting-leastsquares/
 LEAST SQUARES METHOD

• assumes that the best fit curve of a given type is the curve
that has the minimum sum of the squares of the deviations
(least square error) from a given set of data

• can be used for establishing linear as well as non-linear

relationships.

Presented by Engr. Romulo C. Cruz, Jr. Electrical Engineering Department College of Engineering& Architecture
 LEAST SQUARE METHOD
• Let the data points be (𝑥1 , 𝑦1 ), (𝑥2 , 𝑦2 ),…, (𝑥𝑛 , 𝑦𝑛 ) where x is the independent
variable and y is the dependent variable, and that the fitting curve f(x) has the
following deviations (or errors or residuals) from each data points:
𝑑1 = 𝑦1 − 𝑓 𝑥1 , 𝑑2 = 𝑦2 − 𝑓 𝑥2 , … , 𝑑𝑛 = 𝑦𝑛 − 𝑓 𝑥𝑛

• Some of these residuals will be positive and others negative; to give equal
weightage to each, square each of these and form their sum:
𝐷 = 𝑑1 2 + 𝑑2 2 + ⋯ + 𝑑𝑛 2

• According to the method of least squares, the best fitting curve has the property
𝐷 = 𝑑1 2 + 𝑑2 2 + ⋯ + 𝑑𝑛 2 = σ𝑛𝑖=1 𝑑𝑖 2 = σ𝑛𝑖=1 𝑦𝑖 − 𝑓 𝑥𝑖 2
= a minimum

Presented by Engr. Romulo C. Cruz, Jr. Electrical Engineering Department College of Engineering& Architecture
 Presented by Engr. Romulo C. Cruz, Jr. Electrical Engineering Department College of Engineering& Architecture

LEAST SQUARES METHOD FOR FITTING A

LINEAR REGRESSION
• Consider that the equation of a straight line of the form 𝒚 = 𝒎𝒙 + 𝒃 is to be fitted to the
n-data points (𝑥1 , 𝑦1 ), (𝑥2 , 𝑦2 ),…, (𝑥𝑛 , 𝑦𝑛 ), where n ≥ 2, b is the y-intercept, and m is the
slope.

• For the general point (𝑥𝑖 , 𝑦𝑖 ), the vertical distance of this point from the line 𝒚 = 𝒎𝒙 + 𝒃
is the deviation 𝑑𝑖 so that 𝑑𝑖 = 𝑦𝑖 − 𝑓 𝑥𝑖 = 𝑦𝑖 − 𝑚𝑥𝑖 − 𝑏

• By the method of least squares:

𝐷 = σ𝑛𝑖=1 𝑦𝑖 − 𝑚𝑥𝑖 − 𝑏 2
 LEAST SQUARES METHOD FOR FITTING A
LINEAR REGRESSION
Which will be minimum when
𝜕𝐷
= 0; −2 σ𝑛𝑖=1 𝑦𝑖 − 𝑚𝑥𝑖 − 𝑏 = 0
𝜕𝑏
𝜕𝐷
= 0; −2 σ𝑛𝑖=1 𝑥𝑖 𝑦𝑖 − 𝑚𝑥𝑖 − 𝑏 = 0
𝜕𝑚

simplifying the above partial derivatives gives

σ𝑛𝑖=1 𝑦𝑖 = 𝑚 σ𝑛𝑖=1 𝑥𝑖 + 𝑏 σ𝑛𝑖=1 1 = 𝑚 σ𝑛𝑖=1 𝑥𝑖 + 𝑏𝑛
σ𝑛𝑖=1 𝑥𝑖 𝑦𝑖 = 𝑚 σ𝑛𝑖=1 𝑥𝑖 2 + 𝑏 σ𝑛𝑖=1 𝑥𝑖
Presented by Engr. Romulo C. Cruz, Jr. Electrical Engineering Department College of Engineering& Architecture
 LEAST SQUARES METHOD FOR FITTING A
LINEAR REGRESSION

The following are the auxiliary equations for the best fitting
straight line 𝒚 = 𝒎𝒙 + 𝒃:

σ 𝑦 = 𝑚 σ 𝑥 + 𝑏𝑛

σ 𝑥𝑦 = 𝑚 σ 𝑥 2 + 𝑏 σ 𝑥

Presented by Engr. Romulo C. Cruz, Jr. Electrical Engineering Department College of Engineering& Architecture
 EXAMPLE PROBLEM 1
Fit a straight line to the following set of data points: (2, 2), (5,
4), (6, 6), (9, 9), and (11, 10). Determine y when x = 9.1.
Also, make a scatter plot representing these data points.

SOLUTION:

The Table shows the given set of data points

Presented by Engr. Romulo C. Cruz, Jr. Electrical Engineering Department College of Engineering& Architecture
 EXAMPLE PROBLEM 1
SOLUTION…
This gives the following auxiliary xi2
i xi yi xiyi
equations:
1 2 2 4 4
σ 𝑦 = 𝑚 σ 𝑥 + 𝑏𝑛
2 5 4 20 25
31 = 33𝑚 + 5𝑏 3 6 6 36 36
4 9 9 81 81
σ 𝑥𝑦 = 𝑚 σ 𝑥 2 + 𝑏 σ 𝑥 5 11 10 110 121

251 = 267𝑚 + 33𝑏 Σ 33 31 251 267

Presented by Engr. Romulo C. Cruz, Jr. Electrical Engineering Department College of Engineering& Architecture
 EXAMPLE PROBLEM 1
SOLUTION…
By Cramer’s rule:
31 5
251 33 1023−1255 −232
𝑚= 33 5 = = = 0.9431
1089−1335 −246
267 33

33 31
267 251 8283−8277
𝑏= = = −0.0244
−246 −246

Presented by Engr. Romulo C. Cruz, Jr. Electrical Engineering Department College of Engineering& Architecture
 EXAMPLE PROBLEM 1

SOLUTION…
Therefore, the required equation of the straight line is

𝒚 = 𝟎. 𝟗𝟒𝟑𝟏𝒙 − 𝟎. 𝟎𝟐𝟒𝟒

and when 𝑥 = 9.1, 𝒚 = 𝟖. 𝟓578

Presented by Engr. Romulo C. Cruz, Jr. Electrical Engineering Department College of Engineering& Architecture
 EXAMPLE PROBLEM 1
SCATTER PLOT OF EXAMPLE PROBLEM 1
12
SOLUTION…
10
The scatter plot of the given data
points can be obtained using MS 8

Excel; it shows that a straight line is

the best fit curve of the given set of
6

data. 4

0
0 2 4 6 8 10 12

Presented by Engr. Romulo C. Cruz, Jr. Electrical Engineering Department College of Engineering& Architecture
 LEAST SQUARES METHOD FOR FITTING A
NONLINEAR REGRESSION
• Consider that the equation of a parabola of the form 𝑦 = 𝑎𝑥 2 + 𝑏𝑥 + 𝑐 is to
be fitted to the n-data points (𝑥1 , 𝑦1 ), (𝑥2 , 𝑦2 ),…, (𝑥𝑛 , 𝑦𝑛 )

• For the general point (𝑥𝑖 , 𝑦𝑖 ), the vertical distance of this point from the
parabola 𝑦 = 𝑎𝑥 2 + 𝑏𝑥 + 𝑐 is the deviation 𝑑𝑖 so that 𝑑𝑖 = 𝑦𝑖 − 𝑓 𝑥𝑖 = 𝑦𝑖 −
𝑎𝑥𝑖 2 − 𝑏𝑥𝑖 − 𝑐

• By the method of least squares:

𝐷 = σ𝑛𝑖=1 𝑦𝑖 − 𝑎𝑥𝑖 2 − 𝑏𝑥𝑖 − 𝑐 2

Presented by Engr. Romulo C. Cruz, Jr. Electrical Engineering Department College of Engineering& Architecture
 LEAST SQUARES METHOD FOR FITTING A
NONLINEAR REGRESSION
Which will be minimum when
𝜕𝐷
= 0; −2 σ𝑛𝑖=1 𝑥𝑖 2 𝑦𝑖 − 𝑎𝑥𝑖 2 − 𝑏𝑥𝑖 − 𝑐 = 0
𝜕𝑎

𝜕𝐷
= 0; −2 σ𝑛𝑖=1 𝑥𝑖 𝑦𝑖 − 𝑎𝑥𝑖 2 − 𝑏𝑥𝑖 − 𝑐 = 0
𝜕𝑏

𝜕𝐷
= 0; −2 σ𝑛𝑖=1 𝑦𝑖 − 𝑎𝑥𝑖 2 − 𝑏𝑥𝑖 − 𝑐 = 0
𝜕𝑐

Presented by Engr. Romulo C. Cruz, Jr. Electrical Engineering Department College of Engineering& Architecture
 LEAST SQUARES METHOD FOR FITTING A
NONLINEAR REGRESSION

Simplifying these results gives

σ𝑛𝑖=1 𝑥𝑖 2 𝑦𝑖 = 𝑎 σ𝑛𝑖=1 𝑥𝑖 4 + 𝑏 σ𝑛𝑖=1 𝑥𝑖 3 + 𝑐 σ𝑛𝑖=1 𝑥𝑖 2

σ𝑛𝑖=1 𝑥𝑖 𝑦𝑖 = 𝑎 σ𝑛𝑖=1 𝑥𝑖 3 + 𝑏 σ𝑛𝑖=1 𝑥𝑖 2 + 𝑐 σ𝑛𝑖=1 𝑥𝑖

σ𝑛𝑖=1 𝑦𝑖 = 𝑎 σ𝑛𝑖=1 𝑥𝑖 2 + 𝑏 σ𝑛𝑖=1 𝑥𝑖 + 𝑐 σ𝑛𝑖=1 1 = 𝑎 σ𝑛𝑖=1 𝑥𝑖 2 + 𝑏 σ𝑛𝑖=1 𝑥𝑖 + 𝑐𝑛

Presented by Engr. Romulo C. Cruz, Jr. Electrical Engineering Department College of Engineering& Architecture
 LEAST SQUARES METHOD FOR FITTING A
NONLINEAR REGRESSION

The following are the auxiliary equations for the best fit curve:

σ 𝑥 2𝑦 = 𝑎 σ 𝑥 4 + 𝑏 σ 𝑥 3 + 𝑐 σ 𝑥 2

σ 𝑥𝑦 = 𝑎 σ 𝑥 3 + 𝑏 σ 𝑥 2 + 𝑐 σ 𝑥

σ 𝑦 = 𝑎 σ 𝑥 2 + 𝑏 σ 𝑥 + 𝑐𝑛

Presented by Engr. Romulo C. Cruz, Jr. Electrical Engineering Department College of Engineering& Architecture
FITTING POLYNOMIAL FUNCTION
USING LEAST SQUARES METHOD
In general for a polynomial function
𝑦 = 𝑘1 + 𝑘2 𝑥 + 𝑘3 𝑥 2 + ⋯ + 𝑘𝑛 𝑥 𝑛−1

the best fit curve for the set of data (𝑥1 , 𝑦1 ),…,(𝑥𝑛 , 𝑦𝑛 ) by Least Squares Method
can be obtained from the values of 𝑘1 , 𝑘2 ,…, 𝑘𝑛 of the auxiliary equations
σ 𝑦𝑖 = 𝑛𝑘1 + 𝑘2 σ 𝑥𝑖 + 𝑘3 σ 𝑥𝑖 2 + ⋯ + 𝑘𝑛 σ 𝑥𝑖 𝑛−1
σ 𝑥𝑖 𝑦𝑖 = 𝑘1 σ 𝑥𝑖 + 𝑘2 σ 𝑥𝑖 2 + ⋯ + 𝑘𝑛 σ 𝑥𝑖 𝑛
⋮
σ 𝑥𝑖 𝑛−1 𝑦𝑖 = 𝑘1 σ 𝑥𝑖 𝑛−1 + 𝑘2 σ 𝑥𝑖 𝑛 + ⋯ + 𝑘𝑛 σ 𝑥𝑖 2𝑛−2

Presented by Engr. Romulo C. Cruz, Jr. Electrical Engineering Department College of Engineering& Architecture
 EXAMPLE PROBLEM 2
Fit a polynomial of degree two using least squares method for
the following data: (1, 5), (2, 12), (3, 26), (4, 60), and (5,
97). Make a scatter plot representing these set of data.

SOLUTION:

The Table shows the given set of data points

Presented by Engr. Romulo C. Cruz, Jr. Electrical Engineering Department College of Engineering& Architecture
 EXAMPLE PROBLEM 2

SOLUTION…
i xi yi xiyi xi2 xi2yi xi3 xi4
1 1 5 5 1 5 1 1
2 2 12 24 4 48 8 16
3 3 26 78 9 234 27 81
4 4 60 240 16 960 64 256
5 5 97 485 25 2425 125 625
Σ 15 200 832 55 3672 225 979

Presented by Engr. Romulo C. Cruz, Jr. Electrical Engineering Department College of Engineering& Architecture
 EXAMPLE PROBLEM 2
SOLUTION…
which gives the following auxiliary equations:
σ 𝑥2𝑦 = 𝑎 σ 𝑥4 + 𝑏 σ 𝑥3 + 𝑐 σ 𝑥2
3672 = 979𝑎 + 225𝑏 + 55𝑐

σ 𝑥𝑦 = 𝑎 σ 𝑥 3 + 𝑏 σ 𝑥 2 + 𝑐 σ 𝑥
832 = 225𝑎 + 55𝑏 + 15𝑐

σ 𝑦 = 𝑎 σ 𝑥2 + 𝑏 σ 𝑥 + 𝑐 𝑛
200 = 55𝑎 + 15𝑏 + 5𝑐

Presented by Engr. Romulo C. Cruz, Jr. Electrical Engineering Department College of Engineering& Architecture
 EXAMPLE PROBLEM 2
SOLUTION…
By Cramer’s Rule:

3672 225 55
832 55 15
200 15 5 3672 275−225 −225 4160−3000 +55(12480−11000) 4000
𝑎= 979 225 55 = 979 275−225 −225 1125−825 +55(3375−3025)
= 700
225 55 15
55 15 5

𝑎 = 5.71

Presented by Engr. Romulo C. Cruz, Jr. Electrical Engineering Department College of Engineering& Architecture
 EXAMPLE PROBLEM 2

SOLUTION…

979 3672 55
225 832 15
55 200 5 979 4160−3000 −3672 1125−825 +55(45000−45760) −7760
𝑏= 979 225 55 = =
700 700
225 55 15
55 15 5

𝑏 = −11.09

Presented by Engr. Romulo C. Cruz, Jr. Electrical Engineering Department College of Engineering& Architecture
 EXAMPLE PROBLEM 2

SOLUTION…

979 225 3672

225 55 832
55 15 200 979 11000−12480 −225 45000−45760 +3672(3375−3025) 7280
𝑐= 979 225 55 = =
700 700
225 55 15
55 15 5

𝑐 = 10.4

Presented by Engr. Romulo C. Cruz, Jr. Electrical Engineering Department College of Engineering& Architecture
 EXAMPLE PROBLEM 2
SOLUTION… SCATTER PLOT OF EP2

Therefore, the required equation of

120

the polynomial is 100

𝑦 = 𝑎𝑥 2 + 𝑏𝑥 + 𝑐 80

𝒚 = 𝟓. 𝟕𝟏𝒙𝟐 − 𝟏𝟏. 𝟎𝟗𝒙 + 𝟏𝟎. 𝟒 60

40

The scatter plot shows that a 20

parabola is the best fit curve of the
given set of data. 0
0 1 2 3 4 5 6

Presented by Engr. Romulo C. Cruz, Jr. Electrical Engineering Department College of Engineering& Architecture
 FITTING A NONLINEAR RELATIONSHIP
THAT CAN BE CONVERTED INTO LINEAR
FORM

• Nonlinear relationships of the form 𝑦 = 𝑎𝑏 𝑥 , 𝑦 = 𝑎𝑥 𝑏 , and 𝑦 =

𝑎𝑒 𝑏𝑥 can be converted into the linear form by applying
logarithms on both sides of the equation

• The least squares method can then be applied.

Presented by Engr. Romulo C. Cruz, Jr. Electrical Engineering Department College of Engineering& Architecture
 EXAMPLE PROBLEM 3

Fit a curve of the type 𝑦 = 𝑏𝑒 𝑚𝑥 to the following data: (1, 10), (5, 15), (7,
12), (9, 15), and (12, 21).

SOLUTION:
Take the natural logarithm of both sides
𝑙𝑛𝑦 = 𝑙𝑛𝑏𝑒 𝑚𝑥 = 𝑙𝑛𝑏 + 𝑙𝑛𝑒 𝑚𝑥 = 𝑙𝑛𝑏 + 𝑚𝑥

Let 𝑌 = 𝑙𝑛𝑦, 𝑋 = 𝑥, 𝐵 = 𝑙𝑛𝑏, 𝑀 = 𝑚 to convert the nonlinear function into the

linear equation 𝑌 = 𝑀𝑋 + 𝐵

Presented by Engr. Romulo C. Cruz, Jr. Electrical Engineering Department College of Engineering& Architecture
 EXAMPLE PROBLEM 3

SOLUTION…

i 𝒙𝒊 = 𝑿𝒊 𝒚𝒊 𝒀𝒊 = 𝒍𝒏𝒚𝒊 𝑿𝒊 𝒀𝒊 𝑿𝒊 𝟐
1 1 10 2.3026 2.3026 1
2 5 15 2.7081 13.5405 25
3 7 12 2.4849 17.3943 49
4 9 15 2.7081 24.3729 81
5 12 21 3.0445 36.5340 144
Σ 34 13.2482 94.1443 300

Presented by Engr. Romulo C. Cruz, Jr. Electrical Engineering Department College of Engineering& Architecture
 EXAMPLE PROBLEM 3

SOLUTION…
The auxiliary equations are
σ 𝑌 = 𝑀 σ 𝑋 + 𝐵𝑛
13.2482 = 34𝑀 + 5𝐵

σ 𝑋𝑌 = 𝑀 σ 𝑋 2 + 𝐵 σ 𝑋
94.1443 = 300𝑀 + 34𝐵

Presented by Engr. Romulo C. Cruz, Jr. Electrical Engineering Department College of Engineering& Architecture
 EXAMPLE PROBLEM 3

SOLUTION…
By Cramer’s Rule:

13.2482 5
94.1443 34 450.4388−470.7215
𝑀= 34 5 = = 0.0590
1156−1500
300 34

34 13.2482
300 94.1443 3200.9062−3974.4600
𝐵= = = 2.2487
−344 −344

Presented by Engr. Romulo C. Cruz, Jr. Electrical Engineering Department College of Engineering& Architecture
 EXAMPLE PROBLEM 3
SOLUTION…
Since
𝑙𝑛𝑏 = 𝐵
𝑏 = 𝑒 𝐵 = 𝑒 2.2487 = 9.4754,
and
𝑚 = 𝑀 = 0.0590

The required best fit curve is

𝒚 = 𝟗. 𝟒𝟕𝟓𝟒𝒆𝟎.𝟎𝟓𝟗𝟎𝒙

Presented by Engr. Romulo C. Cruz, Jr. Electrical Engineering Department College of Engineering& Architecture
THANK YOU FOR LISTENING!