Scribd
Upload
5 views

Regression

The document explains the principles of regression, focusing on the least-squared approximation and the formation of a least-squared linear regression line. It distinguishes between interpolation and extrapolation, and outlines methods for fitting a regression line, including inspection, semi-averages, and least squares. Additionally, it discusses the roles of dependent and independent variables in regression analysis.

Uploaded by

KIN WEI NG
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
5 views

Regression

The document explains the principles of regression, focusing on the least-squared approximation and the formation of a least-squared linear regression line. It distinguishes between interpolation and extrapolation, and outlines methods for fitting a regression line, including inspection, semi-averages, and least squares. Additionally, it discusses the roles of dependent and independent variables in regression analysis.

Uploaded by

KIN WEI NG
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 15

Regression

Topic Outcomes
• Explain the principle of least-squared approximation.

• Use the formula to form the equation of a least-squared linear regression


line (line of best fit).

• Distinguish between interpolation and extrapolation.


What is a linear function?
• There are two variables
• One is dependent (y)
• One is independent (x)

• If we have an expression of y in the form a+bx


• Then we have a linear function of x
• This can be represented graphically as a straight line

• a is the intercept on the y axis


• b is the slope / gradient of the line
What is regression?
• It is just a mathematical relationship between variables
• It expresses one variable in terms of another

• Examples of applications
• If the student to teacher ratio in schools changes, what effect will it have on
average marks?
• If the return on spending on advertising comparable with that of similar
firms?
Fitting a regression line

 This is the process of obtaining a regression relationship for a set of data

 It may be done by
 Inspection
 Semi-averages
 Least squares
Inspection
• This is a very basic approach
• Draw a scatter plot
• Draw a line of best fit
• This may be helped by plotting (x,ҧ yത)
• Find the equation of the line (y = a + bx)

• It is highly inaccurate
• 2 people will probably draw different lines
• The same person may not even draw the same line twice!
Semi-averages
This method splits the data in half
 Step 1
 Sort the data into size order by x-value
 Step 2
 Split the data into 2 equal parts such that one half contains lower values of x
and the other contains higher values of x
 Step 3
 Calculate the mean points for each group
 Step 4
 Plot the mean points and join with a straight line
 Step 5
 Find the equation of the line
Semi-average regression
This is an improvement on the inspection method where only one
mean was calculated
However, it is still not particularly rigorous in its method
Least squares
• Ordinary least squares is the standard method of regression
• The least squares regression line of y on x is the line that minimises
the sum of squares of vertical deviations from that line
• You do not need to know the derivation of the formula

• For the line y = a + bx…

𝒏 σ 𝒙𝒚−σ 𝒙 σ 𝒚 𝑺𝒙𝒚 σ𝒚 σ𝒙
𝒃=
𝒏 σ 𝒙𝟐 − σ 𝒙 𝟐
=
𝑺𝒙𝒙
and 𝒂 = 𝒏
−𝒃
𝒏
Least squares
• An alternative formula for y = a + bx…

𝑺𝒙𝒚
• 𝒃= and 𝒂 = 𝒚ഥ − 𝒃ഥ𝒙
𝑺𝒙𝒙

• A note of caution with this formula…


• Rounded values must NOT be used for the means of x and y
• The accurate values MUST be used
Dependent and independent values
• Bivariate data will use two variables
• Often, one variable’s value will depend on the other
• This is the dependent variable
• The independent variable occurs naturally
• For example
• Revenue depends on the quantity of a good sold
• Average cost depends on the number of items produced

• Usually the independent variable is x and the dependent


variable is y
• We then find the regression of y on x
Interpolation and extrapolation
• Methods of estimation using a regression line

• Interpolation
• Estimation within the range of values for the independent variable
• We can be quite confident of our interpolated estimates

• Extrapolation
• Estimation outside of the range of values for the independent variable
• Typically used for forecasting
• Lower level of confidence regarding these estimates
You should now be able to…
know how to draw linear graphs
be able to find equations from linear graphs
know the standard methods of finding a regression line
find the y-on-x regression line
calculate least squares regression
know the about non casual relationships
be able to determine and plot least squares line and mean point
be able to use interpolation and extrapolation

You might also like