Scribd
Upload
19 views27 pages

SAIDS - Linear Least Squares

Regression analysis is a predictive modeling technique that examines the relationship between dependent and independent variables, used for forecasting and causal effect analysis. Linear regression, including simple and multiple types, aims to model this relationship and minimize error through cost functions like Mean Squared Error (MSE). Correlation measures the relationship between two variables, which can be positive, negative, or zero, while different types of regression methods exist, such as linear and logistic regression.

Uploaded by

gawadearyan2
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
19 views27 pages

SAIDS - Linear Least Squares

Regression analysis is a predictive modeling technique that examines the relationship between dependent and independent variables, used for forecasting and causal effect analysis. Linear regression, including simple and multiple types, aims to model this relationship and minimize error through cost functions like Mean Squared Error (MSE). Correlation measures the relationship between two variables, which can be positive, negative, or zero, while different types of regression methods exist, such as linear and logistic regression.

Uploaded by

gawadearyan2
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 27

Module 6

Linear Least Squares


What Is Regression Analysis?
● Regression analysis is a form of predictive modelling technique which
investigates the relationship between a dependent (target) and independent
variable (s) (predictor).
● Here variable is a quantitative data such as height, weight etc.
● Dependent Variable: That has to be predicted
● Independent Variable: On basis of which the dependent variable has to be
predicted.
● This technique is used for forecasting, time series modelling and finding the
causal effect relationship between the variables.
● For example, predicting salary based on years of experience, predicting exam
score based on study hours,predicting resale price based on vehicle age.
What Is Regression Analysis?
Independent and Dependent Variable
Independent Variable
Variable that is not dependent on other variables are known as Independent

Variable. It can be manipulated.

Dependent Variable
Variable that depends on another variable (mostly independent variable) are known
as Dependent Variable. It is observed or measured.

Example, salary is the dependent variable and highest educational attainment,


weekly hours worked, and age are the independent variables.
Linear Regression analysis

• Linear Regression analysis is used to create a model that describes the

relationship between a dependent variable and one or more independent

variables.

• Depending on whether there are one or more independent variables, a

distinction is made between simple and multiple linear regression

analysis.
Types of Linear regression

Weight is a dependent variable


Simple Linear Regression
• The goal of a simple linear regression is to predict the value of a

dependent variable based on an independent variable.

• The greater the linear relationship between the independent variable and

the dependent variable, the more accurate is the prediction.


Simple Linear Regression
How the Model will Select the Best Fit
Line?
• For finding the best fit line our model

uses the cost function.

• In machine learning, every algorithm

has a cost function, and in simple

linear regression, the goal of our

algorithm is to find a minimal value

for the cost function.


What is the cost function?
• A cost function is a way of measuring the success of an ML

project.

• Mean error

• Mean squared error (MSE)

• Mean absolute error (MAE)

• Root mean squared error (RMSE)


Regression cost function
• Regression models are used to predict continuous variables,

Such as. Employee salaries, car costs, availability of loans, etc.

“regression cost function” is the cost function used in

regression problems. Depending on the distance-based error,

it is determined as follows:

error = y-y` Y – actual input and Y` – predicted output.

• Cost function is also known as the squared error function.


Regression cost function
• In linear regression (LR), we have many cost functions, but mostly used

cost function is MSE(Mean Squared Error). It is also known as a Least

Squared Method.

Loss Function
It is a calculation of loss for single
training data.

Cost Function
It is a calculation of average loss over
the entire dataset.
Regression cost function

Steps
• Our model will fit all possible
lines and find an overall
average error between the
actual and predicted values
for each line respectively.
• Selects the line which has the
lowest overall error. And that
will be the best fit line.
What is Correlation?

Correlation is a statistical metric that determines the relationship or


association between two variables.
Correlation can be a positive or negative value.
• Positive Correlation
Two variables are considered to be positively correlated when the value of
one variable increases or decreases following an increase or decrease in the
value of the other variable respectively.
• Negative Correlation
Two variables are considered to be negatively correlated when the value of
one variable increases following a decrease in the value of the other variable.
What is Correlation?

• Zero Correlation
This indicates that there is no relationship between two variables. It is also
known as a zero correlation. This is when a change in one variable doesn't
affect the other variable in any way.
What is Regression?

Regression indicates how an independent variable may be mathematically


connected to any dependent variable.
There are different types of regression and some of them have been listed
below:
• Linear Regression
• Logistic Regression
• Ridge Regression
• Laso Regression
Fit a Least square straight line for the following data:
x 2 7 9 1 5 12
y 13 21 23 14 15 21

Here n=6, ∑x=36, ∑y=107 ∑x2=304, ∑y2=2001 Now Regression Coefficient,


∑xy=721
b=Sxy/Sxx
Mean,x=36/6=6 mean y=107/6=17.83
=474/528
Sxx=n∑x2-(∑x)2
=6(304)-362 =0.8977
=528 Intercept a=Mean y -b (Mean x)
Syy=n∑y2-(∑y)2 =17.833- 0.8977(6)
=6(2001)-(107)2
=12.447
=557
L.S.S.I is y=a+bx
Sxy=n∑xiyi-(∑xi)(∑yi)
=6(721)-(36)(107) =12.45+0.8977x

=474
1. Find the least square straight line for the following data.
X 1 2 3 4 5 6
Y 6 4 3 5 4 2
N=6, ∑x=21, ∑y=24 ∑X2=91, ∑y2=106
∑xy=75
Let the LSSL of Y on X is
y=a0 + a1x
Its normal Equation are
∑y= N a0+a1∑x
∑xy=a0∑x + a1∑x2
24=6 a0 + 21a1—-1
75=21a0+91a1—-2
a0=

You might also like