Scribd
Upload
7 views

5 Noman Naseer Linear Regression

Uploaded by

057GV Văn Hiếu
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
7 views

5 Noman Naseer Linear Regression

Uploaded by

057GV Văn Hiếu
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
You are on page 1/ 15

Simple and Multiple Linear Regression

Noman Naseer
Linear regression

• Linear regression is a simple approach to supervised


learning. It assumes that the dependence of Y on X 1 ,
X 2 , . . . X p is linear.
Linear regression
• Linear regression is a simple approach to supervised
learning. It assumes that the dependence of Y on
X 1 , X 2 , . . . X p is linear.
• True regression functions are never linear!
7
6
f(X)

5
4
3

2 4 6 8

X
Linear regression
• Linear regression is a simple approach to supervised
learning. It assumes that the dependence of Y on
X 1 , X 2 , . . . X p is linear.
• True regression functions are never linear!
7
6
f(X)

5
4
3

2 4 6 8

• although it may seem overly simplistic, linear regression is


extremely useful both conceptually and practically.
Linear regression for the advertising data

Consider the advertising data shown on the next slide.


Questions we might ask:
• Is there a relationship between advertising budget and
sales?
• How strong is the relationship between advertising budget
and sales?
• Which media contribute to sales?
• How accurately can we predict future sales?
• Is the relationship linear?
• Is there synergy among the advertising media?
Advertising data
25

25

25
20

20

20
15

15

15
Sales

Sales

Sales
10

10

10
5

5
0 50 200 300 0 10 20 30 40 0 20 40 60 80 100
100 50
TV Radio Newspaper
Simple linear regression using a single predictor X .

• We assume a model

Y = β0 + β 1 X + ϵ,

where β0 and β1 are two unknown constants that represent


the intercept and slope, also known as coefficients or
parameters, and ϵ is the error term.
• Given some estimates βˆ0 and βˆ1 for the model coefficients,
we predict future sales using

yˆ = βˆ 0 + βˆ 1 x,

where yˆ indicates a prediction of Y on the basis of X =


x . The hat symbol denotes an estimated value.
Esti mati on of the parameters by least squares
• Let yˆi = βˆ0 + βˆ1xi be the prediction for Y based on the
ith value of X . Then e i = y i − yˆi represents the ith
residual
e21 +of esquares
R S S =sum
• We define the residual
2
2 + · ·( ·
RS
2
n S ) as
+ e ,
or equivalently as

R S S = (y1−βˆ0−βˆ1x1)2+(y2−βˆ0−βˆ1x2)2+. . .+(yn−βˆ0−βˆ1xn)2.
Example: advertising data

25
20
Sales

15
10
5

0 50 100 150 200 250 300

TV

The least squares fit for the regression of s a l e s onto TV.


In this case a linear fit captures the essence of the relationship,
although it is somewhat deficient in the left of the plot.
Multiple Linear Regression

• Here our model is

Y = β 0 + β 1 X 1 + β 2 X 2 + · · · + β p X p + ϵ,

• We interpret β j as the average effect on Y of a one unit increase in


X j , holding all other predictors fixed. In the advertising
example, the model becomes

s a l e s = β 0 + β 1 × TV + β 2 × radi o + β 3 × newspaper + ϵ.
Thank you

You might also like