.

..
. SLIDES . BY
.. John Loucks
.
.. St. Edward’s
.. University

© 2015 Cengage Learning. All Rights Reserved. May not be scanned, copied
Slide
1
or duplicated, or posted to a publicly accessible website, in whole or in part.
 Chapter 12, Part A
Simple Linear Regression

 Simple Linear Regression Model

 Least Squares Method
 Coefficient of Determination
 Model Assumptions
 Testing for Significance

© 2015 Cengage Learning. All Rights Reserved. May not be scanned, copied
Slide
2
or duplicated, or posted to a publicly accessible website, in whole or in part.
 Simple Linear Regression

 Managerial decisions often are based on the

relationship between two or more variables.
 Regression analysis can be used to develop an
equation showing how the variables are related.
 The variable being predicted is called the dependent
variable and is denoted by y.
 The variables being used to predict the value of the
dependent variable are called the independent
variables and are denoted by x.

© 2015 Cengage Learning. All Rights Reserved. May not be scanned, copied
Slide
3
or duplicated, or posted to a publicly accessible website, in whole or in part.
 Simple Linear Regression

 Simple linear regression involves one independent

variable and one dependent variable.
 The relationship between the two variables is
approximated by a straight line.
 Regression analysis involving two or more
independent variables is called multiple regression.

© 2015 Cengage Learning. All Rights Reserved. May not be scanned, copied
Slide
4
or duplicated, or posted to a publicly accessible website, in whole or in part.
 Simple Linear Regression Model

 The equation that describes how y is related to x and

an error term is called the regression model.
 The simple linear regression model is:

y = b0 + b1x +e

where:
b0 and b1 are called parameters of the model,
e is a random variable called the error term.

© 2015 Cengage Learning. All Rights Reserved. May not be scanned, copied
Slide
5
or duplicated, or posted to a publicly accessible website, in whole or in part.
 Simple Linear Regression Equation

 The simple linear regression equation is:

E(y) = 0 + 1x

• Graph of the regression equation is a straight line.

• b0 is the y intercept of the regression line.
• b1 is the slope of the regression line.
• E(y) is the expected value of y for a given x value.

© 2015 Cengage Learning. All Rights Reserved. May not be scanned, copied
Slide
6
or duplicated, or posted to a publicly accessible website, in whole or in part.
 Simple Linear Regression Equation

 Positive Linear Relationship

E(y)

Regression line

Intercept Slope b1
b0
is positive

© 2015 Cengage Learning. All Rights Reserved. May not be scanned, copied
Slide
7
or duplicated, or posted to a publicly accessible website, in whole or in part.
 Simple Linear Regression Equation

 Negative Linear Relationship

E(y)

Intercept
b0Regression line

Slope b1
is negative

© 2015 Cengage Learning. All Rights Reserved. May not be scanned, copied
Slide
8
or duplicated, or posted to a publicly accessible website, in whole or in part.
 Simple Linear Regression Equation

 No Relationship

E(y)

Intercept Regression line

b0
Slope b1
is 0

© 2015 Cengage Learning. All Rights Reserved. May not be scanned, copied
Slide
9
or duplicated, or posted to a publicly accessible website, in whole or in part.
 Estimated Simple Linear Regression
Equation
 The estimated simple linear regression
equation
ŷ b0  b1x

• The graph is called the estimated regression line.

• b0 is the y intercept of the line.
• b1 is the slope of the line.
• isŷthe estimated value of y for a given x value.

© 2015 Cengage Learning. All Rights Reserved. May not be scanned, copied
Slide
10
or duplicated, or posted to a publicly accessible website, in whole or in part.
 Estimation Process

Regression Model Sample Data:

y = b0 + b1x +e x y
Regression Equation x1 y1
E(y) = b0 + b1x . .
Unknown Parameters . .
b0, b1 xn yn

Estimated
b0 and b1 Regression Equation
provide estimates of ŷ b0  b1x
b0 and b1 Sample Statistics
b0, b1

© 2015 Cengage Learning. All Rights Reserved. May not be scanned, copied
Slide
11
or duplicated, or posted to a publicly accessible website, in whole or in part.
 Least Squares Example

You’re a marketing analyst for Hasbro

Toys. You gather the following data:
Ad Expenditure Sales
increase (Units)
1 1
2 1
3 2
4 2
5 4
Find the least squares line relating
sales and advertising.
© 2015 Cengage Learning. All Rights Reserved. May not be scanned, copied
Slide
12
or duplicated, or posted to a publicly accessible website, in whole or in part.
 Scattergram
Sales vs. Advertising

Sales
4
3
2
1
0
0 1 2 3 4 5
Advertising
© 2015 Cengage Learning. All Rights Reserved. May not be scanned, copied
Slide
13
or duplicated, or posted to a publicly accessible website, in whole or in part.
n Education, Inc

Regression Line Fitted

to the Data

Sales
4
3 yˆ  .1  .7 x
2
1
0
0 1 2 3 4 5
Advertising

© 2015 Cengage Learning. All Rights Reserved. May not be scanned, copied
Slide
14
or duplicated, or posted to a publicly accessible website, in whole or in part.
n Education, Inc

Scattergram

1. Plot of all (xi, yi) pairs

2. Suggests how well model will fit

y
60
40
20
0 x
0 20 40 60

© 2015 Cengage Learning. All Rights Reserved. May not be scanned, copied
Slide
15
or duplicated, or posted to a publicly accessible website, in whole or in part.
 Thinking Challenge

• How would you draw a line through the points?

• How do you determine which line ‘fits best’?

y
60
40
20
0 x
0 20 40 60

© 2015 Cengage Learning. All Rights Reserved. May not be scanned, copied
Slide
16
or duplicated, or posted to a publicly accessible website, in whole or in part.
 Least Squares Method

 Least Squares Criterion

min  (yi  y i )2

where:
yi = observed value of the dependent variable
for the ith observation
yi =^estimated value of the dependent variable
for the ith observation

© 2015 Cengage Learning. All Rights Reserved. May not be scanned, copied
Slide
17
or duplicated, or posted to a publicly accessible website, in whole or in part.
n Education, Inc

Least Squares Graphically

n
LS minimizes   i   1   2   3   4
ˆ 2
ˆ 2
ˆ 2
ˆ 2
ˆ 2

i 1

y y 2  ˆ0  ˆ1 x 2  ˆ2

e^4
e^2
e^1 e^3
yˆ i  ˆ0  ˆ1 xi
© 2015 Cengage Learning. All Rights Reserved. May not be scanned, copied
x
Slide
18
or duplicated, or posted to a publicly accessible website, in whole or in part.
 Least Squares Method

 Slope for the Estimated Regression Equation

b   (x  x)(y  y)
i i

 (x  x)
1 2
i

where:
xi = value of independent variable for ith
observation
yi = value of dependent variable for ith
_ observation
x = mean value for independent variable
_
y = mean value for dependent variable
© 2015 Cengage Learning. All Rights Reserved. May not be scanned, copied
Slide
19
or duplicated, or posted to a publicly accessible website, in whole or in part.
 Least Squares Method

 y-Intercept for the Estimated Regression

Equation
b0 y  b1x

© 2015 Cengage Learning. All Rights Reserved. May not be scanned, copied
Slide
20
or duplicated, or posted to a publicly accessible website, in whole or in part.
 Simple Linear Regression

 Example: Reed Auto Sales

Reed Auto periodically has a special week-long
sale.
As part of the advertising campaign Reed runs
one or
more television commercials during the weekend
preceding the sale. Data from a sample of 5
previous
sales are shown on the next slide.

© 2015 Cengage Learning. All Rights Reserved. May not be scanned, copied
Slide
21
or duplicated, or posted to a publicly accessible website, in whole or in part.
 Simple Linear Regression

 Example: Reed Auto Sales

Number of Number of
TV Ads (x) Cars Sold (y)
1 14
3 24
2 18
1 17
3 27
Sx = 10 Sy = 100
x 2 y 20

© 2015 Cengage Learning. All Rights Reserved. May not be scanned, copied
Slide
22
or duplicated, or posted to a publicly accessible website, in whole or in part.
 Estimated Regression Equation

 Slope for the Estimated Regression

Equation  (xi  x)(yi  y) 20
b1   5
 (xi  x) 2
4
 y-Intercept for the Estimated Regression
Equation b y  b x 20  5(2) 10
0 1

 Estimated Regression Equation

yˆ 10  5x

© 2015 Cengage Learning. All Rights Reserved. May not be scanned, copied
Slide
23
or duplicated, or posted to a publicly accessible website, in whole or in part.
 How well does the estimated
regression equation fit the
data?

© 2015 Cengage Learning. All Rights Reserved. May not be scanned, copied
Slide
24
or duplicated, or posted to a publicly accessible website, in whole or in part.
 Coefficient of Determination
(the goodness of fit)
 Relationship Among SST, SSR, SSE

SST = SSR +
SSE

 i
( y  y )2
 i
( ˆ
y  y )2
  i i
( y  ˆ
y )2

where:
SST = total sum of squares
SSR = sum of squares due to regression
SSE = sum of squares due to error

© 2015 Cengage Learning. All Rights Reserved. May not be scanned, copied
Slide
25
or duplicated, or posted to a publicly accessible website, in whole or in part.
 Coefficient of Determination

 The coefficient of determination is:

r2 = SSR/SST

where:
SSR = sum of squares due to regression
SST = total sum of squares

© 2015 Cengage Learning. All Rights Reserved. May not be scanned, copied
Slide
26
or duplicated, or posted to a publicly accessible website, in whole or in part.
 Coefficient of Determination

r2 = SSR/SST = 100/114 = .8772

The regression relationship is very strong; 87.72%

of the variability in the number of cars sold can be
explained by the linear relationship between the
number of TV ads and the number of cars sold.

© 2015 Cengage Learning. All Rights Reserved. May not be scanned, copied
Slide
27
or duplicated, or posted to a publicly accessible website, in whole or in part.
 Sample Correlation Coefficient

rxy (sign of b1 ) Coefficient of Determination

rxy (sign of b1 ) r 2

where:
b1 = the slope of the estimated regression
equation yˆ b0  b1 x

© 2015 Cengage Learning. All Rights Reserved. May not be scanned, copied
Slide
28
or duplicated, or posted to a publicly accessible website, in whole or in part.
 Sample Correlation Coefficient

rxy (sign of b1 ) r 2

yˆ 10  5 x is “+”.
The sign of b1 in the equation

rxy =+ .8772

rxy = +.9366

© 2015 Cengage Learning. All Rights Reserved. May not be scanned, copied
Slide
29
or duplicated, or posted to a publicly accessible website, in whole or in part.
 Assumptions About the Error Term e

1. The error  is a random variable with mean of zero

2. The variance of  , denoted by  2, is the same for

all values of the independent variable.

3. The values of  are independent.

4. The error  is a normally distributed random

variable.

© 2015 Cengage Learning. All Rights Reserved. May not be scanned, copied
Slide
30
or duplicated, or posted to a publicly accessible website, in whole or in part.
 1. The error  is a random variable with mean of zer
n
LS minimizes   i   1   2   3   4
ˆ 2
ˆ 2
ˆ 2
ˆ 2
ˆ 2

i 1

y y 2  ˆ0  ˆ1 x 2  ˆ2

e^4
e^2
e^1 e^3
yˆ i  ˆ0  ˆ1 xi
© 2015 Cengage Learning. All Rights Reserved. May not be scanned, copied
x
Slide
31
or duplicated, or posted to a publicly accessible website, in whole or in part.
 4. The error  is a normally distributed random
variable.

© 2015 Cengage Learning. All Rights Reserved. May not be scanned, copied
Slide
32
or duplicated, or posted to a publicly accessible website, in whole or in part.
 Testing for Significance

To test for a significant regression relationship, we

must conduct a hypothesis test to determine whether
the value of b1 is zero.

Two tests are commonly used:

t Test and F Test

Both the t test and F test require an estimate of s 2,

the variance of e in the regression model.

© 2015 Cengage Learning. All Rights Reserved. May not be scanned, copied
Slide
33
or duplicated, or posted to a publicly accessible website, in whole or in part.
 Testing for Significance

 An Estimate of s 2

The mean square error (MSE) provides the estimate

of s 2, and the notation s2 is also used.

s 2 = MSE = SSE/(n - 2)

where:

SSE  ( yi  yˆ i ) 2  ( yi  b0  b1 xi ) 2

© 2015 Cengage Learning. All Rights Reserved. May not be scanned, copied
Slide
34
or duplicated, or posted to a publicly accessible website, in whole or in part.
 Testing for Significance

 An Estimate of s
• To estimate s we take the square root of s 2.
• The resulting s is called the standard error of
the estimate.

SSE
s  MSE 
n 2

© 2015 Cengage Learning. All Rights Reserved. May not be scanned, copied
Slide
35
or duplicated, or posted to a publicly accessible website, in whole or in part.
 Testing for Significance: t Test

 Hypotheses

H 0:  1 0
H a:  1 0

 Test Statistic

b1 s
t where sb1 
sb1 (xi  x) 2

© 2015 Cengage Learning. All Rights Reserved. May not be scanned, copied
Slide
36
or duplicated, or posted to a publicly accessible website, in whole or in part.
 Testing for Significance: t Test

 Rejection Rule

Reject H0 if p-value < a

or t < -tor t > t

where:
t is based on a t distribution
with n - 2 degrees of freedom

© 2015 Cengage Learning. All Rights Reserved. May not be scanned, copied
Slide
37
or duplicated, or posted to a publicly accessible website, in whole or in part.
 Testing for Significance: t Test

1. Determine the hypotheses. H 0:  1 0

H a:  1 0
a = .05
2. Specify the level of significance.

b1
3. Select the test statistic.t 
sb1

4. State the rejection rule.

Reject H0 if p-value < .05
or |t| > 3.182 (with
3 degrees of freedom)

© 2015 Cengage Learning. All Rights Reserved. May not be scanned, copied
Slide
38
or duplicated, or posted to a publicly accessible website, in whole or in part.
 Testing for Significance: t Test

5. Compute the value of the test statistic.

b1 5
t  4.63
sb1 1.08

6. Determine whether to reject H0.

t = 4.541 provides an area of .01 in the
upper tail. Hence, the p-value is .02.
(Also, t = 4.63 > 3.182.) We can reject H0.

© 2015 Cengage Learning. All Rights Reserved. May not be scanned, copied
Slide
39
or duplicated, or posted to a publicly accessible website, in whole or in part.
 Confidence Interval for 1

 We can use a 95% confidence interval for 1 to test

the hypotheses just used in the t test.
 H0 is rejected if the hypothesized value of 1 is not
included in the confidence interval for 1.

© 2015 Cengage Learning. All Rights Reserved. May not be scanned, copied
Slide
40
or duplicated, or posted to a publicly accessible website, in whole or in part.
 Confidence Interval for 1

 The form of a confidence interval for 1 is: t / 2sb1

is the
b1  t / 2sb1 margin
b1 is the of error
point
where t / 2 is the t value providing an area
estimat
of a/2 in the upper tail of a t distribution
or
with n - 2 degrees of freedom

© 2015 Cengage Learning. All Rights Reserved. May not be scanned, copied
Slide
41
or duplicated, or posted to a publicly accessible website, in whole or in part.
 Confidence Interval for 1

 Rejection Rule
Reject H0 if 0 is not included in
the confidence interval for 1.
 95% Confidence Interval for 1
b1 t / 2=sb15 +/- 3.182(1.08) = 5 +/- 3.44
or 1.56 to 8.44
 Conclusion
0 is not included in the confidence interval.
Reject H0

© 2015 Cengage Learning. All Rights Reserved. May not be scanned, copied
Slide
42
or duplicated, or posted to a publicly accessible website, in whole or in part.
 Testing for Significance: F Test

 Hypotheses

H 0:  1 0
H a:  1 0
 Test Statistic
F = MSR/MSE

© 2015 Cengage Learning. All Rights Reserved. May not be scanned, copied
Slide
43
or duplicated, or posted to a publicly accessible website, in whole or in part.
 Testing for Significance: F Test

 Rejection Rule

Reject H0 if
p-value < a
or F > F
where:
F is based on an F distribution with
1 degree of freedom in the numerator and
n - 2 degrees of freedom in the denominator

© 2015 Cengage Learning. All Rights Reserved. May not be scanned, copied
Slide
44
or duplicated, or posted to a publicly accessible website, in whole or in part.
 Testing for Significance: F Test

1. Determine the hypotheses. H 0:  1 0

H a:  1 0

a = .05
2. Specify the level of significance.

3. Select the test statistic.F = MSR/MSE

4. State the rejection rule.

Reject H0 if p-value < .05
or F > 10.13 (with 1 d.f.
in numerator and
3 d.f. in denominator)

© 2015 Cengage Learning. All Rights Reserved. May not be scanned, copied
Slide
45
or duplicated, or posted to a publicly accessible website, in whole or in part.
 Testing for Significance: F Test

5. Compute the value of the test statistic.

F = MSR/MSE = 100/4.667 = 21.43

6. Determine whether to reject H0.

F = 17.44 provides an area of .025 in
the upper tail. Thus, the p-value
corresponding to F = 21.43
is less
Thethan .025. evidence
statistical Hence, we is reject H0. to
sufficient
conclude
that we have a significant relationship
between the
number of TV ads aired and the number of
© 2015
cars sold.
Cengage Learning. All Rights Reserved. May not be scanned, copied
Slide
46
or duplicated, or posted to a publicly accessible website, in whole or in part.
 Some Cautions about the
Interpretation of Significance Tests
 Rejecting H0: b1 = 0 and concluding that
the
relationship between x and y is significant
does not enable us to conclude that a cause-
and-effect
 Just because we are able to reject H0: b1 = 0 and
relationship is present between x and y.
demonstrate statistical significance does not enable
us to conclude that there is a linear relationship
between x and y.

© 2015 Cengage Learning. All Rights Reserved. May not be scanned, copied
Slide
47
or duplicated, or posted to a publicly accessible website, in whole or in part.
 End of Chapter 12, Part A

© 2015 Cengage Learning. All Rights Reserved. May not be scanned, copied
Slide
48
or duplicated, or posted to a publicly accessible website, in whole or in part.