Chapter 5

The Multiple Regression Model

Walter R. Paczkowski
Rutgers University
Principles of Econometrics, 4th Edition Chapter 5: The Multiple Regression Model Page 1
 Chapter Contents

 5.1 Introduction
 5.2 Estimating the Parameters of the Multiple
Regression Model
 5.3 Sampling Properties of the Least Squares
Estimators
 5.4 Interval Estimation
 5.5 Hypothesis Testing
 5.6 Polynomial Equations
 5.7 Interaction Variables
 5.8 Measuring Goodness-of-fit

Principles of Econometrics, 4th Edition Chapter 5: The Multiple Regression Model Page 2
 5.1
Introduction

Principles of Econometrics, 4th Edition Chapter 5: The Multiple Regression Model Page 3
 5.1
Introduction

Multiple Regression: A regression model

involving more than one explanatory variables.
Let’s set up an economic model in which sales
revenue depends on one or more explanatory
variables
– We initially hypothesize that sales revenue is
linearly related to price and advertising
expenditure
Eq. 5.1 – The economic model is:
SALES  1  2 PRICE  3 ADVERT

Principles of Econometrics, 4th Edition Chapter 5: The Multiple Regression Model Page 4
 5.1
Introduction

5.1.1
The Economic
Model

β2 is the change in monthly sales SALES ($1000)

when the price index PRICE is increased by one
unit ($1), and advertising expenditure ADVERT is
held constant
SALES
2 
PRICE  ADVERT held constant 
SALES

PRICE

Principles of Econometrics, 4th Edition Chapter 5: The Multiple Regression Model Page 5
 5.1
Introduction

5.1.1
The Economic
Model

Similarly, β3 is the change in monthly sales SALES

($1000) when the advertising expenditure is
increased by one unit ($1000), and the price index
PRICE is held constant
SALES
3 
ADVERT  PRICE held constant 
SALES

ADVERT

Principles of Econometrics, 4th Edition Chapter 5: The Multiple Regression Model Page 6
 5.1
Introduction

5.1.2
The Econometric
Model

The econometric model is:

E (SALES )  β1  β2 PRICE  β3 ADVERT

– To allow for a difference between observable

sales revenue and the expected value of sales
revenue, we add a random error term,
e = SALES - E(SALES)

Eq. 5.2 SALES  E  SALES   e  β1  β2 PRICE  β3 ADVERT  e

Principles of Econometrics, 4th Edition Chapter 5: The Multiple Regression Model Page 7
 5.1
Introduction FIGURE 5.1 The multiple regression plane

5.1.2
The Econometric
Model

Principles of Econometrics, 4th Edition Chapter 5: The Multiple Regression Model Page 8
 5.1 Table 5.1 Observations on Monthly Sales, Price, and Advertising in Big
Introduction
Andy’s Burger Barn

5.1.2
The Econometric
Model

Principles of Econometrics, 4th Edition Chapter 5: The Multiple Regression Model Page 9
 5.1
Introduction

5.1.2a
The General Model

In a general multiple regression model, a

dependent variable y is related to a number of
explanatory variables x2, x3, …, xK through a linear
equation that can be written as:
Eq. 5.3 y  β1  β2 x2  β3 x3   β K xK  e

Principles of Econometrics, 4th Edition Chapter 5: The Multiple Regression Model Page 10
 5.1
Introduction

5.1.2a
The General Model

A single parameter, call it βk, measures the effect

of a change in the variable xk upon the expected
value of y, all other variables held constant
E  y  E  y 
βk  
xk other xs held constant
xk

Principles of Econometrics, 4th Edition Chapter 5: The Multiple Regression Model Page 11
 5.1
Introduction

5.1.2a
The General Model

The parameter β1 is the intercept term.

– We can think of it as being attached to a
variable x1 that is always equal to 1
– That is, x1 = 1

Principles of Econometrics, 4th Edition Chapter 5: The Multiple Regression Model Page 12
 5.1
Introduction

5.1.2a
The General Model

The equation for sales revenue can be viewed as a

special case of Eq. 5.3 where K = 3, y = SALES,
x1 = 1, x2 = PRICE and x3 = ADVERT
Eq. 5.4 y  β1  β2 x2  β3 x3  e

Principles of Econometrics, 4th Edition Chapter 5: The Multiple Regression Model Page 13
 5.1
Introduction ASSUMPTIONS of the Multiple Regression Model

5.1.2b
The Assumptions
of the Model

MR1. yi  1  2 xi 2    K xiK  ei , i  1, , N
MR2. E ( yi )  1  2 xi 2    K xiK  E (ei )  0
MR3. var( yi )  var(ei )  2
MR4. cov( yi , y j )  cov(ei , e j )  0
MR5. The values of each xtk are not random and are not
exact linear functions of the other explanatory variables
MR6. yi ~ N (1  2 xi 2   K xiK ), 2   ei ~ N (0, 2 )

Principles of Econometrics, 4th Edition Chapter 5: The Multiple Regression Model Page 14
 Explaining assumption MR. 5

Principles of Econometrics, 4th Edition Chapter 5: The Multiple Regression Model Page 15
 5.2
Estimating the Parameters of the
Multiple Regression Model

Principles of Econometrics, 4th Edition Chapter 5: The Multiple Regression Model Page 16
y  ( X ' X )1 X ' y
5.2
Estimating the
 ( X ' X )1 Xof
y Parameters ' ythe
Multiple
Regression Model

Write the model for i = 1 to N observation

yi  1  2 xi 2    K xiK  ei , i  1, , N

Eq. 5.4
– Express the model in matrix notation and show
that (Will be done on white board)

Principles of Econometrics, 4th Edition Chapter 5: The Multiple Regression Model Page 17
Principles of Econometrics, 4th Edition Chapter 5: The Multiple Regression Model Page 18
 5.2
Estimating the
Parameters of the
Multiple
Regression Model

5.2.2
Least Squares
Estimates Using
Hamburger Chain
Data

Estimates along with their standard errors and the

equation’s R2 are typically reported in equation format
as:

SALES  118.91  7.908PRICE  1863 ADVERT R 2  0.448

Eq. 5.6 ( se) (6.35) (1.096) (0.683)

44.8% variation in sales is explained by price and advertisement

through this model.
Remaining 55.2% variation may be due to other variables affecting
sales e.g. sales force size, location of outlet, population of city where
outlet is located, income of residents of city where outlet is located and
so on.

Principles of Econometrics, 4th Edition Chapter 5: The Multiple Regression Model Page 19
 5.2
Estimating the
Parameters of the Table 5.2 Least Squares Estimates for Sales Equation for Big Andy’s
Multiple Burger Barn
Regression Model

5.2.2
Least Squares
Estimates Using
Hamburger Chain
Data

Principles of Econometrics, 4th Edition Chapter 5: The Multiple Regression Model Page 20
 5.2
Estimating the
Parameters of the
Multiple
Regression Model

5.2.2
Least Squares
Estimates Using
Interpretations of the results:
Hamburger Chain
Data 1. The negative coefficient on PRICE suggests
that demand is price elastic; we estimate that,
with advertising held constant, an increase in
price of $1 will lead to a fall in monthly
revenue of $7,908
2. The coefficient on advertising is positive; we
estimate that with price held constant, an
increase in advertising expenditure of $1,000
will lead to an increase in sales revenue of
$1,863

Principles of Econometrics, 4th Edition Chapter 5: The Multiple Regression Model Page 21
 5.2
Estimating the
Parameters of the
Multiple
Regression Model

5.2.2
Least Squares
Estimates Using
Interpretations of the results (Continued):
Hamburger Chain
Data 3. The estimated intercept implies that if both price
and advertising expenditure were zero the sales
revenue would be $118,914
• Clearly, this outcome is not possible; a zero
price implies zero sales revenue
• In this model, as in many others, it is important
to recognize that the model is an approximation
to reality in the region for which we have data
• Including an intercept improves this
approximation even when it is not directly
interpretable
Principles of Econometrics, 4th Edition Chapter 5: The Multiple Regression Model Page 22
 5.2
Estimating the
Parameters of the
Multiple
Regression Model

5.2.2
Least Squares
Estimates Using
Hamburger Chain
Data Using the model to predict sales if price is $5.50
and advertising expenditure is $1,200:

SALES  118.91 - 7.908PRICE  1.863 ADVERT

 118.914 - 7.9079  5.5  1.8626 1.2
 77.656

– The predicted value of sales revenue for

PRICE = 5.5 and ADVERT =1.2 is $77,656.

Principles of Econometrics, 4th Edition Chapter 5: The Multiple Regression Model Page 23
 Predicting y for simultaneous change in x
variables:

Principles of Econometrics, 4th Edition Chapter 5: The Multiple Regression Model Page 24
 Elasticity of y wrt x:
In many cases we are interested in percentage
change in y in response to one percent increase in
any X variable. This is called elasticity of y wrt x.
For example we talk about price elasticity of
demand and income elasticity of consumption
expenditure.

Principles of Econometrics, 4th Edition Chapter 5: The Multiple Regression Model Page 25
Principles of Econometrics, 4th Edition Chapter 5: The Multiple Regression Model Page 26
 5.2
Estimating the
Parameters of the
Multiple
Regression Model

5.2.2
Least Squares
A word of caution is in order about interpreting regression
Estimates Using
Hamburger Chain
results:
Data
– The negative sign attached to price implies that
reducing the price will increase sales revenue.
– If taken literally, why should we not keep reducing the
price to zero?
– Obviously that would not keep increasing total revenue
– This makes the following important point:
• Estimated regression models describe the
relationship between the economic variables for
values similar to those found in the sample data
• Extrapolating the results to extreme values is
generally not a good idea
• Predicting the value of the dependent variable for
values of the explanatory variables far from the
sample values invites disaster
Principles of Econometrics, 4th Edition Chapter 5: The Multiple Regression Model Page 27
 5.2
Estimating the
Parameters of the
Multiple
Regression Model

5.2.3
Estimation of the
Error Variance σ2

We need to estimate the error variance, σ2

– Recall that:
  var(ei )  E ei
2 2
 
– But, the squared errors are unobservable, so we
develop an estimator for σ2 based on the
squares of the least squares residuals:
eˆi  yi  yˆi  yi  b1  b2 xi 2  b3 xi 3 

Principles of Econometrics, 4th Edition Chapter 5: The Multiple Regression Model Page 28
 5.2
Estimating the
Parameters of the
Multiple
Regression Model

5.2.3
Estimation of the
Error Variance σ2

An estimator for σ2 that uses the information from

êi2 and has good statistical properties is:

 i 1 ei
N
ˆ 2
Eq. 5.7
 
ˆ 2

N K
where K is the number of β parameters being
estimated in the multiple regression model.

Principles of Econometrics, 4th Edition Chapter 5: The Multiple Regression Model Page 29
 5.2
Estimating the
Parameters of the
Multiple
Regression Model

5.2.3
Estimation of the
Error Variance σ2

For the hamburger chain example:

 i 1 ei
75
ˆ 2
1718.943
ˆ 
2
  23.874
N K 75  3

Principles of Econometrics, 4th Edition Chapter 5: The Multiple Regression Model Page 30
 5.2
Estimating the
Parameters of the
Multiple
Regression Model

5.2.3
Estimation of the
Error Variance σ2
Note that:
N
SSE   eˆi2  1718.943
i 1

– Also, note that

ˆ  23.874  4.8861
– Both quantities typically appear in the output
from your computer software
• Different software refer to it in different
ways.
Principles of Econometrics, 4th Edition Chapter 5: The Multiple Regression Model Page 31
 5.3
Sampling Properties of the Least
Squares Estimators

Principles of Econometrics, 4th Edition Chapter 5: The Multiple Regression Model Page 32
 5.3
Sampling
Properties of the
Least Squares THE GAUSS-MARKOV THEOREM
Estimators

For the multiple regression model, if assumptions

MR1–MR5 hold, then the least squares estimators
are the best linear unbiased estimators (BLUE) of the
parameters.

Principles of Econometrics, 4th Edition Chapter 5: The Multiple Regression Model Page 33
 5.3
Sampling
Properties of the
Least Squares
Estimators

If the errors are not normally distributed, then the

least squares estimators are approximately
normally distributed in large samples
– What constitutes ‘‘large’’ is tricky
– It depends on a number of factors specific to
each application
– Frequently, N – K = 50 will be large enough

Principles of Econometrics, 4th Edition Chapter 5: The Multiple Regression Model Page 34
 5.3
Sampling
Properties of the
Least Squares
Estimators

5.3.1
The Variances and
Covariances of the
We can show that: (how can you explain this)
Least Squares
Estimators

2
var(b2 )  N
(1  r232 ) ( xi 2  x2 ) 2
Eq. 5.8

i 1

where

Eq. 5.9 r23   ( xi 2  x2 )( xi 3  x3 )

 i 2 2  i3 3
( x  x ) 2
( x  x ) 2

Principles of Econometrics, 4th Edition Chapter 5: The Multiple Regression Model Page 35
 5.3
Sampling
Properties of the
Least Squares
Estimators

5.3.1
The Variances and
Covariances of the
Least Squares
Estimators We can see that:
1. Larger error variances 2 lead to larger
variances of the least squares estimators
2. Larger sample sizes N imply smaller
variances of the least squares estimators
3. More variation in an explanatory variable
around its mean, leads to a smaller variance of
the least squares estimator
4. A larger correlation between x2 and x3 leads to
a larger variance of b2
Principles of Econometrics, 4th Edition Chapter 5: The Multiple Regression Model Page 36
 5.3
Sampling
Properties of the
Least Squares
Estimators

5.3.1
The Variances and
Covariances of the
Least Squares
Estimators

We can arrange the variances and covariances in a

matrix format:
 var  b1  cov  b1 , b2  cov  b1 , b3  
 
cov  b1 , b2 , b3   cov  b1 , b2  var  b2  cov  b2 , b3  
 cov  b1 , b3  cov  b2 , b3  var  b3  

Principles of Econometrics, 4th Edition Chapter 5: The Multiple Regression Model Page 37
 5.3
Sampling
Properties of the
Least Squares
Estimators

5.3.1
The Variances and
Covariances of the
Least Squares
Estimators

Using the Andy’s hamburger data:

 40.343 6.795 0.7484

Eq. 5.10 cov  b1 , b2 , b3    6.795 1.201 0.0197 
 0.7484 0.0197 0.4668 

Principles of Econometrics, 4th Edition Chapter 5: The Multiple Regression Model Page 38
 5.3
Sampling
Properties of the
Least Squares
Estimators

5.3.1
The Variances and
Covariances of the
Least Squares
Estimators

Therefore, we have:

var  b1   40.343 cov  b1 , b2   6.795

var  b2   1.201 cov  b1 , b3   0.7484
var  b3   0.4668 cov  b2 , b3   0.0197

Principles of Econometrics, 4th Edition Chapter 5: The Multiple Regression Model Page 39
 5.3
Sampling
Properties of the
Least Squares
Estimators

5.3.1
The Variances and
Covariances of the
Least Squares
Estimators

We are particularly interested in the standard

errors:
se  b1   var  b1   40.343  6.3516

se  b2   var  b2   1.201  1.0960

se  b3   var  b3   0.4668  0.6832

Principles of Econometrics, 4th Edition Chapter 5: The Multiple Regression Model Page 40
 5.3
Sampling
Properties of the
Least Squares Table 5.3 Covariance Matrix for Coefficient Estimates
Estimators

5.3.1
The Variances and
Covariances of the
Least Squares
Estimators

Principles of Econometrics, 4th Edition Chapter 5: The Multiple Regression Model Page 41
 5.3
Sampling
Properties of the
Least Squares
Estimators

5.3.2
The Distribution of
the Least Squares
Estimators
Consider the general form of a multiple regression
model:
yi  1  2 xi 2  3 xi 3    K xiK  ei
– If we add assumption MR6, that the random
errors ei have normal probability distributions,
then the dependent variable yi is normally
distributed:

yi ~ N (1  2 xi 2   K xiK ), 2   ei ~ N (0, 2 )

Principles of Econometrics, 4th Edition Chapter 5: The Multiple Regression Model Page 42
 5.3
Sampling
Properties of the
Least Squares
Estimators

5.3.2
The Distribution of
the Least Squares
Estimators

Since the least squares estimators are linear

functions of dependent variables, it follows that
the least squares estimators are also normally
distributed:

 k , var  bk 
bk ~ N 

Principles of Econometrics, 4th Edition Chapter 5: The Multiple Regression Model Page 43
 5.3
Sampling
Properties of the
Least Squares
Estimators

5.3.2
The Distribution of
the Least Squares
Estimators

We can now form the standard normal variable Z:

bk  k
z ~ N  0,1 , for k  1, 2,, K
var  bk 
Eq. 5.11

Principles of Econometrics, 4th Edition Chapter 5: The Multiple Regression Model Page 44
 5.3
Sampling
Properties of the
Least Squares
Estimators

5.3.2
The Distribution of
the Least Squares
Estimators

Replacing the variance of bk with its estimate:

bk  k bk  k
Eq. 5.12 t  ~ t N K 
var  bk  se(bk )

– Notice that the number of degrees of freedom

for t-statistics is N - K

Principles of Econometrics, 4th Edition Chapter 5: The Multiple Regression Model Page 45
 5.3
Sampling
Properties of the
Least Squares
Estimators

5.3.2
The Distribution of What happens if the errors are not normally
the Least Squares
Estimators distributed?
– Then the least squares estimator will not be
normally distributed and Eq. 5.11, Eq. 5.12, and
Eq. 5.13 will not hold exactly
– They will, however, be approximately true in large
samples
– Thus, having errors that are not normally
distributed does not stop us from using Eq. 5.12
and Eq. 5.13, but it does mean we have to be
cautious if the sample size is not large
– A test for normally distributed errors was given in
Chapter 4.3.5
Principles of Econometrics, 4th Edition Chapter 5: The Multiple Regression Model Page 46
 5.5
Hypothesis Testing

Principles of Econometrics, 4th Edition Chapter 5: The Multiple Regression Model Page 47
 5.5
Hypothesis Testing COMPONENTS OF HYPOTHESIS TESTS

1. A null hypothesis H0
2. An alternative hypothesis H1
3. A test statistic
4. A rejection region
5. A conclusion

Principles of Econometrics, 4th Edition Chapter 5: The Multiple Regression Model Page 48
 5.5
Hypothesis Testing

5.5.1
Testing the
Significance of a
Single Coefficient

We need to ask whether the data provide any

evidence to suggest that y is related to each of the
explanatory variables
– If a given explanatory variable, say xk, has no
bearing on y, then βk = 0
– Testing this null hypothesis is sometimes called
a test of significance for the explanatory
variable xk

Principles of Econometrics, 4th Edition Chapter 5: The Multiple Regression Model Page 49
 5.5
Hypothesis Testing

5.5.1
Testing the
Significance of a
Single Coefficient

Null hypothesis:
H 0 : k  0

Alternative hypothesis:

H1 :  k  0

Principles of Econometrics, 4th Edition Chapter 5: The Multiple Regression Model Page 50
 5.5
Hypothesis Testing

5.5.1
Testing the
Significance of a
Single Coefficient

Test statistic:
bk
t ~ t( N  K )
se  bk 
t values for a test with level of significance α:

tc  t(1 /2, N  K ) and  tc  t(  /2, N  K )

Principles of Econometrics, 4th Edition Chapter 5: The Multiple Regression Model Page 51
 5.5
Hypothesis Testing

5.5.1
Testing the
Significance of a
Single Coefficient
For our hamburger example, we can conduct a test
that sales revenue is related to price:
1. The null and alternative hypotheses are:
H 0 : 2  0 and H1 : 2  0
2. The test statistic, if the null hypothesis is true, is:
t  b2 se  b2  ~ t( N  K )
3. Using a 5% significance level (α=.05), and 72 degrees
of freedom, the critical values that lead to a
probability of 0.025 in each tail of the distribution are:
t.975,72  1.993 and t.025,72   1.993

Principles of Econometrics, 4th Edition Chapter 5: The Multiple Regression Model Page 52
 5.5
Hypothesis Testing

5.5.1
Testing the For our hamburger example (Continued) :
Significance of a
Single Coefficient 4. The computed value of the t-statistic is:
7.908
t  7.215
1.096
and the p-value from software is:

   
P t 72  7.215  P t 72  7.215  2  (2.2 1010 )  0.000

5. Since -7:215 < -1.993, we reject H0: β2 = 0

and conclude that there is evidence from the
data to suggest sales revenue depends on price
• Using the p-value to perform the test, we
reject H0 because 0.000 < 0.05 .

Principles of Econometrics, 4th Edition Chapter 5: The Multiple Regression Model Page 53
 5.5
Hypothesis Testing

5.5.1
Testing the
Significance of a
Single Coefficient
Similarly, we can conduct a test that sales revenue is
related to advertising expenditure:
1. The null and alternative hypotheses are:
H 0 : 3  0 and H1 : 3  0
2. The test statistic, if the null hypothesis is true, is:
t  b3 se  b3  ~ t( N  K )
3. Using a 5% significance level (α=.05), and 72 degrees
of freedom, the critical values that lead to a
probability of 0.025 in each tail of the distribution are:
t.975,72  1.993 and t.025,72   1.993

Principles of Econometrics, 4th Edition Chapter 5: The Multiple Regression Model Page 54
 5.5
Hypothesis Testing

5.5.1
Testing the For our hamburger example (Continued) :
Significance of a
Single Coefficient 4. The computed value of the t-statistic is:
1.8626
t  2.726
0.6832
and the p-value from software is:

   
P t 72  2.726  P t 72  2.726  2  0.004  0.008
5. Since 2.726 > 1.993, we reject H0: β3 = 0: the
data support the conjecture that revenue is related
to advertising expenditure
• Using the p-value to perform the test, we
reject H0 because 0.008 < 0.05 .

Principles of Econometrics, 4th Edition Chapter 5: The Multiple Regression Model Page 55
 5.5
Hypothesis Testing

5.5.2a
Testing for Elastic
Demand

We now are in a position to state the following

questions as testable hypotheses and ask whether
the hypotheses are compatible with the data
1. Is demand price-elastic or price-inelastic?
2. Would additional sales revenue from
additional advertising expenditure cover the
costs of the advertising?

Principles of Econometrics, 4th Edition Chapter 5: The Multiple Regression Model Page 56
 5.5
Hypothesis Testing

5.5.2a
Testing for Elastic
Demand

For the demand elasticity, we wish to know if:

– β2 ≥ 0: a decrease in price leads to a decrease in
sales revenue (demand is price-inelastic or has
an elasticity of unity), or
– β2 < 0: a decrease in price leads to a decrease in
sales revenue (demand is price-inelastic)

Principles of Econometrics, 4th Edition Chapter 5: The Multiple Regression Model Page 57
 5.5
Hypothesis Testing

5.5.2a
Testing for Elastic
Demand
As before:
1. The null and alternative hypotheses are:
H 0 : 2  0 (demand is unit-elastic or inelastic)

H1 : 2 < 0 (demand is elastic)

2. The test statistic, if the null hypothesis is true,

is:
t  b2 se(b2 ) ~ t N  K 

3. At a 5% significance level, we reject H0 if

t ≤ -1.666 or if the p-value ≤ 0.05

Principles of Econometrics, 4th Edition Chapter 5: The Multiple Regression Model Page 58
 5.5
Hypothesis Testing

5.5.2a
Testing for Elastic
Demand
Hypothesis test (Continued) :
4. The test statistic is:
b2 7.908
t   7.215
se  b2  1.096

and the p-value is:


P t 72  7.215  0.000 
5. Since -7.215 < 1.666, we reject H0: β2 ≥ 0
and conclude that H0: β2 < 0 (demand is price
elastic)

Principles of Econometrics, 4th Edition Chapter 5: The Multiple Regression Model Page 59
 5.5
Hypothesis Testing

5.5.2b
Testing Advertising
Effectiveness

The other hypothesis of interest is whether an

increase in advertising expenditure will bring an
increase in sales revenue that is sufficient to cover
the increased cost of advertising
– Such an increase will be achieved if β3 > 1

Principles of Econometrics, 4th Edition Chapter 5: The Multiple Regression Model Page 60
 5.5
Hypothesis Testing

5.5.2b
Testing Advertising
Effectiveness
As before:
1. The null and alternative hypotheses are:
H 0 : 3  1

H 1 : 3 > 1

2. The test statistic, if the null hypothesis is true,

is: b3  1
t ~ t N  K 
se(b3 )
3. At a 5% significance level, we reject H0
if t ≥ 1.666 or if the p-value ≤ 0.05

Principles of Econometrics, 4th Edition Chapter 5: The Multiple Regression Model Page 61
 5.5
Hypothesis Testing

5.5.2b
Testing Advertising
Effectiveness

Hypothesis test (Continued) :

4. The test statistic is:
b3  β3 1.8626  1
t   1.263
se  b2  0.6832

and the p-value is:


P t 72  1.263  0.105 
5. Since 1.263 < 1.666, we do not reject H0

Principles of Econometrics, 4th Edition Chapter 5: The Multiple Regression Model Page 62
 5.3
Sampling
Properties of the
Least Squares
Estimators

5.3.2
The Distribution of
the Least Squares
Estimators
Testing Linear Combination of Parameters
We can form a linear combination of the
coefficients as:
  c1β1  c2β 2   cK β K   k 1 ck β k
K

And then we have

ˆ    ck bk   ck β k
t  ~ t N  K 
 se( ck bk )
Eq. 5.13
se ˆ

Principles of Econometrics, 4th Edition Chapter 5: The Multiple Regression Model Page 63
 5.3
Sampling
Properties of the
Least Squares
Estimators

5.3.2
The Distribution of
the Least Squares
Estimators
If = 3, the we have:

se  c1b1  c2 b2  c3 b K   var  c1b1  c2 b2  c3 b K 

where

var  c1b1  c2 b2  c3 b K   c12 var  b1   c22 var  b2   c32 var  b3 

Eq. 5.14

2c1c2 cov  b1 , b2   2c1c3 cov  b1 , b3   2c2 c3 cov  b2 , b3 

Principles of Econometrics, 4th Edition Chapter 5: The Multiple Regression Model Page 64
 The marketing adviser claims that dropping the
price by 20 cents will be more effective for
increasing sales revenue than increasing
advertising expenditure by $500
– In other words, she claims that -0.2β2 > 0.5β3,
or -0.2β2 - 0.5β3> 0
– We want to test a hypothesis about the linear
combination -0.2β2 – 0.5β3

Principles of Econometrics, 4th Edition Chapter 5: The Multiple Regression Model Page 65
 5.5
Hypothesis Testing

5.5.3
Hypothesis Testing
for a Linear
As before:
Combination of
Coefficients 1. The null and alternative hypotheses are:
H 0 : 0.2β 2  0.5β3  0 (marketer's claim is not correct)

H1 : 0.2β 2  0.5β3  0 (marketer's claim is correct)

2. The test statistic, if the null hypothesis is true,

is: 0.2b2  0.5b3
t ~ t 72
se(0.2b2  0.5b3 )
3. At a 5% significance level, we reject H0
if t ≥ 1.666 or if the p-value ≤ 0.05

Principles of Econometrics, 4th Edition Chapter 5: The Multiple Regression Model Page 66
 5.5
Hypothesis Testing

5.5.3
Hypothesis Testing
for a Linear
Combination of
Coefficients

We need the standard error:

se(0.2b2  0.5b3 )  var  se(0.2b2  0.5b3 ) 

  0.2  var  b2    0.5  var  b3   2   0.2    0.5  cov  b2, b3

2 2

 0.04  1.2012  0.25  0.4668  0.2   0.0197 

 0.4010

Principles of Econometrics, 4th Edition Chapter 5: The Multiple Regression Model Page 67
 5.5
Hypothesis Testing

5.5.3
Hypothesis Testing
for a Linear
Combination of
Coefficients Hypothesis test (Continued) :
4. The test statistic is:
0.2b2  0.5b3 1.58158  0.9319
t   1.622
se  0.2b2  0.5b3  0.4010

and the p-value is:


P t 72  1.622  0.055 
5. Since 1.622 < 1.666, we do not reject H0

Principles of Econometrics, 4th Edition Chapter 5: The Multiple Regression Model Page 68
Principles of Econometrics, 4th Edition Chapter 5: The Multiple Regression Model Page 69
 Test that there are increasing returns to scale at 5% level of significance

Principles of Econometrics, 4th Edition Chapter 5: The Multiple Regression Model Page 70
Principles of Econometrics, 4th Edition Chapter 5: The Multiple Regression Model Page 71
 5.6
Polynomial Equations

Principles of Econometrics, 4th Edition Chapter 5: The Multiple Regression Model Page 72
 4.3
Modeling Issues

What is the implication of linearity?

In the context of food expenditure model, an increase
in income by same amount bring equal increase in y.
That is the rate of change of y wrt x is constant
everywhere.
But the graphs p141 says it is not
Other examples of non linearities
Cost function marginal and average cost
Liquidity risk and required returns

There are a number of issues we must address when

building an econometric model
Principles of Econometrics, 4th Edition Chapter 5: The Multiple Regression Model Page 73
 5.6
Polynomial
Equations

We have studied the multiple regression model

Eq. 5.17 y  β1  β2 x2  β3 x3   β K xK  e

– Sometimes we are interested in polynomial

equations such as the quadratic
y = β1 + β2x + β3x2 + e
or the cubic
y = α1 + α2x + α3x2 + α4x3 + e.

Principles of Econometrics, 4th Edition Chapter 5: The Multiple Regression Model Page 74
 5.6
Polynomial
Equations

5.6.1
Hypothesis Testing
for a Linear
Combination of
Coefficients

Consider the average cost equation

Eq. 5.18 AC  β1  β 2Q  β3Q  e 2

– And the total cost function

Eq. 5.19 TC  1   2Q   3Q   4Q  e 2 3

Principles of Econometrics, 4th Edition Chapter 5: The Multiple Regression Model Page 75
 5.6
Polynomial FIGURE 5.2 (a) Total cost curve and (b) total product curve
Equations

5.6.1
Hypothesis Testing
for a Linear
Combination of
Coefficients

Principles of Econometrics, 4th Edition Chapter 5: The Multiple Regression Model Page 76
 5.6
Polynomial
Equations FIGURE 5.3 Average and marginal (a) cost curves and (b) product curves

5.6.1
Hypothesis Testing
for a Linear
Combination of
Coefficients

Principles of Econometrics, 4th Edition Chapter 5: The Multiple Regression Model Page 77
 5.6
Polynomial
Equations

5.6.1
Hypothesis Testing
for a Linear
For the general polynomial function:
Combination of
Coefficients

y  a0  a1x  a2 x2  a3 x3   ap x p

– The slope is
dy p 1
Eq. 5.20  a1  2a2 x  3a3 x 
2
 pa p x
dx
– Evaluated at a particular value, x = x0 , the slope
is:
dy
Eq. 5.21  a1  2a2 x0  3a3 x02   pa p x0p 1
dx x  x0

Principles of Econometrics, 4th Edition Chapter 5: The Multiple Regression Model Page 78
 5.6
Polynomial
Equations

5.6.1
Hypothesis Testing
for a Linear
Combination of
Coefficients

The slope of the total cost curve Eq. 5.18 is:

dE TC 
  2  23Q  3 4Q2
dQ
– For a U-shaped marginal cost curve, we expect
the parameter signs to be α2 > 0, α3 < 0, and
α4 > 0

Principles of Econometrics, 4th Edition Chapter 5: The Multiple Regression Model Page 79
 5.6
Polynomial
Equations

5.6.1
Hypothesis Testing
for a Linear
Combination of
Coefficients

The slope of the average cost curve Eq. 5.19 , is:

dE  AC 
 β2  2β3Q
dQ
– For this U-shaped curve, we expect β2 < 0 and
β3 > 0

Principles of Econometrics, 4th Edition Chapter 5: The Multiple Regression Model Page 80
 5.6
Polynomial
Equations

5.6.1
Hypothesis Testing
for a Linear
Combination of
Coefficients

It is sometimes true that having a variable and its

square or cube in the same model causes
collinearity problems (not exact)
– This will be discussed in Chapter 6

Principles of Econometrics, 4th Edition Chapter 5: The Multiple Regression Model Page 81
 5.6
Polynomial
Equations

5.6.2
Extending the
Model for Burger
Barn Sales

The linear sales model with the constant slope β3

for advertising

SALES  β1  β2 PRICE  β3 ADVERT  e

does not capture diminishing returns in advertising

expenditure

Principles of Econometrics, 4th Edition Chapter 5: The Multiple Regression Model Page 82
 5.6
Polynomial
Equations

5.6.2
Extending the
Model for Burger
Barn Sales

A new, better model might be:

Eq. 5.22 SALES  β1  β 2 PRICE  β3 ADVERT  β 4 ADVERT 2  e

The change in expected sales to a change in

advertising is:
E  SALES  E  SALES 

Eq. 5.23 ADVERT  PRICE held constant 
ADVERT
= β3  2β 4 ADVERT
Principles of Econometrics, 4th Edition Chapter 5: The Multiple Regression Model Page 83
 5.6
Polynomial FIGURE 5.4 A model where sales exhibits diminishing returns to
Equations advertising expenditure

5.6.2
Extending the
Model for Burger
Barn Sales

Principles of Econometrics, 4th Edition Chapter 5: The Multiple Regression Model Page 84
 5.6
Polynomial
Equations

5.6.2
Extending the
Model for Burger
Barn Sales

We refer to

E  SALES  ADVERT
as the marginal effect of advertising on sales

Principles of Econometrics, 4th Edition Chapter 5: The Multiple Regression Model Page 85
 5.6
Polynomial
Equations

5.6.2
Extending the
Model for Burger
Barn Sales

The least squares estimates are:

SALES  109.72  7.640 PRICE  12.151ADVERT  2.768 ADVERT 2
Eq. 5.24
 se   6.80  1.046   3.556   0.941

Principles of Econometrics, 4th Edition Chapter 5: The Multiple Regression Model Page 86
 5.6
Polynomial
Equations

5.6.2
Extending the
Model for Burger
Barn Sales

The estimated response of sales to advertising is:

 SALES
 12.151  5.536 ADVERT
ADVERT

Principles of Econometrics, 4th Edition Chapter 5: The Multiple Regression Model Page 87
 5.6
Polynomial
Equations

5.6.2
Extending the
Model for Burger
Substituting, we find that when advertising is at its
Barn Sales
minimum value in the sample of $500 (ADVERT =
0.5), the marginal effect of advertising on sales is
9.383
– When advertising is at a level of $2,000
(ADVERT = 2), the marginal effect is 1.079
Find marginal effect of advertising on sales when
adv equals 0.5 thousand dollar and when it equals
$3000
Determine the sales maximizing level of
advertising.
Principles of Econometrics, 4th Edition Chapter 5: The Multiple Regression Model Page 88
 5.7
Interaction Variables

Principles of Econometrics, 4th Edition Chapter 5: The Multiple Regression Model Page 89
 5.7
Interaction
Variables

We saw that in quadratic model the effect of

advertising on sales depends on current
advertisement level. What if effect of a variable
X2 on Y depends on some other variable X3? If so
the two variables X2 and X3 are said to interact.
Suppose that we wish to study the effect of income
and age on an individual’s expenditure on pizza
Eq. 5.27 – An initial model would be:

PIZZA  β1  β2 AGE  β3 INCOME  e

Principles of Econometrics, 4th Edition Chapter 5: The Multiple Regression Model Page 90
 5.7
Interaction
Variables

Implications of this model are:

1. E  PIZZA AGE  β2 : For a given level of
income, the expected expenditure on pizza
changes by the amount β2 with an additional
year of age
2. E  PIZZA INCOME  β3 : For individuals of
a given age, an increase in income of $1,000
increases expected expenditures on pizza by
β3

Principles of Econometrics, 4th Edition Chapter 5: The Multiple Regression Model Page 91
 5.7
Interaction
Variables
Table 5.4 Pizza Expenditure Data

Principles of Econometrics, 4th Edition Chapter 5: The Multiple Regression Model Page 92
 5.7
Interaction
Variables

The estimated model is:

PIZZA  342.88  7.576 AGE  1.832 INCOME
(t)  -3.27   3.95
– The signs of the estimated parameters are as we
anticipated
• Both AGE and INCOME have significant
coefficients, based on their t-statistics

Principles of Econometrics, 4th Edition Chapter 5: The Multiple Regression Model Page 93
 5.7
Interaction
Variables

It is not reasonable to expect that, regardless of the age of

the individual, an increase in income by $1,000 should lead
to an increase in pizza expenditure by $1.83?
– It would seem more reasonable to assume that as a
person grows older, his or her marginal propensity to
spend on pizza declines
• That is, as a person ages, less of each extra dollar is
expected to be spent on pizza
– This is a case in which the effect of income depends on
the age of the individual.
• That is, the effect of income on pizza depends on
age of the person. Thus there is an interaction of
income and age.
– One way of accounting for such interactions is to
include an interaction variable that is the product of
the two variables involved
Principles of Econometrics, 4th Edition Chapter 5: The Multiple Regression Model Page 94
 5.7
Interaction
Variables

We will add the interaction variable

(AGE x INCOME) to the regression model
– The new model is:
Eq. 5.28 PIZZA  β1  β2 AGE  β3 INCOME  β4  AGE  INCOME   e

Principles of Econometrics, 4th Edition Chapter 5: The Multiple Regression Model Page 95
 5.7
Interaction
Variables

Implications of this revised model are:

1. E  PIZZA AGE  β2  β4 INCOME
2. E  PIZZA INCOME  β3  β4 AGE

Principles of Econometrics, 4th Edition Chapter 5: The Multiple Regression Model Page 96
 5.7
Interaction
Variables

The estimated model is:

PIZZA  161.47  2.977 AGE  6.980 INCOME  0.1232  AGE  INCOME 
(t)  -0.89   2.47   1.85

Principles of Econometrics, 4th Edition Chapter 5: The Multiple Regression Model Page 97
 5.7
Interaction
Variables

The estimated marginal effect of income upon

pizza expenditure for two individuals—one with
of age 25 years and one of age 50 years is:

Principles of Econometrics, 4th Edition Chapter 5: The Multiple Regression Model Page 98
 5.7
Interaction
Variables

5.7.1
Log-Linear Models Consider a wage equation where ln(WAGE)
depends on years of education (EDUC) and years
of experience (EXPER):

Eq. 5.29 ln WAGE   β1  β2 EDUC  β3 EXPER  e

– If we believe the effect of an extra year of

experience on wages will depend on the level of
education, then we can add an interaction
variable
Eq. 5.30 ln WAGE   β1  β2 EDUC  β3 EXPER  β4  EDUC  EXPER   e

Principles of Econometrics, 4th Edition Chapter 5: The Multiple Regression Model Page 99
 5.7
Interaction
Variables

5.7.1
Log-Linear Models

The effect of another year of experience, holding

education constant, is roughly:

 ln WAGE 
 β3  β 4 EDUC
EXPER EDUC fixed

– The approximate percentage change in wage

given a one-year increase in experience is
100(β3+β4EDUC)%

Principles of Econometrics, 4th Edition Chapter 5: The Multiple Regression Model Page 100
 5.7
Interaction
Variables

5.7.1
Log-Linear Models

An estimated model is:

ln WAGE   1.392  0.09494 EDUC  0.00633EXPER

0.0000364  EDUC  EXPER 

Find the marginal effect of one year increase in

education on wage and interpret it.

Principles of Econometrics, 4th Edition Chapter 5: The Multiple Regression Model Page 101
 R-Sqaure:
Adjusted R-Square
Adjusted R-square in terms of unadjusted R-
Square (a model selection tool)
On slide 18, the R-Sq is reported as 0.448. This
means 44.8% variation in sales is explained by the
multiple regression with variables price and
advertisement.

Principles of Econometrics, 4th Edition Chapter 5: The Multiple Regression Model Page 102
 Typical values of R2 in research
In macroeconomics time series research where we
have trending variables which move with a common
trend e.g. aggregate income and aggregate
consumption. R2 are typically found in excess of 0.9
In Finance where due to market efficiency, it is
difficult to explain current returns based on past
history so R2 are expected to be very small say 0.05-
0.2
Survey based cross sectional studies fall in the middle
where we typically get R2 in the range 0.3-0.6

Principles of Econometrics, 4th Edition Chapter 5: The Multiple Regression Model Page 103
 6.3
Model Specification

6.3.4a
The Adjusted
Coefficient of
Determination Adjusted R-Square: Unfortunately R2 increase if #
explanatory variables are increased in the model even if the
additional variables are non-sense. Thus apparently the
explanatory power of the model increases but there is a
downside: the degrees of freedom are lost so the precision
of estimated coefficients is reduced. To counter this we
need to adjust sums of square by their degrees of freedom.
Eq. 6.25 The resulting measure is called Adjusted R-square. This
can serve as a model selection criteria denoted as R 2

SSE  N  K 
R  1
2

SST  N  1

Principles of Econometrics, 4th Edition Chapter 5: The Multiple Regression Model Page 104