Scribd
Upload
54 views

Statistics II - Asymptotic Theory For Least Squares: Marcelo Sant'Anna

1) The document presents the asymptotic theory for the linear regression model, showing that under regularity conditions, the OLS estimators are asymptotically normal and consistent. 2) It derives the asymptotic variance of the estimators and shows that plug-in estimators of the variance are consistent. 3) It extends the results to functions of the parameters and develops test statistics like t-statistics and Wald statistics that have asymptotic chi-squared distributions.

Uploaded by

OIB
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
54 views

Statistics II - Asymptotic Theory For Least Squares: Marcelo Sant'Anna

1) The document presents the asymptotic theory for the linear regression model, showing that under regularity conditions, the OLS estimators are asymptotically normal and consistent. 2) It derives the asymptotic variance of the estimators and shows that plug-in estimators of the variance are consistent. 3) It extends the results to functions of the parameters and develops test statistics like t-statistics and Wald statistics that have asymptotic chi-squared distributions.

Uploaded by

OIB
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 12

Statistics II - Asymptotic Theory for Least Squares

Marcelo Sant’Anna

FGV EPGE

July 31, 2019

Marcelo Sant’Anna (FGV EPGE) Statistics II - Lec 6 July 31, 2019 1 / 12


Introduction

The asymptotic theory developed here applies to the broader linear projection
model:
yi = xi0 β + ei ,
−1
where β = (E [xi xi0 ]) E [xi yi ].

Assumption (Regularity conditions)


1 The observations (yi , xi ), for i = 1, . . . , n are i.i.d.;
 
2 E yi4 < ∞;
 
3 E ||xi ||4 < ∞;
4 Qxx = E [xi xi0 ] is p.d.

Marcelo Sant’Anna (FGV EPGE) Statistics II - Lec 6 July 31, 2019 2 / 12


Asymptotic normality and variance

Theorem
Under the Assumption above,
√  
d
n β̂ − β −→ N(0, Vβ ),

where
−1 −1
Vβ = Qxx ΩQxx ,
Qxx = E [xi xi0 ] and E xi xi0 ei2 .
 

p
Vβ 6= Vβ̂ , from Chp 4. Indeed nVβ̂ −→ Vβ .

−1 2
In the homoskedastic case: Ω = Qxx σ 2 , so Vβ = Qxx σ .

Marcelo Sant’Anna (FGV EPGE) Statistics II - Lec 6 July 31, 2019 3 / 12


Joint distribution of coefficients

Example
In the homoskedastic errors case, when k = 2, and

σ12
 
ρσ1 σ2
E [xi xi0 ] =
ρσ1 σ2 σ22

if ρ > 0 (ρ < 0), then β̂1 and β̂2 are asymptotically negatively (positively)
correlated.

The conclusion relies heavily on the assumption of homoskedasticity. It is in


general hard to predict the asymptotic joint dependence of OLS estimates.

Marcelo Sant’Anna (FGV EPGE) Statistics II - Lec 6 July 31, 2019 4 / 12


Asymptotic variance estimation

Theorem (Consistency of error variance estimators)


Under the regularity conditions above,
p p
σ̂ 2 −→ σ 2 and s 2 −→ σ 2 .
A natural estimator for the asymptotic covariance in the homoskedastic case is the
plug-in estimator:
−1 2
V̂β0 = Q̂xx s ,
1
xi xi0 .
P
where Q̂xx = n

Theorem (Consistency of homoskedastic variance estimation)


Under the regularity conditions above,
p −1 2
V̂β0 −→ Qxx σ = Vβ0 .

Marcelo Sant’Anna (FGV EPGE) Statistics II - Lec 6 July 31, 2019 5 / 12


Asymptotic variance estimation

In a similar fashion, a natural estimator for the asymptotic variance in the case of
heteroskedasticity is the plug-in estimator:
−1 −1
V̂βHC 0 = Q̂xx Ω̂ Q̂xx ,

1
xi xi0 êi .
P
where Ω̂ = n i

Theorem (Consistency of heteroskedastic variance estimation)


Under the regularity conditions above,
p p
Ω̂ −→ Ω and V̂βHC 0 −→ Vβ .

Marcelo Sant’Anna (FGV EPGE) Statistics II - Lec 6 July 31, 2019 6 / 12


Functions of parameters

In many applications, the researcher is interested in a specific function of model


parameters θ = r (β). Let’s work with the plug-in estimator θ̂ = r (β̂).

Assumption (Parameter transformation)


∂ r (β) 0
r : Rk → Rq is continuously differentiable at the true value of β and R = ∂β
has rank q.

Theorem (Consistency - functions of parameters)


Under the regularity conditions and assumption above,
p
θ̂ −→ θ.

Marcelo Sant’Anna (FGV EPGE) Statistics II - Lec 6 July 31, 2019 7 / 12


Functions of parameters

Theorem (Asymptotic distribution - functions of parameters)


Under the regularity conditions and assumption above,
√  
d
n θ̂ − θ −→ N (0, Vθ ) ,

where Vθ = R 0 Vβ R.

Example (Asymptotic distribution of specific coefficients)


√  
d
 
n β̂j − βj −→ N 0, [Vβ ]jj

Marcelo Sant’Anna (FGV EPGE) Statistics II - Lec 6 July 31, 2019 8 / 12


Functions of parameters

Example (Inference on counterfactuals of interest)


Suppose a researcher is interested in doing inference about the Consumer Surplus
in an econometric model of demand with constant elasticity:

log(qm ) = β0 − β1 log(pm ) + εm ,

where β1 > 1. The Consumer Surplus evaluated at εm = 0 and prices p̄ is


 β1 −1 
p̄
CS(β; p̄) = e β0 .
β1 − 1
√  
d
Suppose the researcher has established that n β̂ − β −→ N(0, Vβ ), then
√  
d
n CS(β̂; p̄) − CS(β; p̄) −→ N(0, R 0 Vβ R),
0
where R = CS(β; p̄), exp(β0 )p̄ −β1 .


Marcelo Sant’Anna (FGV EPGE) Statistics II - Lec 6 July 31, 2019 9 / 12


t-statistic and confidence intervals

Let θ = r (β) ∈ R, consider the t-statistic:

θ̂ − θ
T (θ) = ,
s(θ̂)

where s(θ̂)2 = n1 V̂θ and V̂θ is a consistent estimator for the asymptotic variance of
θ.
Theorem
Under the regularity conditions and assumption above, provided Vθ > 0,

d N(0, Vθ )
T (θ) −→ √ ∼ N(0, 1).
Vθ

This allow us to construct t-tests with asymptotic significance levels.

Marcelo Sant’Anna (FGV EPGE) Statistics II - Lec 6 July 31, 2019 10 / 12


Confidence intervals

We want to construct confidence interval (CI) estimators for θ̂ with asymptotic


coverage probability 1 − α:
( )
θ̂ − θ
Ĉ = {θ : |T (θ)| ≤ c} = θ : −c ≤ ≤c ,
s(θ̂)
where c is chosen such that
 
Pr θ ∈ Ĉ = Pr (|T (θ)| ≤ c) → Pr (|Z | ≤ c) = 1 − α.

There are important differences between the asymptotic CI and the normal
regression model CI. In particular the normal regression CI we derived earlier:
only applies to β and not to functions θ;
is constructed under the assumption of homoskedastic errors and here we
allow for heteroskedasticity;
uses student-t distribution to compute c.

Marcelo Sant’Anna (FGV EPGE) Statistics II - Lec 6 July 31, 2019 11 / 12


Wald statistic

CI and t-statistic derived earlier applied to scalar θ, however what if we want to


evaluate joint restrictions or to create, not CI for specific parameters, but
confidence regions for groups of parameters?

Let θ = r (β) ∈ Rq . Consider the quadratic form:

 0    0  
W (θ) = θ̂ − θ V̂θ̂−1 θ̂ − θ = n θ̂ − θ V̂θ−1 θ̂ − θ

Theorem (Asymptotic distribution of Wald statistic)


Under the regularity conditions and assumption above,
d
W (θ) −→ χ2q .

Marcelo Sant’Anna (FGV EPGE) Statistics II - Lec 6 July 31, 2019 12 / 12

You might also like