Some Basic Ideas

Lecture 6: Classical Normal OLS estimators are random variables

Linear Regression Model
^
Recall: β 2 β 2 kiui

Econ 131 AY 2018 - 2019 ^

The probability distribution of the β' s
September 07-12, 2018
depend on the assumptions made on the
probability distribution of ui

The Classical Normal Linear The Normal Distribution

Regression Model
A random variable X is said to be normal if its pdf
Assumes that ui is normally distributed with
is of the form
Mean: E(ui|Xi)=0
Variance: E(ui2|Xi)=σ2 1 1 x μ
2

Cov (ui,uj): E(uiuj|Xi,Xj)=0 f (x) exp ,x R

σ 2π 2 σ
ui~N(0, σ2)
X has a normal distribution with expected value
or and variance 2

ui~NID(0, σ2)
“normally and independently distributed”

The Normal Distribution for error term

Relevant Theorems
Pdf of error term ui : “If two random variables are statistically independent, the
covariance between the two is zero. But the converse is not
necessarily true; that is, zero covariance does not imply
2 statistical independence. However, if two variables are
1 1 ui
f (ui) exp , ui R normally distributed, zero covariance necessarily implies
2 2 statistical independence.”
ui has a normal distribution with expected value
(?) and variance 2
 Some Implications of the
What is the distribution of Yi? Normality Assumption
^

Yi = β1+β2Xi + ui OLS estimators ( β' s ) are normally distributed

Error term ui is a random variable! More relevant representation:
ˆ xiYi
2 2
kiYi
xi

If ui is normally distributed, Yi is normally

distributed (why?) ^
Since Yi is normally distributed, β 2 is normally
distributed (why?)

^
^
Properties of β 2 Under Normality Deriving mean and variance of
β1
^
E 2 2

^ σ2
var β 2
x i2

^
2
β 2 ~ N(β 2 , σ ^ )
β2

^
β2 β2
Z Z ~ N(0,1)
σ^
β2

^ Properties of the OLS

Properties of β1 Under Normality Estimators Under Normality
^
E β1 β1 Unbiased

^ X i2 2 Minimum variance (in the entire class

var 1
n xi2 of unbiased estimators, whether linear
or not)
^
2
β1 ~ N(β1 , σ ^ )
β1

Consistent
^
β1 β1
Z Z ~ N(0,1)
σ^
β1
 Hypothesis Testing (1)
β is unknown but we can make a hypothesis about it

Is the observation/finding consistent/compatible with

a hypothesis?

Insert Dougherty power point Null hypothesis (H0) to be tested against an

alternative hypothesis (H1)
C02G05_2016_04_19
Consider testing a hypothesis about a single
parameter, say β2

To test whether a variable has a partial effect or not

Example 1 Example 2
Yi = β1 + β2Xi + ui Yi = β1+β2Xi+ui
where Y is son’s height and X is father’s height
where Y = consumption
H0: β2=0 X = income
father’s height does not affect son’s height β2= MPC
H1: β2≠0
father’s height affects son’s height
H0: β2 =MPC=0.9
H1: β2 = MPC 0.9
rejection of H0 : β2=0
=> “statistically significant” finding

Recall:
Hypothesis Testing (2) Interpretation of Confidence Intervals

Confidence interval approach ^ ^

Pr β 2 δ β2 β2 δ 1 α

Random vs. fixed interval

Test of significance approach
Repeated sampling context
 Confidence Interval Approach (1) Confidence Interval Approach (2)
Recall: by normality Pr tα t tα 1 α
2 2
^
2
^ β2 β2 xi
β2 β2
Z ^
σ^ σ
β2 β2 β2
Pr tα ^
tα 1 α
2 2
Since i2 is unknown, we use the t se(β 2 )
distribution to construct a confidence
interval ^
2
^
Pr β 2 t α se(β 2 ) β 2
^ ^ ^
β 2 t α se(β 2 ) 1 α
^ β2 β2 xi 2 2
β2 β2
t ^ ^
σ β^ 2 σ

Confidence interval approach (3) Test of significance approach

Estimate β2 Alternatively,
Construct a confidence interval for β2:
^
^ ^ ^ ^ *
Pr β 2 se(β 2 ) t α / 2 β2 β 2 se(β 2 ) t α / 2 1 α (i) Compute T 2 2
^
se 2

This t distribution has d.f. = n-k

(ii) If computed t falls outside the interval
If β2 under H0 lies within this interval, do not t / 2 , t / 2 reject the null hypothesis.
reject H0. If it falls outside the interval, reject
H0.

Example 3 Some practical considerations

GDI GDP

GDP and GDI example year

2002
growth
rate
-3.48
growth
rate
4.43
2001
2000
1999
2.08
23.93
-1.98
2.96
5.97
3.4
Interpretation:
1998 -16.28 -0.58

Test the hypothesis that β2=0 1997

1996
1995
11.7
13.26
2.77
5.19
5.85
4.68
what it means not to reject a null
hypothesis accepting H0
1994 8.65 4.39
1993 7.87 2.12
1992 7.83 0.34

(i) State H0 and H1

1991 -17.29 -0.58

(ii) Determine the probability distribution of the test statistic

(iii) Choose α = 5%
The t=2 rule of thumb in testing the
(iv) Confidence interval
null hypothesis that the slope is
(v) Decision
zero

Choosing α
Type I and II errors (1) In addition,

True State of the World

Ho H1 ^ ^ ^2

Decision
Ho Correct
non-
Type II
error
β1 , β 2 are distributed independently of σ
rejection
H1 Type I error Correct
(α) Rejection

^ 2

In addition, Testing the significance of σ

^
( n 2) σ 2 has a chi-square or 2 ^ 2

σ 2 Properties of σ under Normality

distribution ^2
(n 2) σ 2
~ χn 2
σ2

A confidence interval for σ The chi-square distribution

In general,

^2
2 (n k) 2
Pr 1 / 2 ,n k 2 / 2 ,n k 1

When k=2:

^2 ^2
σ σ
Pr (n 2) σ2 (n 2) 1 α
χ α2 / 2,n 2 χ 12 α / 2, n 2
Chi-square test of signficance Analysis of Variance

Compute ^2
2 (n k ) σ
χ
σ2
Compare with critical chi-square values
If the computed chi-square statistic (lies
outside the limits, reject the null hypothesis.
Example 4. GDP and GDI

Test the null hypothesis that σ2=10.

Digression

Analysis of Variance Example of ANOVA

 Reporting the Results

Insert Dougherty power point

C02G08_2016_04_19

Prediction Mean Prediction

E(Y0|X=X0)
Two kinds: ^
Y 0 is an unbiased estimator of E(Y|X=X0):
^
Prediction of the conditional mean value of Y E Y0 β1 β 2X0
for a given X, say X0 (i.e., a point on the PRF) ^
~”mean prediction” Confidence interval for Y 0
^ ^ ^ ^
Pr Y 0 se Y 0 t α / 2 β1 β 2 X 0 Y 0 se Y 0 t α / 2 95 %
Prediction of an individual Y value
corresponding to X0 _
2
X0 X
~”individual prediction” ^ 1
var Y0 σ2
n x i2

Example of PRF from Gujarati Individual Prediction

Y|X=X0
^
Y0 is also an unbiased predictor of Y|X=X0

Confidence interval
^ ^ ^ ^
Pr Y 0 se Y0 Y0 t /2 Y0 Y0 se Y0 Y0 t /2 95%

_ 2

X0 X
^
2 1
var Y0 Y0 1
n xi2
 Confidence intervals (bands) for
Implications mean Y and individual Y values.

Confidence interval for individual Y0 is

wider than that for the mean value of
Y0

Width of confidence intervals is

smallest when X=X-bar, implying that
the predictive ability of the SRF falls as
X departs from X-bar.

Histogram of Residuals Jarque-Bera Test of Normality

Large sample test on the OLS

residuals
2
S2 K 3
JB n
6 4
Where S = skewness coefficient,
K= kurtosis coefficient, n=sample size
Ho: residuals are normally distributed
JB follows the chi-square distribution
with d.f. =2

Other features of normal distribution

Implementing J-B in Stata 11
Zero skewness (3rd moment: degree of asymmetry, or departure from
symmetry of a distribution).
.predict ehat, residuals
Kurtosis, distinguishes between two symmetric distribution, = 3 for a
. summarize ehat, detail
normal distribution. . scalar jb = (r(N)/6)*( (r(skewness)^2)
+ ((r(kurtosis)-3)^2)/4 )
In Excel (and in some other textbooks), the measure of kurtosis is . display "Jarque-Bera Statistic = " jb
measured as (α4 – 3), referred to as excess kurtosis. If > 0, the
distribution has fatter tails than the normal distribution, such as with Jarque-Bera Statistic = 6.4065609
the t distribution. If <0, then it has thinner tails (rarer situation) . scalar pvalue = chi2tail(2,jb)
. display "Jarque-Bera p-value = "
pvalue
Jarque-Bera p-value = .0406287
Skewness-Kurtosis Test in Maximum Likelihood Estimation
Stata (sktest) Power point presentations from Dougherty
(C10G06_2016, C10G07_2016)
Please view
Properties of MLE estimators:
https://www.youtube.com/watch?v=dq-
Consistency: the sequence of MLEs converges in probability
HNPVO_SI to the value being estimated (ppt RF21 Dougherty).
Sktest adjusts for the sample size; J-B Asymptotic normality: as the sample size increases, the
distribution of the MLE tends to the Gaussian distribution with
test valid only for large samples mean θ and covariance matrix equal to the inverse of the
Interpretation is just the same Fisher information matrix.
Efficiency, i.e., it achieves the Cramér–Rao lower bound
when the sample size tends to infinity. This means that no
consistent estimator has lower asymptotic mean squared error
than the MLE (or other estimators attaining this bound).