HYPOTHESIS TESTING Likelihood Ratio Tests

Let X1 , X2 , . . . , Xn be independent random variables with Xi having probability den-

sity function f (xi ; θ1 , θ2 , . . . , θk ) for i = 1, 2, . . . , n. Let the parameter space of points
(θ1 , θ2 , . . . , θk ) be denoted by Ω.

To test a null hypothesis H0 : (θ1 , θ2 , . . . , θk ) ∈ ω ⊂ Ω against all alternatives, define

Y
L(ω) = f (xi ; θ1 , θ2 , . . . , θk ) (θ1 , θ2 , . . . , θk ) ∈ ω
Y
L(Ω) = f (xi ; θ1 , θ2 , . . . , θk ) (θ1 , θ2 , . . . , θk ) ∈ Ω

and let L(ω̂) and L(Ω̂) be the maximum values attained by L(ω) and L(Ω) respectively.

If λ is defined by
L(ω̂)
λ = λ(x1 , x2 , . . . , xn ) =
L(Ω̂)
the likelihood ratio tests rejects H0 iff λ ≤ λ0 where λ0 is such that

P (λ(X1 , X2 , . . . , Xn ) ≤ λ0 : H0 ) = α.

1
HYPOTHESIS TESTING Likelihood Ratio Tests

Example One

Let X1 , X2 , . . . , Xn be a random sample from a normal distribution N (θ1 , θ2 ) and consider

a test of
H0 : θ1 = 0 θ2 > 0 against H1 : θ1 6= 0 θ2 > 0.
In this case
ω = {(θ1 , θ2 ) : θ1 = 0, 0 < θ2 < ∞}
Ω = {(θ1 , θ2 ) : −∞ < θ1 < ∞, 0 < θ2 < ∞}
and  n2
(xi − θ1 )2
  P 
1
L(Ω) = exp −
2πθ2 2θ2
  n2  P 2
1 xi
L(ω) = exp −
2πθ2 2θ2

 P 2
∂ ln L(ω) ∂ n xi
= − ln 2πθ2 −
∂θ2 ∂θ2 2 2θ2
P 2
n xi
=− +
2θ2 2θ22
and
x2i
P
∂ ln L(ω)
=0 if θ2 =
∂θ2 n
giving
  n2  P 2 
1 x
L(ω̂) = P exp − P i2
x2i x
2π n 2 ni
 −1
 n2
ne
=
2π x2i
P

2
HYPOTHESIS TESTING Likelihood Ratio Tests

 
∂ ln L(Ω) ∂ n 1 X 2
= − ln 2πθ2 − (xi − θ1 )
∂θ2 ∂θ2 2 2θ2
(xi − θ1 )2
P
n
=− +
2θ2 2θ22

P
∂ ln L(Ω) (xi − θ1 )
=
∂θ1 θ2
and solving
∂ ln L(Ω) ∂ ln L(Ω)
= 0 and =0
∂θ2 ∂θ1
gives
(xi − x̄)2
P P
xi
θ1 = = x̄ and θ2 = .
n n

Thus  n2
(xi − x̄)2
  P 
1
L(Ω̂) = P exp − P
(xi −x̄)2 (xi −x̄)2
2π n 2 n
 −1
 n2
ne
=
2π (xi − x̄)2
P

and  n2
(xi − x̄)2
P
λ= P 2
xi
1
=  n2
nx̄2
1 + P(x −x̄)2 i

In the null hypothesis θ1 = 0 and x̄ = 0Pconfirms the null hypothesis and x̄ = 0 and
x2i > 0 imply λ = 1. As x̄ and nx̄2 / (xi − x̄)2 deviate from zero and negate H0 ,
P
λ < 1 so region for testing H0 is λ ≤ λ ≤ λ0 .

λ ≤ λ0 iff
1
  n2 ≤ λ0 .
nx̄ 2
1 + P(x −x̄)2
i

3
HYPOTHESIS TESTING Likelihood Ratio Tests

That is
nx̄2 −2/n
1+ P 2
≥ λ0
(xi − x̄)

nx̄2 −2/n
P 2
≥ λ0 −1
(xi − x̄)

√
n|x̄|
q
−2/n
rP ≥ (n − 1)(λ0 − 1) = c
(xi −x̄)2
n−1

which is the usual t–test.

4
HYPOTHESIS TESTING Likelihood Ratio Tests

Example Two

Consider random samples of size n and m from two normal distributions X1 ∼ N (θ1 , θ3 )
and X2 ∼ N (θ2 , θ3 ) to test

H0 : θ 1 = θ 2 θ3 > 0 against H1 : θ1 6= θ2 θ3 > 0.

In this case

ω = {(θ1 , θ2 , θ3 ) : −∞ < θ1 = θ2 < ∞, 0 < θ3 < ∞}

Ω = {(θ1 , θ2 , θ3 ) : −∞ < θ1 < ∞, −∞ < θ2 < ∞, 0 < θ3 < ∞}

and
 n+m Pn Pm
− θ1 )2 + 1 (yi − θ1 )2
  
1 1 (xi
2
L(ω) = exp −
2πθ3 2θ3
  n+m  Pn 2
Pm 2

1 1 (xi − θ1 ) + 1 (yi − θ2 )
2
L(Ω) = exp −
2πθ3 2θ3

Solving
∂ ln L(ω) ∂ ln L(ω)
= 0 and =0
∂θ1 ∂θ3
gives
(xi − θ1 )2 + (yi − θ1 )2
P P P P
x i + yi
θ̂1 = and θ̂3 = =w
n+m n+m
and
  n+m
e−1 2
L(ω̂) =
2πw

Solving
∂ ln L(ω) ∂ ln L(ω) ∂ ln L(ω)
= 0, =0 and =0
∂θ1 ∂θ2 ∂θ3
gives

(xi − θ1 )2 + (yi − θ2 )2
P P P P
xi yi
θ̂1 = , θ̂2 = and θ̂3 = = w′ .
n m n+m

5
HYPOTHESIS TESTING Likelihood Ratio Tests

Thus
  n+m
e−1 2
L(Ω̂) =
2πw′
so that
 w′  n+m
2
λ= .
w

(Xi − X̄)2 + (Yi − Ȳ )2

P P
2
λ n+m = Pn
Ȳ 2 Ȳ 2
Pm
Xi − nX̄+m + 1 Yi − nX̄+m



1 n+m n+m

1
= nm 2
n+m (X̄−Ȳ )
1+ P 2
(Xi −X̄) +
P
(Yi −Ȳ )2

n+m−2
=
(n + m − 2) + T 2
for q
nm
n+m (X̄ − Ȳ )
T = rP P
(Xi −X̄)2 + (Yi −Ȳ )2
n+m−2

which is again the usual t–test.

6
HYPOTHESIS TESTING Likelihood Ratio Tests

Example Three

Consider a random sample of size n from a uniform distribution on the interval [0, θ] to
test
H0 : θ = θ0 against H1 : θ < θ0 .
In this case
ω = {θ : θ = θ0 }
Ω = {θ : 0 < θ ≤ θ0 }
and  1 n
L(θ) = 0 ≤ xi ≤ θ
θ
so that  1 n
L(ω̂) = .
θ0

The maximum of L(θ) under Ω is obtained by using the maximum likelihood estimator
of θ which is X(n) so that
 1 n
L(Ω̂) = .
x(n)

L(ω̂) x n
(n)
λ= =
L(Ω̂) θ0
and the test is to reject H0 if x n
(n)
≤ λ0
θ0
x(n) 1/n
≤ λ0
θ0
1/n
x(n) ≤ θ0 λ0
1/n
subject to P (x(n) ≤ θ0 λ0 |H0 ) = α.

In this case
 θ λ1/n n
1/n 0 0
P (x(n) ≤ θ0 λ0 |H0 ) =
θ0
= λ0
and so the test is to reject H0 if
x(n)
≤ α1/n .
θ0

7
HYPOTHESIS TESTING Likelihood Ratio Tests

Example Four

Consider a random sample of size n from a distribution with probability density function

f (x; θ) = θe−θx θ > 0.

To test
H0 : θ = θ 0 against H1 : θ > θ 0
we have
ω = {θ : θ = θ0 }
Ω = {θ : θ ≥ θ0 }
and P
xi
L(θ) = θn e−θ
so that P
xi
L(ω̂) = θ0n e−θ0 .

To find the maximum of L(θ) under Ω note that

X
ln L(Ω) = n ln θ − θ xi
∂ ln L(Ω) n X
= − xi
∂θ θ
and the derivative is zero if θ = Pnx so that
i

 n n
L(Ω̂) = P e−n .
xi

Then P
L(ω̂)θ0n e−θ0 xi
λ= = n
L(Ω̂) Pn e−nxi

and the test is to reject H0 if λ ≤ λ0 .

8
HYPOTHESIS TESTING Likelihood Ratio Tests

We reject if
 θ P x n P
0 i
e−θ0 xi +n ≤ λ0
n
and letting y = θ0 x̄ this becomes to reject if

y n e−n(y−1) ≤ λ0 .

∂
n ln y − n(y − 1) = 0 if y = 1 the function y n e−n(y−1) ≤ λ0 has a
 
Noting that ∂y
maximum value of 1 and
y n e−n(y−1) ≤ λ0 iff y ≤ k
where k is chosen so that size of the test is α.

9
HYPOTHESIS TESTING Likelihood Ratio Tests

Example Five

Let X1 , X2 , . . . , Xn be a random sample from a normal distribution N (µ, σ 2 ).

To test
H0 : σ 2 = σ02 against H1 : σ 2 6= σ02
we have
ω = {µ, σ 2 : −∞ < µ < ∞, σ 2 = σ02 }
Ω = {µ, σ 2 : −∞ < µ < ∞, σ 2 ≥ 0}
and  1 n h 1 X  x − µ 2 i
i
L(θ) = √ exp −
σ 2π 2 σ
so that  1 n/2 h 1 X  x − µ 2 i
i
L(ω̂) = 2 exp − .
2πσ0 2 σ0

To find the maximum of L(θ) under Ω note that

√ 1 X  xi − µ 2
ln L(Ω) = −n ln σ 2π −
2 σ
∂ ln L(Ω) n 1 X
=− + 3 (xi − µ)2
∂σ σ σ
and the derivative is zero if rP
(xi − µ)2
σ=
n
so that n/2 i2 i
 n h n Xh xi − x̄
L(Ω̂) = exp −
2π (xi − = µ)2
P p
2 (xi − x̄)2
P
h n in/2
= e−n/2 .
2π (xi − x̄)2
P

Then
L(ω̂)
λ=
L(Ω̂)
 P(x − x̄)2 n/2 h 1 X  x − x̄ 2 n i
i i
= 2 exp − +
nσ0 2 σ0 2
 σ̂ 2 n/2 h n  σ̂ 2  n i
= exp − +
σ02 2 σ02 2

10
HYPOTHESIS TESTING Likelihood Ratio Tests

Let
σ̂ 2
y=
σ02
so that  n n
λ = y n/2 exp − y +
2 2
and λ ≤ λ0 is equivalent to
ye−y ≤ e−1 (λ0 )2/n .

If we have
 σ̂ 2  1 P(X − X̄)2
 
i
P 2 ≤a =P n ≤ a
σ0 σ02
 (X − X̄)2
P 
i
=P ≤ na
σ02
 (n − 1)S 2 
=P ≤ na
σ02
= P χ2n−1 ≤ na

= α/2
and similarly
 σ̂ 2 
≥ b = P χ2n−1 ≥ nb = α/2


P
σ02
the test becomes to reject H0 if

σ̂ 2 σ̂ 2
≤a or ≥b
σ02 σ02

where
na = χ2n−1;1−α/2 and nb = χ2n−1;α/2 .

11