Scribd
Upload
427 views5 pages

Show Sample Mean and Variance Are Independent Under Normality

X and S^2 are independent when samples are drawn from a normal distribution. This is proved using a series of lemmas: 1) The sum of squared random variables equals (n-1)S^2 + nX^2 2) The sum of squared centered random variables equals (n-1)S^2 + n(X-μ)^2 3) Z^2 ~ χ^2(1) if Z ~ N(0,1) 4) The sum of independent χ^2 random variables is χ^2 distributed. It follows that X is independent of Xi - X, and S^2 is a function of Xi - X,

Uploaded by

Fatin
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
427 views5 pages

Show Sample Mean and Variance Are Independent Under Normality

X and S^2 are independent when samples are drawn from a normal distribution. This is proved using a series of lemmas: 1) The sum of squared random variables equals (n-1)S^2 + nX^2 2) The sum of squared centered random variables equals (n-1)S^2 + n(X-μ)^2 3) Z^2 ~ χ^2(1) if Z ~ N(0,1) 4) The sum of independent χ^2 random variables is χ^2 distributed. It follows that X is independent of Xi - X, and S^2 is a function of Xi - X,

Uploaded by

Fatin
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 5

Math 321: Show X and S 2 are independent

(Under the assumption the random sample is normally distributed)



A well known result in statistics is the independence of X and S 2 when X1 , X2 , , Xn N , 2 .
This handout presents a proof of the result using a series of results. First, a few lemmas are presented
2
is
which will allow succeeding results to follow more easily. In addition, the distribution of (n1)S
2
derived.
Definition 1. The sample variance is defined as
n

S2 =

1 X
(Xi X)2
n 1 i=1

Lemma 1. The sum of the squares of the random variables X1 , X2 , , Xn is


n
X

Xi2 = (n 1)S 2 + nX

i=1

Proof. By Definition 1,
(n 1)S 2 =

n
X

n
X

(Xi X)2 =

i=1

Xi2 2X

i=1

n
X

Xi +

i=1

n
X

X =

i=1

n
X

Xi2 2nX + nX =

n
X

i=1

Xi2 nX

i=1

It follows that
n
X

Xi2 = (n 1)S 2 + nX

i=1

Lemma 2. The sum of squares of the random variables X1 , X2 , Xn centered about the mean, , is
n
X

(Xi )2 =

n
X
(Xi X)2 + n(X )2
i=1

i=1

Proof. The sum of squares can be simplified as


n
X

(Xi )2 =

i=1

n
X

Xi2 2

i=1

n
X

n
X
i=1

Xi +

n
X

i=1

Xi2 2nX + n2

(1)

i=1

By Lemma 1, (1) simplifies to


n
n
X
X
2
(Xi )2 =
Xi2 2nX + n2 = (n 1)S 2 + nX 2nX + n2
i=1

i=1

= (n 1)S 2 + n(X )2
=

n
X
(Xi X)2 + n(X )2
i=1

(2)

X and S 2 are independent, page 2


Lemma 3. If Z N(0, 1), then Z 2 2 (1).
Proof. The moment generating function for Z 2 is defined as
 2 Z 1
2
2
etz ez /2 dz
2
MZ (t) = E etZ =
2

Z
=



1
1
exp (1 2t)z 2 dz
2
2

#
"
1
1 z2
exp
dz
=
1
2 12t
2

|
{z
}
1
kernel of a N (0, 12t
)
#
"
Z
1
1
1 z2
 exp
dz
=

1
2 12t
1 2t 2 1
12t
|
{z
}

integrates to 1

1
1 2t

= (1 2t)1/2
Note that this is the moment generating function for a 2 random variable with one degree of freedom.
Hence,
Z 2 2 (1)

Lemma 4. Suppose X1 , X2 , , Xn are independent and identically distributed 2 (1) random


variables. It follows that
Y =

n
X

Xi 2 (n)

i=1

Proof. The moment generating function of Xi is


MXi (t) = (1 2t)1/2 .
It follows that the moment generating function for Y is
n
n
Y
 

 Y
MY (t) = E etY = E etX1 +tX2 ++tXn =
MXi (t) =
(1 2t)1/2
i=1

= (1 2t)

Pn

i=1

i=1

1/2

= (1 2t)n/2
It follows that this is the MGF for a 2 distribution with n degrees of freedom. Hence,
Y =

n
X
i=1

Xi 2 (n)

X and S 2 are independent, page 3

Theorem 1. Suppose X1 , X2 , , Xn is a random sample from a normal distribution with mean, ,


and variance, 2 . It follows that the sample mean, X, is independent of Xi X, i = 1, 2, , n.
Proof. The joint distribution of X1 , X2 , , Xn is
(
2 )
n 
1
1 X xi
fX (x1 , x2 , , xn ) =
exp
n
2 i=1

(2) 2 n
Transform the random variables Xi , i = 1, 2, , n to
Y1 = X

X = Y1

Y2 = X2 X

X2 = Y2 + Y1

Y3 = X3 X

X3 = Y3 + Y1

..
. =

..
.

..
. =

Yn = Xn X

..
.

Xn = Yn + Y1

The Jacobian of the transformation can be shown to not depend on Xi or X and is equal to the
constant n. It follows that
fY1 ,Y2 , ,Yn (y1 , y2 , , yn ) = fX (x1 , x2 , , xn )|J|
= nfX (x1 , y1 + y2 , , y1 + yn )
(

1X
= constants exp
2 i=1

xi

2 )
(3)

Note that the sum in the exponent of the joint pdf can be simplified using Lemma 2. It follows that
2
n 
X
xi
i=1

n
1 X
(xi )2
2 i=1

" n
#
1 X
= 2
(xi x
)2 + n(
x )2
i=1
"
#
n
X
1
2
2
2
= 2 (x1 x
) +
(xi x
) + n(
x )

i=2
Note that since

n
X

(xi x
) = 0, it follows that

i=1

x1 x
=

n
X
i=2

(xi x
)

(4)

X and S 2 are independent, page 4

Therefore, equation (4) simplifies to


2
n 
X
xi
i=1

"
#
n
X
1
)2 +
= 2 (x1 x
(xi x
)2 + n(
x )2

i=2

!2
n
n
X
1 X
= 2
(xi x
) +
(xi x
)2 + n(
x )2

i=2
i=2

!2
n
n
X
1 X
= 2
yi
+
yi2 + n(y1 )2

i=2
i=2

Therefore, the pdf of Y1 , Y2 , , Yn , equation (1), simplifies to


(
2 )
n 
1 X xi
fY1 ,Y2 , ,Yn (y1 , y2 , , yn ) = constants exp
2 i=1

= constants exp

2 2

n
X

!2
yi

i=2

n
X
i=2

yi2 + n(y1 )2

!2
n
n

n n
o
X
1 X
yi
+
yi2 exp 2 (y1 )2
= constants exp 2
|
2
2{z
}
i=2
i=2
|
{z
}
g(y1 )

h(y2 ,y3 , ,yn )

= constants h(y2 , y3 , , yn ) g(y1 )


Because fY1 ,Y2 , ,Yn (y1 , y2 , , yn ) can be factored into a product of functions that depend only their
respective set of statistics, it follows that Y1 = X is independent of Yi = Xi X, i = 2, 3, , n.
Pn
Finally, since X1 X = i=2 (Xi X), it follows that X1 X is a function of Xi X,
i = 2, 3, , n. Therefore, X1 X is independent of Y1 = X.
Theorem 2. Suppose X1 , X2 , , Xn is a random sample from a normal distribution with mean, ,
and variance, 2 . It follows that the sample mean, X, is independent of the sample variance, S 2 .
Proof. The definition of S 2 is given in Definition 1. Because S 2 is a function of Xi X, i = 1, 2, , n,
it follows that S 2 is independent of X.
Theorem 3. Suppose X1 , X2 , , Xn is a random sample from a normal distribution with mean, ,
and variance, 2 . It follows that the distribution of a mulitiple of the sample variance follows a 2
distribution with n 1 degrees of freedom. In particular,
(n 1)S 2
2 (n 1)
2
Proof. Equation (2) states
n
X
(Xi )2 = (n 1)S 2 + n(X )2 .
i=1

X and S 2 are independent, page 5

It follows that
2
n 
X
Xi
i=1

2
n 
X
Xi
i=1


2
X
(n 1)S 2
+
n
2

(n 1)S 2
+
=
2

/ n

2

U =W +V



2
Xi
Note that since Xi N , 2 , it follows that Zi =
N(0, 1). Similarly, since X N , n ,
2


2
X
Xi
X
N(0, 1). By Lemma 3, it follows that

then
2 (1) and V =
2 (1).

/ n
/ n
2
n 
X
Xi
2 (n). Therefore, since W and V are independent,
By Lemma 4, it follows that U =

i=1
then the moment generating function of U is
MU (t) = MW (t)MV (t)
(1 2t)n/2 = MW (t)(1 2t)1/2
= MW (t) =

(1 2t)n/2
= (1 2t)(n1)/2
(1 2t)1/2

The moment generating function for W is recognized as coming from a 2 distribution with n 1
degrees of freedom. Hence,
W =

(n 1)S 2
2 (n 1)
2

You might also like