Lecture 04
Lecture 04
Bill KP Chan
Office: HW8-14
Tel.: 3917 7059
Email: [email protected]
Point Estimators
• Assume that the population follows a certain
distribution with ! parameters ("! , "" , … , "# ). Our
objective is to estimate the parameters based on a
random sample of size & '! , '" , … , '$ .
• Based on '! , '" , … , '$ , the estimators ("(! , "(" , … , "(# )
are calculated where "(% is an estimate of "% , ) =
1,2, … , !.
• ("! , "" , … , "# ) are unknown constants to be estimated.
("(! , "(" , … , "(# ) are statistics calculated based on the
random sample, i.e., the r.v.s '! , '" , … , '$ .
Point Estimators
• (&! , &" , … , &# ) are unknown constants to be estimated.
(&*! , &*" , … , &*# ) are statistics calculated based on the r.v.s +! , +" , … , +$ .
$% ! # ( # ) ) *
where 1 =
#
. Hence, ! " = $
&Γ
#
=
$
& &Γ
# #
=
#$
Also, =) ≈ 2198. Putting =) = ! " , we have
D
C+ = ≈ 3.25×10 &+
2×2198#
Example 4 Use the method of moment to estimate
the parameters f and g " of a normal random variable ,.
The observations are 103.84, 99.26, 99.64, 99.86, 102.25, 98.99.
;
;
$ % = 1*2 % + $ % = *) ,+ ; + *) ; ,+ ;
$ % ; = '; ⇒ *) ,+ ; + *) ; ,+ ; = '; … (2)
'9;
1 and 2 ⇒ *) =
'; – '9;
'; – '9;
and ,+ =
'9
Method of Maximum Likelihood
Let ! be a r.v. whose pdf "1 ($) depends on an unknown vector of parameters
&. Let !! , !" , . . . , !$ be a random sample of !. The likelihood function of the
sample space is defined as
, = - !! , !" , … , !$
.
To find the global maximum, an effective way is to find the critical points of
the log-likelihood function ln )(&) . Since the log function is monotonic
increasing, if ln )(&) attains its global maximum at &,, )(&) will also attain
,
its global maximum at &.
The steps of the method of maximum likelihood are
1. Identify the pdf 9< (:) (or pmf in discrete case).
2. Construct the likelihood function ;(<) from the
pdf and the sample
3. Calculate the log-likelihood function
4. Find the critical point(s) of ln ;(<) by putting
all first order partial derivatives with respect to
>9 , >; , … , >= of ln ;(<) to zero.
5. Solve for >+9 , >+; , … , >+= from the r equations
resulted in step 4.
Example 1 A continuous r.v. $ has pdf
% & = ' + 1 & ! (0 ≤ & ≤ 1)
where ' > 0 is a constant.
Observed values of $ are: 0.2, 0.4, 0.5, 0.7, 0.8, 0.8, 0.9, 0.9
Estimate '.
Solution The likelihood function > ? =
? + 1 0.2 & B ? + 1 0.4 & B ? + 1 0.5 & B ? + 1 0.7 & B
? + 1 0.8 & B ? + 1 0.8 & B ? + 1 0.9 & B ? + 1 0.9 &
= ? + 1 ' 0.2×0.4×0.5×0.7×0.8×0.8×0.9×0.9 &
' &
≈ ?+1 0.014515
G$ > ? ≈ 8 B ln ? + 1 + ? B ln 0.014515
J 8
G$ > ? =0⇒ + ln 0.014515 = 0
J? ?L + 1
'
⇒ ?L = −1 − ≈ N. OP
() *.*!,-!-
Note: the method of moment estimator #" ≈ 0.857
Example 2 Time to failure . of 9 independent transistors of
the same type are observed:
620, 710, 1050, 2100, 2510, 2790, 3050, 3320, 3630
Assume that . has an exponential distribution with constant failure rate
λ per unit time. Estimate 6.
B$ = 620×710×1050×2100×2510×2790×3050×3320×3630
B# = 620# + 710# + 1050# + 2100# + 2510# + 2790# + 3050# + 3320# + 3630#
Thus, ln ) 2 = 9 ln 2 + ln B! − 2 D B" /2
F 9 B" 18
ln ) 2 =0⇒ − = 0 ⇒ 2= ≈ ≈ H. HH×;<*B
F2 2= 2 B"
! F !
The likelihood function ) N, O " = exp − -(N)
"-D "D!
J Y! + ⋯ + Y? !$ − Y! + ⋯ + Y?
ln > ` =0⇒ − =0
J` ` 1−`
Y! + ⋯ + Y?
⇒ `̂ =
!$
In general, the likelihood function of a normal random variable . is
+ -. * + -. * + -. *
! / , * ! / * * ! / 0 *
> S, T # = R */ B R */ … R */
#=> #=> #=>
! ? ! ? #
= exp − ∑ Y" − S
#=> #> * "@!
# ? ? ! ?
Thus, ln > S, T = − ln 2Z − ln T # − *
∑ "@! Y" − S
#
# # #>
A A
From ln > S, T # = 0 and ln > S, T # = 0, we get
AB A> *
! !
S[ = ∑?"@! Y" = Y̅ and T[ # = ∑?"@! Y" − Y̅ # = ]?#
? ?
U8
V U9
V
U8
V U9
V
U8
V U9
V
U8
V U9
V
U8
V U9
V
U8
V U9
V
U8
V U9
V
U8
V U9
V
U8
V U9
V
U8
V U9
V
U8
V U9
V
Q=T
Assume
1. &~j S, T # (Population is normally distributed)
2. &! , … , &? is a random sample of size $ from population &.
3. &^ and _?/!#
are the sample mean and sample variance.
/m and l/,(
.
are statistically independent
/m − J
~ o 0,1
Q/ L
.
L − 1 l/,( . L−1
~ n
Q.
/m − J
~ @(L − 1)
l/,(/ L
Example The diameters of the cups produced in a production line are
normally distributed with unknown mean N (cm) and known variance 0.2"
(cm2). Random sample of size 7 from the production line is drawn and the
diameters are measured as follows (unit: cm).
7.89, 7.75, 8.33, 8.04, 7.64, 7.65, 8.23
Find the 95% confidence interval for the mean N.
!X − N
! = 0.05 ]= ~ ` 0,1
O/ ^
Solution We have O = 0.2, ^ = 7
!X = 7.89 + 7.75 + 8.33 + 8.04 + 7.64 + 7.65 + 8.23 /7 ≈ 7.93
..O,*E
Thus, −Z&.&"N < ] = < ]&.&"N . From a statistical table of the standard
&."/ .
normal distribution, it can be found that the upper 2.5% point of b&.&"N is
;. cd.
95% = O −1.96 < X < 1.96
7.93 − T
−1.96 < < 1.96
0.2/ 7
0.2 0.2
⇒ 7.93 − 1.96× < T < 7.93 + 1.96×
7 7 Area =α/2 Area =α/2
⇒ 7.785 < T < 8.081
-zα/2 zα/2
Exercise A firm produces light bulbs that have a length of life that is
approximately normally distributed, with a standard deviation of 40 hours. If
a random sample of 30 bulbs has an average life of 780 hours,
(a) find a 96% C.I. for the population mean of all bulbs produced by this firm;
(b) how large a sample is needed if we wish to be 96% confident that our
sample mean will be within 10 hours of the true mean?
780 − T
−2.0537 < < 2.0537
40/ 30
40 40
⇒ 780 − 2.0537× < T < 780 + 2.0537×
30 30
⇒ 765 < T < 795
Exercise A firm produces light bulbs that have a length of life that is
approximately normally distributed, with a standard deviation of 40 hours. If
a random sample of 30 bulbs has an average life of 780 hours,
(a) find a 96% C.I. for the population mean of all bulbs produced by this firm;
(b) how large a sample is needed if we wish to be 96% confident that our
sample mean will be within 10 hours of the true mean?
& '" = 7.89 + 7.75 + 8.33 + 8.04 + 7.64 + 7.65 + 8.23 ≈ 55.53
":$
55.53
∴ '] = ≈ 7.93
7
;
& '"# = 7.89# + 7.75# + 8.33# + 8.04# + 7.64# + 7.65# + 8.23# ≈ 440.97
":$
; ; ; #
1 55.53#
& '" − '] # = & '"# − & '" ≈ 440.97 − = 0.4585
% 7
":$ ":$ ":$
;
1 0.4585
#
∴ ^;2$ = & '" − '] #
≈ = 0.0764
%−1 6
":$
Notation Let q 0 < q < 1 be a real value number.
@Z L is used to denote the point associated with a r.v.
;~@ L such that,
`
s ; ≥ @Z L =0 2a (1) 31 = q
_! /
That is, @Z L is the point such that the area to its right is q.
#
We have, % = 7, ^;2$ ≈ 0.0764.
# 7 − 1 0.0764 #
∴ k%.<<' 7−1 ≤ ≤ k %.%%' 7−1
l#
From a statistical table of the chi square
distribution, it can be found that Area=α/2 Area=α/2
#
k%.<<' 6 = n. ofpf and
#
k%.%%' 6 = qr. pp χ21-α/2, n-1 χ2α/2, n-1
# 0.4584 #
∴ O k%.<<' 6 ≤ ≤ k%.%%' 6 = 99%
l#
# 0.4584 0.4584 #
0.4584
k%.<<' 6 ≤ ⇒ 0.6757 ≤ ⇒l ≤ ≈ 0.6784
l# l# 0.6757
0.4584 # 0.4584 # ≥
0.4584
≤ k%.%%' 6 ⇒ ≤ 18.55 ⇒ l ≈ 0.0247
l# l# 18.55
∴ 0.0247 ≤ l # ≤ 0.6784
Exercise
The weights of 10 bags of rice are 10.2, 9.7, 10.1, 10.3, 10.1, 9.8, 9.9,
10.4, 10.3, and 9.8 kg. Assuming an approximate normal distribution, (a)
find a 99% C.I. for the mean of all such bags; (b) find a 99% C.I. for T # .
Answer: (a) 9.81, 10.31 (b) (0.023, 0.313)
Solution ^ = 10, o = 1 − 99% = 0.01 [,-.
VU ± X)/+ Y − 1 ×
JR/" ^ − 1 = J&.&&N 9 = 3.25 Y
" " 0.246
n!*R/" ^ − 1 = n&.OON 9 = 1.735 = 10.06 ± 3.25×
10
" " = 9.807,10.313
nR/" ^ − 1 = n&.&&N 9 = 23.59
!&&.F
∑$JS! !J ≈ 100.6, !X = = 10.06 +
!& Y − 1 [,-.
∑$JS! !J" ≈ 1012.58 At d = 0.01, ~g + Y − 1
e+
! Hence, we have
∑$JS! !J − !X " = ∑$JS! !J" − ∑$JS! !J "
$ 9×0.0604
!&&.F! +
g/.112 9 ≤ +
≤ g/.//2 9
= 1012.58 − = 0.544 e +
!&
9×0.0604
∑) U ! 1.735 ≤ ≤ 23.59
" '(" 1' *1 &.NAA e+
m$*! = = = 0.0604 ⇒ 0.023 ≤ e + ≤ 0.311
$*! O
m$*! = 0.0604 = 0.246
Example
A random sample of size 20 from a normal distribution has a mean Y̅ =
32.8 and a standard deviation ]?/! = 4.51.
(a) Construct a 95% C.I. for S. (b) Construct a 95% C.I. for T.
Solution We have:
$ = 20, ]?/! = 4.51, &^ = 32.8, f = 0.05,
(a) kH/# $ − 1 = k*.*#- 19 = 2.093.
I
The C.I. is &^ ± kH/# $ − 1 0-,
?
,.-!
= 32.8 ± 2.093× = (30.69, 34.91)
#*
# #
(b) mH/# $ − 1 = m*.*#- 19 = 32.85
# #
m!/H/# $ − 1 = m*..J- 19 = 8.907
*
?/! K0-, *
?/! K0-, !.×,.-!* !.×,.-!*
#
The C.I. for T is * , * = ,
L=/* ?/! L,-=/* ?/! %#.'- '..*J
= 11.76, 43.39
The C.I. for T is 11.76, 43.39 = (3.430, 6.587)
Summary
Assume that
1. !~` N, O " (Population is normally distributed)
2. !! , … , !$ is a random sample of size ^ from population !.
3. !X and m$*!" are the sample mean and sample variance.
/m − J
~ o 0,1
Q/ L
1 − f % C.I. on S (T # known)
T T
^
& − nH/# B , &^ + nH/# B
$ $
Summary
Assume that
1. '~v T, l # (Population is normally distributed)
2. '$ , … , '; is a random sample of size % from population '.
3. '] and ^;2$#
are the sample mean and sample variance.
'] − T #
% − 1 ^;2$
~ b(% − 1) # ~ k# % − 1
^;2$ / % l
N O* = O+ + ⋯ + O, N O*& = O+& + ⋯ + O,&
&
1 1 1
OR = N O* &
T,-+ = N O*& − N O*
S S−1 S
1 − S % C.I. on T (l # unknown)
^;2$ ^;2$
'] − b?/# % − 1 x , '] + b?/# % − 1 x
% %
1 − S % C.I. on l #
# #
% − 1 ^;2$ % − 1 ^;2$
# , #
k?/# %−1 k$2?/# %−1