Parameters estimation

1 / 83
 Outline
Point estimate and construct confidence interval for
popular parameters
• mean population µ
• variance population σ 2
• proportion population p
Based on sample data, one makes inferences or
generalizations about population parameter
1 / 83
Statistics inference: generalize and
prediction

State that the average cost to build a residence in

Charleston, South Carolina, is between $330,000 and
$335,000, based on the estimates of 3 contractors
selected at random from the 30 now building in this city

2 / 83
 Inference about Population from
Sample Information
• compute statistics from a selected sample from a
population
• From this statistics, make some statement about
a parameter of a population

3 / 83
Two Majors in Statistic inference

• Estimation
• Hypothesis testing

4 / 83
 Estimation
• Population parameters (unknown): Mean, variance,
standard deviation ...
• Statistics (from data): Sample mean, sample
variance ...
• Use statistics to estimate parameter: point estimate
• How accurate: interval estimate
5 / 83
A random sample of size n is a sequence of RVs

X1, . . . , Xn

Point estimate
A point estimate of some population parameter θ from
random sample X1, X2, . . . , Xn is a single value θ̂ of a
statistic Θ̂ = Θ̂(X1, . . . , Xn)

6 / 83
 Example
• A value of sample mean
X1 + . . . Xn
X̄ =
n
is a point estimate of the population mean µ.
• p̂ = nx is a point estimate of the true proportion p for
a binomial experiment.

7 / 83
 Statistics is a random variable

Example

• value of sample mean depends on the sample that

you observe
• sample is chosen randomly → different value of
sample mean for difference sample
• Sample mean is a Random variable

8 / 83
• We do not expect X̄ to estimate µ exactly, but we
certainly hope that it is not far off.
• it is possible to obtain a closer estimate of µ by
using the sample median X̃ as an estimator
• Not knowing the true value of µ, we must decide in
advance whether to use X̄ or X̃ as our estimator.
• What are the desirable properties of a “good”
decision function that would influence us to
choose one estimator rather than another?
9 / 83
• We do not expect X̄ to estimate µ exactly, but we
certainly hope that it is not far off.
• it is possible to obtain a closer estimate of µ by
using the sample median X̃ as an estimator
• Not knowing the true value of µ, we must decide in
advance whether to use X̄ or X̃ as our estimator.
• What are the desirable properties of a “good”
decision function that would influence us to
choose one estimator rather than another?
9 / 83
• We do not expect X̄ to estimate µ exactly, but we
certainly hope that it is not far off.
• it is possible to obtain a closer estimate of µ by
using the sample median X̃ as an estimator
• Not knowing the true value of µ, we must decide in
advance whether to use X̄ or X̃ as our estimator.
• What are the desirable properties of a “good”
decision function that would influence us to
choose one estimator rather than another?
9 / 83
• We do not expect X̄ to estimate µ exactly, but we
certainly hope that it is not far off.
• it is possible to obtain a closer estimate of µ by
using the sample median X̃ as an estimator
• Not knowing the true value of µ, we must decide in
advance whether to use X̄ or X̃ as our estimator.
• What are the desirable properties of a “good”
decision function that would influence us to
choose one estimator rather than another?
9 / 83
 Unbiased estimator

A statistics Θ̂(X1, . . . , Xn) is said to be an unbiased

estimator for (population) parameter θ if

E(Θ̂) = θ

10 / 83
 Example
Population with mean µ and variance σ 2
X1 +...Xn
• Sample mean X̄ = n
E(X̄) = µ
(X1 −X̄)2 +...(Xn −X̄)2
• Sample variance S2 = n−1
2 2
E(S ) = σ
Sample mean X̄ and sample variance S2 are unbiased
estimators of µ and σ 2 respectively
11 / 83
 Variance of estimator
• Θ̂1, Θ̂2: unbiased estimators for θ
• Θ̂1 is a more efficient estimator than Θ̂2 if
Var(Θ̂1) ≤ Var(Θ̂2)

^
1

^
3

^
2

^
θ
θ 12 / 83
 Efficient estimator

• The most efficient estimator: unbiased estimator

with smalles variance
• X̄ and S2 are the most efficient estimators of µ and
σ2

13 / 83
 Example

Given the sample data

1, 1, 4, 6

Find the best point esimators for the population mean and
population variance

14 / 83
Solution
The best estimator for the population mean is the sample
mean
1+1+4+6
x̄ = =3
4
The best esimator for the population variance is the
sample variance

2 (1 − 3)2 + (1 − 3)2 + (4 − 3)2 + (6 − 3)2

s = =6
4−1
15 / 83
Solution
The best estimator for the population mean is the sample
mean
1+1+4+6
x̄ = =3
4
The best esimator for the population variance is the
sample variance

2 (1 − 3)2 + (1 − 3)2 + (4 − 3)2 + (6 − 3)2

s = =6
4−1
15 / 83
 Interval estimate
• estimation accuracy increases with large samples
• but don’t expect X̄ to be exactly µ
• Want to find an interval around X̄ so we can be sure
that µ is in it.
• Ex: want to find [a, b] so that 95% of the time
µ ∈ [a, b]
• [a, b] is called 95% confidence interval estimate of µ
16 / 83
 Interval Estimates
An interval estimate of a population parameter θ is an
interval of the form

θ̂L < θ < θ̂U

where θ̂L and θ̂U depend on the value of the statistic θ̂ for
a particular sample and also on the distribution of θ̂

17 / 83
 Interpretation of Interval Estimates

• different samples will generally yield different

values of θ̂ and different values for θ̂L and θ̂U
• These end points θ̂L and θ̂U are random variables
• If distribution of θ̂ is known then we can determine
P(θ̂L < θ < θ̂U ) = 1 − α
then we have a probability of 1 − α of selecting a
random sample that will produce an interval
containing θ 18 / 83
 Interpretation of Interval Estimates

• different samples will generally yield different

values of θ̂ and different values for θ̂L and θ̂U
• These end points θ̂L and θ̂U are random variables
• If distribution of θ̂ is known then we can determine
P(θ̂L < θ < θ̂U ) = 1 − α
then we have a probability of 1 − α of selecting a
random sample that will produce an interval
containing θ 18 / 83
 Interpretation of Interval Estimates

• different samples will generally yield different

values of θ̂ and different values for θ̂L and θ̂U
• These end points θ̂L and θ̂U are random variables
• If distribution of θ̂ is known then we can determine
P(θ̂L < θ < θ̂U ) = 1 − α
then we have a probability of 1 − α of selecting a
random sample that will produce an interval
containing θ 18 / 83
• The interval
θ̂L < θ < θ̂U
computed from the selected sample is called a
100(1 − α)% confident interval
• The fraction 100(1 − α)%: confidence coefficient
or degree of confidence
• θ̂L and θ̂U : lower and upper confident limits

19 / 83
• The interval
θ̂L < θ < θ̂U
computed from the selected sample is called a
100(1 − α)% confident interval
• The fraction 100(1 − α)%: confidence coefficient
or degree of confidence
• θ̂L and θ̂U : lower and upper confident limits

19 / 83
• The interval
θ̂L < θ < θ̂U
computed from the selected sample is called a
100(1 − α)% confident interval
• The fraction 100(1 − α)%: confidence coefficient
or degree of confidence
• θ̂L and θ̂U : lower and upper confident limits

19 / 83
Estimate the mean when the
variance population σ2 is known

20 / 83
 Sample mean
Select a random sample of size n, X1, . . . , Xn, from a
population with mean µ and finite variance σ 2.
Sample mean
X1 + . . . + Xn
X̄ =
n
is used to estimate the true mean µ of the population -
called a point estimate of µ

21 / 83
 Properties of sample mean
• Expectation
E(X̄) = µX̄ = µ
unbiased estimator of the true mean µ
• Variance
σ2
Var(X̄) = σX̄2 =
n

22 / 83
 Sample mean from normal
population
• Population has normal distribution N(µ, σ 2)
• Each observation X1, . . . , Xn ∼ N(µ, σ 2)
• (X1 + · · · + Xn) ∼ N(nµ, nσ 2)
• Sample mean X̄ ∼ N(µ, σ 2/n)

23 / 83
 Central limit theorem
• Suppose X1, . . . , Xn i.i.d with mean µ and variance
σ2.
• then for n large enough, X̄ has distribution
approximately normal with mean µ and variance
σ2
.
n

24 / 83
25 / 83
 Distribution of X̄
• Sample mean
Pn
σ2
 
i=1 Xi
X̄ = ∼N µ,
n n
• Standadize
X̄ − µ
√ ∼ N (0, 1)
σ/ n
• Valid for large sample (n ≥ 30) or not severely
nonnormal population
26 / 83
For sample size n ≥ 30 or normal
population

X̄ − µ
√ ∼ N(0, 1)
σ/ n

27 / 83
zα/2 is the critical
value determined by

P(Z > zα/2) = α/2

P(−zα /2 < Z < zα/2) = 1 − α

28 / 83
 X̄ − µ
P(−zα/2 < √ < zα/2) = 1 − α
σ/ n
Hence
σ σ
P(X̄ − zα/2 √ < µ < X̄ + zα/2 √ ) = 1 − α
n n

29 / 83
a 100(1 − α)% confidence interval (CI) for population mean µ

σ σ
X̄ − zα/2 √ < µ < X̄ + zα/2 √
n n
(two-sided CI)
100(1 − α)%: confidence level
• variance population σ 2 known
• Normal population or large sample size n ≥ 30
30 / 83
a 100(1 − α)% confidence interval (CI) for population mean µ

σ σ
X̄ − zα/2 √ < µ < X̄ + zα/2 √
n n
(two-sided CI)
100(1 − α)%: confidence level
• variance population σ 2 known
• Normal population or large sample size n ≥ 30
30 / 83
 Interval estimate for different samples

10
9
8
7
Sample

6
5
4
3
2
1
x
µ
31 / 83
 Remark
• Given z α , all of these intervals are of the same
2
width, since their widths once x̄ is determined
• The larger the value z α is, the wider the intervals are
2
and the more confident we can be that the particular
sample selected will produce an interval that
contains the unknown parameter µ
• For each z α , 100(1 − α)% of the intervals will cover
2
µ
32 / 83
If x̄ is used as an estimate of µ, we can be confident that
the error will not exceed z α2 √σn at 100(1 − α)% .

Margin of error ME = zα/2 √σn is used to evaluate

accuracy of estimation.
Alternative formular for IC
(x̄ − ME, x̄ + ME)
33 / 83
If x̄ is used as an estimate of µ, we can be confident that
the error will not exceed z α2 √σn at 100(1 − α)% .

Margin of error ME = zα/2 √σn is used to evaluate

accuracy of estimation.
Alternative formular for IC
(x̄ − ME, x̄ + ME)
33 / 83
 Example

The average zinc concentration recovered from a sample

of measurements taken in 36 different locations in a river
is found to be 2.6 grams per milliliter. Find the 98% CIs
for the mean zinc concentration in the river. Assume
that the population standard deviation is .3 gram per
milliliter.

34 / 83
 Solution for 98% CI

• µ: the mean zinc concentration in the river

• population std σ = .3
• sample size n = 36, sample mean x̄ = 2.6 g/mil

35 / 83
• Find α/2 from level of confidence
1 − α = 98% = .98 ⇒ α/2 = .01

= 0.02

2
= 0.01 2
= 0.01

0.98

–z /2 0 z /2

36 / 83
• Find z α = 2.33
2

z .00 .01 .02 .03 ... .09

0.0
0.1
...

2.3 0.9901

In Excel, NORMINV(1 - α2 , 0, 1)
37 / 83
• Marginal of error
ME = z α2 √σn = 2.33 × √.336 = .1165
• lower bound x̄ − ME = 2.6 − .1165 = 2.4835
• upper bound x̄ + ME = 2.6 + .1165 = 2.7165
• 98% CI
2.4835 < µ < 2.7165

38 / 83
 Practice

A survey of 30 emergency room patients found that the

average waiting time for treatment was 174.3 minutes.
Assuming that the population standard deviation is 46.5
minutes, find the best point estimate of the population
mean and the 99% confidence of the population mean

39 / 83
 Sample size

Given level of confidence and accuracy ME, one can

determine necessary size of sample
 α 2
z2σ
n=
ME

40 / 83
 Example
From past experience it is known that the weights of
salmon grown at a commercial hatchery are normal with
a mean that varies from season to season but with a
standard deviation that remains fixed at 0.3 pounds. If we
want to be 95 percent certain that our estimate of the
present season’s mean weight of a salmon is correct to
within ±0.1 pounds, how large a sample is needed?

41 / 83
 Solution
• Find sample size n such that ME = .1
• Information
• population std σ = .3
• Confidence level 1 − α = .95 ⇒ z α = 1.96
2

• ME = z α √σn = 1.96 √
0.3
n
= .1
2

• then n = 35
42 / 83
 Practice
An electrical firm manufactures light bulbs that have a
length of life that is approximately normally distributed
with a standard deviation of 40 hours.
1 If a sample of 30 bulbs has an average life of 780
hours, find a 96% confidence interval for the
population mean of all bulbs produced by this firm
2 How large a sample is needed if we wish to be 96%
confident that our sample mean will be within 10
hours of the true mean? 43 / 83
Estimate the mean when the population
variance σ 2 is not known
In statistics - replace unknown population standard
deviation σ by computable sample standard deviation S
X̄ − µ
√
S/ n
Distribution of statistics? depends on
• distribution of sample mean X̄
• distribution of sample standard deviation S
44 / 83
Estimate the mean when the population
variance σ 2 is not known
In statistics - replace unknown population standard
deviation σ by computable sample standard deviation S
X̄ − µ
√
S/ n
Distribution of statistics? depends on
• distribution of sample mean X̄
• distribution of sample standard deviation S
44 / 83
Estimate the mean when the population
variance σ 2 is not known
In statistics - replace unknown population standard
deviation σ by computable sample standard deviation S
X̄ − µ
√
S/ n
Distribution of statistics? depends on
• distribution of sample mean X̄
• distribution of sample standard deviation S
44 / 83
 Distribution of sample variance S2?

2 (X1 − X̄)2 + · · · + (Xn − X̄)2

S =
n−1

Chi-square distribution

• X1, . . . , Xn i.i.d. N(0, 1)

• Y = X12 + · · · + Xn2 is said to have the chi-square
distribution with n degree of freedom, Y ∼ χ2n
45 / 83
 Distribution of sample variance S2?

2 (X1 − X̄)2 + · · · + (Xn − X̄)2

S =
n−1

Chi-square distribution

• X1, . . . , Xn i.i.d. N(0, 1)

• Y = X12 + · · · + Xn2 is said to have the chi-square
distribution with n degree of freedom, Y ∼ χ2n
45 / 83
 Distribution of sample variance S2?

2 (X1 − X̄)2 + · · · + (Xn − X̄)2

S =
n−1
Distribution of sample variance for normal distribution

2 (n − 1)S2
χ = 2
∼ χ2n−1
σ

46 / 83
 Distribution of sample mean X̄ for unknown σ 2

Statistics
X̄ − µ
T= √
S/ n

t - distribution
If Z ∼ N(0, 1) and C ∼ χ2n then T = √Z has
C/n
t−distribution of n degree of freedom, denoted by
T ∼ T(n)
47 / 83
 Distribution of sample mean X̄ for unknown σ 2

Statistics
X̄ − µ
T= √
S/ n

t - distribution
If Z ∼ N(0, 1) and C ∼ χ2n then T = √Z has
C/n
t−distribution of n degree of freedom, denoted by
T ∼ T(n)
47 / 83
 For normal distribution

• X̄: normal distribution

• S2: related to χ2(n − 1)

Statistics
X̄ − µ
T= √ ∼ t(n − 1)
S/ n

48 / 83
 For normal distribution

• X̄: normal distribution

• S2: related to χ2(n − 1)

Statistics
X̄ − µ
T= √ ∼ t(n − 1)
S/ n

48 / 83
 100(1 − α)% CI of µ
• normal population
• variance population σ 2 unknown

S S
(X̄ − tα/2,n−1 √ , X̄ + tα/2,n−1 √ )
n n
Margin of error
S
ME = tα/2,n−1 √
n
49 / 83
 100(1 − α)% CI of µ
• normal population
• variance population σ 2 unknown

S S
(X̄ − tα/2,n−1 √ , X̄ + tα/2,n−1 √ )
n n
Margin of error
S
ME = tα/2,n−1 √
n
49 / 83
 t - distribution looks like
t is symmetric about 0

degree of freedom ≥ 30 then t is approximated by

N (0, 1) 50 / 83
 Remark

For sample size large n ≥ 30) then t is approximated by

N (0, 1) so
tα/2,n−1 ≈ zα/2
for n large enough

51 / 83
 Example

The contents of seven similar containers of sulfuric acid

are 9.8, 10.2, 10.4, 9.8, 10.0, 10.2, and 9.6 liters. Find a
95% confidence interval for the mean contents of all
such containers, assuming an approximately normal
distribution.

52 / 83
 Solution
• µ: mean contents of all containers
• Sample size n = 7, sample mean x̄ = 10.0, sample
std s = 0.283
• Find α
2 from confidence level
α
1 − α = .95 ⇒ = .025
2

53 / 83
• Find tα/2,n−1 = t.025,6 = 2.447

54 / 83
• ME = tα/2,n−1 √sn = 2.447 × .283
√ = .26
7

• Lower bound x̄ − ME = 10.0 − .26 = 9.74

• Upper bound x̄ + ME = 10.0 + .26 = 10.26
• 95% CI
9.74 ≤ µ ≤ 10.26

55 / 83
 Practice
The following measurements were recorded for the
drying time, in hours, of a certain brand of latex paint:
3.4, 2.5, 4.8, 2.9, 3.6, 2.8, 3.3, 5.6, 3.7, 2.8, 4.4, 4.0, 5.2,
3.0, 4.8 Assuming that the measurements represent a
random sample from a normal population, find a 95%
confidence interval for the average drying time of the
paint.

56 / 83
 Remark
For large sample size n ≥ 30 then

t α2 ,n−1 ≈ z α2

Hence CI of µ is

S S
(X̄ − zα/2 √ , X̄ + zα/2 √ )
n n

57 / 83
 Practice

Scholastic Aptitude Test (SAT) mathematics scores of a

random sample of 500 high school seniors in the state of
Texas are collected, and the sample mean and standard
deviation are found to be 501 and 112, respectively. Find
a 99% confidence interval on the mean SAT
mathematics score for seniors in the state of Texas.

58 / 83
Estimate a proportion - mean of
Bernoulli RV

59 / 83
• Sample n independent trials from a population, each
success with unknown probability p
• Each observation X1, . . . , Xn ∼ Ber(p) has two
value 1 - success and 0 - failure
• X = X1 + · · · + Xn: number of successes in n
sample trials
• point estimate for p is p̂ = X̄ = X
n - fraction of
successes in sample
• Want to find confidence interval for p
60 / 83
Estimator of population proportion
• Point estimate
X
p̂ =
n
then E(p̂) = p
• Use p̂ as unbiased estimator for p
• For large sample size, by central limit theorem
X − np p̂ − p
p =q ≈ Z ∼ N(0, 1)
p(1 − p) p(1−p)
n
61 / 83
 Contruct inteval confidence for p
P(−zα/2 < Z < zα/2) = 1 − α
So
 
r r
 p(1 − p) p(1 − p) 
p̂ − zα/2
P < p < p̂ + zα/2 
n | {zn }

standard error of point estimator

However p is unknown. Replace p by p̂ in standard error

term
62 / 83
 100(1 − α)% IC for p

q q
p̂(1−p̂) p̂(1−p̂)
p̂ − zα/2 n < p < p̂ + zα/2 n

with q
p̂(1−p̂)
ME = zα/2 n

63 / 83
 Sample size

z 2
α/2
n = (p̂)(1 − p̂)
ME

64 / 83
 Example
In a random sample of 85 automobile engine crankshaft
bearings, 10 have a surface finish that is rougher than the
specifications allow.
1 Find a 95% two-sided confidence interval for p -
the proportion of bearings in the population that
exceeds the roughness specification.
2 How large a sample is required if we want to be
95% confident that the error in using p̂ to estimate p
is less than 0.05 65 / 83
 Solution
1 • Information
• sample size n = 85
• sample proportion of bearing that exceeds... :
10
p̂ = nx = 85 ≈ 0.12
• Confidence level 1 − α = .95 ⇒ zα/2 = 1.96
q q
p̂(1−p̂) (.12)(.88)
• ME = zα/2 n = 1.96 85 ≈ .07
• 95%CI for p

.12 − .07 < p < .12 + .07 or .05 < p < .19
66 / 83
2 Need to find sample size n such that

ME = 0.05

or q
p̂(1−p̂)
zα/2 n = 0.05
Solve q
1.96 (.12)(.88)
n = 0.05
and round up to obtain n = 163
67 / 83
 Example

On October 14, 2003, the New York Times reported that

a recent poll indicated that 52 percent of the population
was in favor of the job performance of President Bush,
with a margin of error of ±4 percent and 95% confidence
level. Can we infer how many people were questioned?

68 / 83
 Solution
• α = .05, z.025 = 1.96
• p̂ = .52
q q
p̂(1−p̂) .52(.48)
• ME = zα/2 = 1.96
n n
p
• 1.96 .52(.48)/n = .04
• n = 599
69 / 83
 Practice

A sample of 100 transistors is randomly chosen from a

large batch and tested to determine if they meet the
current standards. If 80 of them meet the standards, then
find 95% confidence interval for p, the fraction of all
the transistors that meet the standards.

70 / 83
Estimation of population
variance σ2 for normal
population
(n − 1)S2 2 ∼ χn−1
σ2
• n: sample size
• S2: sample variance 71 / 83
Estimation of population
variance σ2 for normal
population
(n − 1)S2 2 ∼ χn−1
σ2
• n: sample size
• S2: sample variance 71 / 83
 1
2
2

2 2
left right

72 / 83
 S2
P(χ21−α/2,n−1 < (n − 1) 2 < χ2α/2,n−1) = 1 − α
σ
or
(n − 1)S2 2 (n − 1)S2
P( 2 <σ < 2 )=1−α
χα/2,n−1 χ1−α/2,n−1

73 / 83
 100(1 − α)% CI of σ2

(n − 1)S2 2 (n − 1)S2
<σ < 2
χ2α/2,n−1 χ1−α/2,n−1

74 / 83
 Example

The sugar content of the syrup in canned peaches is

normally distributed. A random sample of n = 10 cans
yields a sample standard deviation of s = 4.8 milligrams.
Calculate a 95% two-sided CI for the population
variance σ 2.

75 / 83
 Solution

• Information
• sample size n = 10
• sample standard deviation s = 4.8mg
• Confidence level 1 − α = .95 ⇒ α/2 = .025

76 / 83
• Critical value of χ2
χ21−α/2,n−1 = χ2.975,9 = 2.7

77 / 83
• Critical value of χ2
χ2α/2,n−1 = χ2.025,9 = 19.023

78 / 83
• Upper bound
(n − 1)s2 9 ∗ (4.8)2
= = 76.8
χ21−α/2,n−1 2.7
• Lower bound
(n − 1)s2 9 ∗ (4.8)2
= = 10.9
χ2α/2,n−1 19.023
• 95% CI for population variance
10.9 < σ 2 < 76.8(mg2)
79 / 83
 Practice
The following are the weights, in decagrams, of 10
packages of grass seed distributed by a certain company:
46.4, 46.1, 45.8, 47.0, 46.1, 45.9, 45.8, 46.9, 45.2, and
46.0. Find a 95% confidence interval for the variance
of the weights of all such packages of grass seed
distributed by this company, assuming a normal
population

80 / 83
 Keywords

• point estimate and efficient estimator for population

mean, proportion and variance are sample mean,
sample proportion and sample variance

81 / 83
• two-sided 100(1 − α)% CI for population mean µ

(x̄ − ME, x̄ + ME)

• Case 1: population variance σ 2 known, large

sample size or normal population
ME = zα/2 √σn
• Case 2: population variance σ 2 unknown,
normal population ME = tα/2,n−1 √sn

82 / 83
• two-sided 100(1 − α)% CI for population
proportion p
(p̂ − ME, p̂ + ME)
where r
p̂(1 − p̂)
ME = zα/2
n
• two-sided 100(1 − α)% CI for population variance
σ 2 - normal population
(n − 1)S2 2 (n − 1)S2
<σ < 2
χ2α/2,n−1 χ1−α/2,n−1 83 / 83