Classification into Two Parameter Exponential

Populations with a Common Guarantee Time

Nabakumar Jana

Department of Mathematics
National Institute of Technology Meghalaya
Shillong-793003, India

IIT (ISM) Dhanbad

06-Sept-2017

(N. Jana) Classification into Two Parameter Exponential Populations with a Common Guarantee Time 1 / 15
 Introduction

Introduction

The problem of classification arises in agriculture, military surveil-

lance, clinical diagnosis, image processing and pattern recognition.
Fisher (1936) proposed LDA for classifying an observation into one
of several possible groups based on many measurements.

Suppose Π1 , Π2 are two independent populations. A classification rule is designed to

partition the sample space S by a suitable criterion into two parts R1 and R2 such that
R1 ∪ R2 = S, R1 ∩ R2 = φ.
If x ∈ R1 , allocate x to Π1 ;
If x ∈ R2 , allocate x to Π2 .

(N. Jana) Classification into Two Parameter Exponential Populations with a Common Guarantee Time 2 / 15
 Introduction

Classification Under Order Restrictions

Conde et al. (2005): proposed classification rule for one parameter exponential
populations, assuming that the means are ordered.

Fernández et al. (2006): the problem of classification into two normal distributions with
unknown mean vectors µ1 , µ2 , where µ1 − µ2 ∈ C and C is a general cone.
Conde et al. (2012): applied the rules for k populations to a cancer trial. The proteins
were used for correctly classifying the patients in one of the five levels of bladder cancer.
Tripathi et al. (2014) improved the usual MLE, UMVUE, BSE of µ under a known
ordering σ1 ≤ σ2 .

The estimation of σ1 , σ2 has not been studied under order restrictions σ1 ≤ σ2 .

(N. Jana) Classification into Two Parameter Exponential Populations with a Common Guarantee Time 3 / 15
 Introduction

The Exponential Distribution: Exp (µ, σi )

Suppose x is an observation that is to be classified into one of the k (≥ 2) exponential

populations Π1 , Π2 , · · · , Πk . The probability density function associated with Πi is
 
1 x−µ
fi (x; σi , µ) = exp − , x > µ, − ∞ < µ < ∞, σi > 0, i = 1, 2, · · · , k.
σi σi
(1)
The location parameter µ and the scale parameters σi s are unknown but σi s follow the
restriction σ1 ≤ σ2 ≤ · · · ≤ σk .

Define Xi = (Xi1 , Xi2 , · · · , Xini ), (ni ≥ 2) as a random sample from the population Πi and
these random samples are mutually independent. Suppose
ni
X
Zi = min Xij , Z = min Zi and Ti = (Xij − Z), i = 1, · · · , k.
1≤j≤ni 1≤i≤k
j=1

Denote T = (T1 , T2 , · · · , Tk ). The statistic (Z, T) is complete and sufficient for

θ = (µ, σ) where σ = (σ1 , σ2 , · · · , σk ) ∈ (0, ∞)k .

(N. Jana) Classification into Two Parameter Exponential Populations with a Common Guarantee Time 4 / 15
 Estimation Under Restrictions on Scale Parameters
P −1
k ni (ni −1)
The MLE of µ is Z, the UMVUE of µ is µ̂u = Z − i=1 Ti
and MMLE is
2 −1
P 
k ni
µ̂m = Z − i=1 Ti (Ghosh and Razmpour (1984)). The MLE of σi is σ̂i = Ti /ni .
Under the ordering σ1 ≤ σ2 , the MLEs of σ1 , σ2 are
T T + T  T T + T 
1 1 2 2 1 2
σ1RML = min , , σ2RML = max , ,
n1 n1 + n2 n2 n1 + n2
respectively. The isotonic regression of σ̂i s with appropriate weights ni s are given by the
following min-max formula.
Pt
nr σ̂r
σ̂ir = min max Av(s, t), i = 1, 2, · · · , k, where Av(s, t) = Pr=s t , s ≤ t, t ∈ {1, 2, · · ·
i=1 nr
i≤k 1≤s≤i

Theorem 1
The UMVUEs of σ1 , σ2 are

1  n2 (n2 − 1)T12 
σ1U = T1 − ,
n1 − 1 n1 (n1 (n1 − 1)T2 + n2 (n2 − 1)T1 )
1  n1 (n1 − 1)T22 
σ2U = T2 − ,
n2 − 1 n2 (n1 (n1 − 1)T2 + n2 (n2 − 1)T1 )
respectively.
(N. Jana) Classification into Two Parameter Exponential Populations with a Common Guarantee Time 5 / 15
 Estimation Under Restrictions on Scale Parameters Affine Equivariant Estimators

Affine Equivariant Estimators of σ1

Consider the estimation of σi with respect to the loss function

δ 2
i
L(σi , δi ) = −1 . (2)
σi
Affine group of transformations is GA = {ga,b : ga,b (x) = ax + b, a > 0, b ∈ R}. Under
the loss function (2), δi → aδi for i = 1, 2, and δ(aZ + b, aT1 , aT2 ) = aδ(Z, T1 , T2 ),
where a > 0. Therefore δ(Z, T1 , T2 ) = T1 φ(V), where V = T2 /T1 .
For a given function φ : (0, ∞) → R, define the function φ1 : (0, ∞) → R by

φ∗ (v) if φ(v) < φ∗ (v),

φ1 (v) = φ(v) if φ∗ (v) ≤ φ(v) ≤ φ∗ (v), (3)
 ∗
φ (v) if φ(v) > φ∗ (v),


where φ∗ (v) = 1
n1 +n2
and φ∗ (v) = 1+v
n1 +n2
.

Theorem 2

Let δφ (T) = T1 φ(V) be an affine equivariant estimator of σ1 and let δφ1 (T) = T1 φ1 (V), where
V = T2 /T1 and φ1 is given by (3). Suppose that Pθ (δφ1 (T) 6= δφ (T)) > 0, for some θ ∈ Θ.
Then the estimator δφ1 (T) improves upon the estimator δφ (T) under the loss function (2).

(N. Jana) Classification into Two Parameter Exponential Populations with a Common Guarantee Time 6 / 15
 Estimation Under Restrictions on Scale Parameters Affine Equivariant Estimators

Corollary 1
The affine equivariant estimator of σ1 improving over the UMVUE δ1U is δ1AU = T1 φ1 (V), where
n2 v+n2 (n2 −1)

1 1

 n1 +n2 2
 if n1 (n1 (n11 −1)v+n 2 (n2 −1))
< n1 +n 2
,
2
n1 v+n2 (n2 −1) 1 n v+n (n −1)
φ1 (v) = n (n (n −1)v+n (n −1)) if n +n
1 1 1 2 2 1 2
≤ 1 2 2
n1 (n1 (n1 −1)v+n2 (n2 −1))
≤ n1v+1
+n2
,
n21 v+n2 (n2 −1)


 v+1 v+1
n1 +n2
if n1 (n1 (n1 −1)v+n2 (n2 −1)) > n1 +n2 .

The affine equivariant estimator of σ1 improving the MLE is δ1AM = T1 φ1 (V), where
(
1 1
if n1 +n ≤ n11 ≤ n1v+1
+n2
,
φ1 (v) = n1v+1 1
2
v+1
n1 +n2
if n1
> n1 +n2
.

The affine equivariant estimator of σ1 improving the RMLE δ1RML is δ1ARM = T1 φ1 (V), where
  
1

 n1 +n 2
if min n11 , n1v+1
+n 2
1
< n1 +n ,

    2
1 v+1 1 1 1+v
φ1 (v) = min n1 , n1 +n2 if n1 +n2 ≤ min n1 , n1 +n2 ≤ n1v+1+n2
,

  
v+1 1 v+1 v+1
if min n1 , n1 +n2 > n1 +n2 ,


n1 +n2

where V = T2 /T1 .

(N. Jana) Classification into Two Parameter Exponential Populations with a Common Guarantee Time 7 / 15
 Estimation Under Restrictions on Scale Parameters Affine Equivariant Estimators

Comparison of the Estimators

The relative risk improvements (in percentage) of the estimators δiML of σi over the UMVUE δiU
is
 R(δ U , σ ) − R(δ ML , σ ) 
i i
RI(δiML , δiU ) = i i
× 100.
R(δiU , σi )

Table: Percentage risk improvement of estimators of σ1 for n1 = n2 = 4 and µ = 0.5

ρ RI(δ1RML , δ1U ) RI(δ1AU , δ1U ) RI(δ1SU , δ1U ) RI(δ1SM , δ1U ) RI(δ1SRM , δ1U )
0.1 22.0892 0.6950 0.0000 21.8895 22.0861
0.2 20.1179 5.0297 0.0000 18.6767 20.1114
0.3 20.5297 12.5316 0.0326 16.1783 20.5197
0.4 22.3012 20.3881 0.5098 14.3310 22.2876
0.5 24.4306 27.4322 2.3984 13.2090 24.4132
0.6 26.5843 33.1281 5.8294 13.2987 26.5632
0.7 28.4995 37.1004 10.5679 14.5283 28.4746
0.8 29.9755 39.6007 16.0542 16.6007 29.9467
0.9 30.9082 40.8484 21.8957 19.3141 30.8755
1.0 31.3496 41.1561 27.6256 22.4120 31.3130

To estimate the scale parameter σ1 , we have suggested the restricted MLE δ1RML for small
values of ρ and the estimator δ1AU for moderate and large values of ρ.
(N. Jana) Classification into Two Parameter Exponential Populations with a Common Guarantee Time 8 / 15
 Estimation Under Restrictions on Scale Parameters Mixed Estimators

Mixed Estimators

Define the mixed estimators when the ordering σ1 ≤ σ2 is known.

T T1 T2 
1
σ̃1α = min , α + (1 − α) , 0 ≤ α < 1, (4)
n1 n1 n2
T T2 T1 
2
σ̃2α = max , α + (1 − α) , 0 ≤ α < 1. (5)
n2 n2 n1
Note that when α = n1 /(n1 + n2 ), the estimator σ̃1α is the restricted MLE of σ1 . When
α = n2 /(n1 + n2 ), the estimator σ̃2α is the restricted MLE of σ2 .
Comparison of the risk of estimator δi of the parameter σi under the loss function
δ 2
i
L(σi , δi ) = −1 . (6)
σi

Theorem 3
Let σ̂i and σ̃iα be the MLE and the mixed estimators of σi as defined above. Suppose R(θ, σi∗ )
denotes the risk of the estimator σi∗ in estimating σi for i = 1, 2. Then for all 0 < σ1 ≤ σ2 ,
(i) R(θ, σ̃1α ) < R(θ, σ̂1 ), α ∈ [α1 , 1], where α1 = n1 /(n1 + n2 ), n1 ≥ 1, n2 ≥ 1,
(ii) R(θ, σ̃2α ) < R(θ, σ̂2 ), α ∈ [α2 , 1], where α2 = n2 /(n1 + n2 ), n1 ≥ 1, n2 ≥ 1,
(iii) For α ∈ [0, 1], and n1 = n2 = n, R(θ, σ̃1α ) ≤ R(θ, σˆ1 ) and R(θ, σ̃2α ) ≤ R(θ, σˆ2 ).
(N. Jana) Classification into Two Parameter Exponential Populations with a Common Guarantee Time 9 / 15
 The Problem of Classification

Classification Rule

qi : priori probability that an observation comes from population Πi ( ki=1 qi = 1)

P
P(i|j): probability of classifying x into Πi when it actually belongs to Πj
C(i|j): cost of misclassification where i 6= j and i, j = 1, 2, · · · , k

Expected probability of correct classification (EPC) = ki=1 qi P(i|i)

P
Pk Pk
Expected cost of misclassification (ECM) = i=1 qi j=1,j6=i P(j|i)C(j|i)
The probability density function associated with the population Πi is fi (x). When all
parameters are known, minimizing the ECM, Anderson (1951) derived the Bayes
classification rule:
fi (x) qj C(i|j)
Classify x into Πi if ≥ , j = 1, 2, · · · , k, j 6= i. (7)
fj (x) qi C(j|i)

Rules for two populations assuming C(1|2) = C(2|1) and q1 = q2 : When the parameters
are known, minimizing the ECM, the classification rule is: Classify the observation x into
Π1 if and only if U(x) ≤ 0, where
1 1 σ 
2
U(x) = (x − µ) − − log . (8)
σ1 σ2 σ1

(N. Jana) Classification into Two Parameter Exponential Populations with a Common Guarantee Time 10 / 15
 The Problem of Classification Classification Rules for Two Populations

Proposed Rules for Two Populations

When the parameters are unknown, plugging the MLEs, we propose the classification
rule, say, R1 : Classify the observation x into Π1 if and only if U1 (x) ≤ 0, where
n n2  n T 
1 2 1
U1 (x) = (x − Z) − − log . (9)
T1 T2 n1 T2

(α)
The rule based on the mixed estimators of the parameters σ1 , σ2 is given by R2 : Classify
(α)
the observation x into Π1 if and only if U2 (x) ≤ 0, where
 1 1   σ̃ 
(α) 2α
U2 (x) = (x − Z) − − log . (10)
σ̃1α σ̃2α σ̃1α
Plugging-in the estimators δ1gb , δ2gb and Z in (8), we have the classification rule R3 .
Similarly, the rule R4 is obtained by substituting the estimators δ1b , δ2b , Z for the parameters
σ1 , σ2 , µ respectively in (8).

Theorem 4
(0)
(i) The P(1|1) of rule R1 , is greater than the P(1|1) corresponding to the rule R2 .
(0)
(ii) The P(2|2) of rule R1 , is less than the P(2|2) corresponding to the rule R2 .
(0)
(iii) The ordered classification rule R2 is better than the rule R1 with respect to the EPC.
(N. Jana) Classification into Two Parameter Exponential Populations with a Common Guarantee Time 11 / 15
 The Problem of Classification Likelihood Ratio Based Rule

Likelihood Ratio Based Rule

Test the hypothesis H1 versus the alternative hypothesis H2

H1 : x, X11 , X12 , · · · , X1n1 belongs to Π1 and X21 , X22 , · · · , X2n2 belongs to Π2
H2 : X11 , X12 , · · · , X1n1 belongs to Π1 and x, X21 , X22 , · · · , X2n2 belongs to Π2 .

The likelihood ratio is given by

λ∗ = sup L1 /sup L2 , (11)

Ω Ω

where Ω = {(µ, σ1 , σ2 ) : µ ∈ R, 0 < σ1 , σ2 < ∞} and L1 , L2 are given by

n
−(n1 +1) − σ1 (n1 X̄1 +x−(n1 +1)µ)− σ2 (X̄2 −µ)
L1 = σ1 σ2−n2 e 1 2 ,
n
−(n2 +1) − σ1 (X̄1 −µ)− σ1 (n2 X̄2 +x−(n2 +1)µ)
L2 = σ1−n1 σ2 e 1 2

respectively.

(N. Jana) Classification into Two Parameter Exponential Populations with a Common Guarantee Time 12 / 15
 The Problem of Classification Likelihood Ratio Based Rule

Likelihood Ratio Based Rule (cont.)

Suppose r = (n1 + 1)−1 , s = (n2 + 1)−1 .

Under the hypothesis H1 , the MLEs of µ, σ1 and σ2 are µ̂ = min{x, X1(1) , X2(1) },
σˆ1 = (n1 (X¯1 − µ̂) + x − µ̂)r and σ̂2 = X̄2 − µ̂ respectively.

Under the hypothesis H2 , the MLEs of µ, σ1 and σ2 are µ̂ = min{x, X1(1) , X2(1) }
σ̂1 = X̄1 − µ̂ and σˆ2 = (n2 (X¯2 − µ̂) + x − µ̂)s respectively.

The LR rule, say, Rl1 is : Classify x into Π1 if λ∗ ≥ c else classify x into Π2 , where

(n1 + 1)n1 +1 (X̄1 − µ̂)n1 (n2 (X̄2 − µ̂) + x − µ̂)n2 +1

λ∗ = . (12)
(n2 + 1)n2 +1 (X̄2 − µ̂)n2 (n1 (X¯1 − µ̂) + x − µ̂)n1 +1

The cut-off point c should be chosen such that the EPC is high.

(N. Jana) Classification into Two Parameter Exponential Populations with a Common Guarantee Time 13 / 15
 The Problem of Classification Rules for k (> 2) Populations

Rules for k (> 2) Populations

Assume the prior probabilities are equal and the costs of the misclassifications are also
equal. When the parameters are known, the classification function related to the jth and
ith population is
fj (x) 1 1 σ 
i
uji (x) = log = (x − µ) − + log . (13)
fi (x) σi σj σj

Plugging in the MLEs µ̂m , σ̂i and σ̂j of the parameters µ, σi and σj respectively, the
estimated classification function is
n nj  n T 
i j i
ûji (x) = (x − µ̂m ) − + log . (14)
Ti Tj ni Tj
Rm : Classify the observation x into Πj if ûji (x) > 0 for i = 1, · · · , k, i 6= j.
Consider a simple order restriction 0 < σ1 ≤ σ2 ≤ · · · ≤ σk < ∞. Plugging in the
isotonized version estimators µ̂ir , σ̂ir , we get
 1 1   σ̂ 
ir
ũji (x) = (x − µ̂ir ) − + log . (15)
σ̂ir σ̂jr σ̂jr
Rir : Classify the observation x into Πj if ũji (x) > 0 for i = 1, · · · , k, i 6= j.
(N. Jana) Classification into Two Parameter Exponential Populations with a Common Guarantee Time 14 / 15
 Numerical Comparison and Example

1.0
R1

1.0
(0) R2
R2
( 1 4) R3

0.9
R2 R4
0.9
( 1 2)
R2
( 3 4)
R2

0.8
0.8

EPC
EPC

0.7
0.7

0.6
0.6

(µ,n1,n2) = (0,3,3) (n1,n2) = (4,4)

0.5
0.5

0.0 0.2 0.4 0.6 0.8 1.0 0.0 0.2 0.4 0.6 0.8 1.0

ρ ρ

(α)
Figure: (a). EPC of Rules R2 for different values of α; (b). Plot of the EPC corresponding to proposed rules vs ρ

Example: Operational times (in hours) between successive failure of air conditioning
equipments in two aircraft [Barlow et al. (1972)]:
Plane 7915 (n1 = 9): 359, 9, 12, 270, 603, 3, 104, 2, 438
Plane 8044 (n2 = 10): 487, 18, 100, 7, 98, 5, 85, 91, 43, 230
Classify the observation x = 130.
(0)
X = 2, T1 = 1782, T2 = 1144 and U2 (x) = −0.0761. Therefore the observation 130
(0)
belongs to Π1 according to rule R2 .
(N. Jana) Classification into Two Parameter Exponential Populations with a Common Guarantee Time 15 / 15
 References

References

Conde, D., M. A. Fernández, C. Rueda, & B. Salvador. (2012). Classification of samples into two or
more ordered populations with application to a cancer trial. Statistics in Medicine,
31(28):3773–3786.
Batsidis, A. (2012). Errors of misclassification in discrimination with data from truncated t
populations. Statistical Papers, 53(2):281–298.
Conde, D. and M.A. Fernandez and B. Salvador (2005). Classification rule for ordered exponential
populations. J. Statist. Plann. Inference, 135(2):339–356.
Fernandez, M. A. and C. Rueda and B. Salvador (2006). Incorporating additional information to
normal linear discriminant rules, J. Amer. Statist. Assoc. 101(474):569–577.
Long, T. and R. D. Gupta (1998). Alternative linear classification rules under order restrictions.
Comm. Statist. Theory Methods 27(3):559–575.
Figueiredo, A. (2009). Discriminant analysis for the von Mises-Fisher distribution. Comm. Statist.
Simulation Comput. 38(9):1991-2003.
Wakaki, H. and M. Aoshima (2009). Optimal discriminant functions for normal populations. J.
Multivariate Anal. 100(1):58–69
Figueiredo, A. and P. Gomes (2006). Discriminant analysis based on the Watson distribution defined
on the hypersphere. Statistics 40(5):435-445.
Chung, H. C., and C. P. Han (2009). Conditional confidence intervals for classification error rate.
Comput.
(N. Stat.
Jana) Data Anal., 53(12):4358–4369.
Classification into Two Parameter Exponential Populations with a Common Guarantee Time 15 / 15
 Publications

N. Jana, S. Kumar & N. Misra (2016). Classification rules for two parameter exponential populations
under order restrictions on parameters. J. Stat. Comput. Simul. 86(8):1559–1581. Taylor & Francis

N. Jana, S. Kumar & K. Chatterjee (2016). Bayes estimation for exponential distributions with
common location parameter and applications to multistate reliability models. J. Appl. Stat.
43(15):2697–2712 Taylor & Francis

N. Jana & S. Kumar (2016). Classification into two parameter exponential populations with a
common guarantee time. Amer. J. Math. Management Sci. 35(1):36–54. Taylor & Francis

N. Jana & S. Kumar (2017). Classification into two normal populations with a common mean and
unequal variances. Commun. Statist. Simulation Comput. 46(1):546–558. Taylor & Francis

Q. Hong, N. Jana, S. Kumar & K. Chatterjee (2017). Stress-strength models with more than two
states under exponential distribution. Commun. Statist. Theory Methods 46(1):120–132. Taylor &
Francis
N. Jana & S. Kumar (2015). Estimation of ordered scale parameters of two exponential distributions
with a common guarantee time. Math. Methods Statist. 24:122–134. Springer

N. Jana, S. Kumar & N. Misra (2014). Classification rules for exponential populations under order
restrictions on parameters. Springer Proc. Math. Stat. 91:243–250. Springer

S. Basu, N. Jana, Mahadevappa M., J. Mukherjee, S. Kumar (2016). Emotion recognition based on
physiological signals using valence-arousal model. IEEE Conference Publications pp.50-55

N. Jana, S. Kumar & K. Chatterjee (2017). Inference on stress-strength reliability for exponential
distributions with a common scale parameter. Revision submitted to Metrika. Springer
(N. Jana) Classification into Two Parameter Exponential Populations with a Common Guarantee Time 15 / 15
 Presented Papers in International Conferences

Classification procedures under the restrictions on parameters. VIII International Symposium on

Statistics and Optimization in conjunction with XXXVI-Annual Convention of ISPS, AMU, 18-20
Dec-2016 (Received Best Ph.D. Thesis Award in Statistics by ISPS)

Ordered classification rules into several exponential populations. Interdisciplinary Statistical

Research Unit, Indian Statistical Institute Kolkata, 7-June 2017

Order constrained classification into inverse gaussian populations. IMS International Conference
on Statistics and Probability, Kunming, China, 1-4 July-2015

Estimation of ordered scale parameters of two exponential distributions with a common guarantee
time. IMSCT FIM-XXIII 18-20 Dec-2014, NIT Karnataka (Received Best Paper Award)

Bayes estimation for exponential distributions with common location parameter and applications to
multi-state reliability models. AMISTAT-2015, November 10-13, Prague, Czech Republic

Classification into two parameter exponential populations with a common guarantee time. XVII
SSCA, 23-25 Feb-2015, Bhubaneswar (Selected for Young Scientists Award Competition)

Classification rules for exponential populations under order restrictions on parameters. ICMC 27-29
Dec-2013, HIT Haldia

Classification into two normal populations with a common mean and unequal variances.
OPTIMA-2012, 29 Nov-1 Dec, University of Delhi

Estimating linear functions of scale parameters and reciprocals of scale parameters of two gamma
populations under ordering. Research Scholar Day-2015, 13-14 Feb, IIT Kharagpur

(N. Jana) Classification into Two Parameter Exponential Populations with a Common Guarantee Time 15 / 15
 Thank You

(N. Jana) Classification into Two Parameter Exponential Populations with a Common Guarantee Time 15 / 15