ISSN: 0976-9102(ONLINE) ICTACT JOURNAL ON IMAGE AND VIDEO PROCESSING, FEBRUARY 2014, VOLUME: 04, ISSUE: 03

LOCAL TEXTURE DESCRIPTION FRAMEWORK FOR TEXTURE BASED FACE

RECOGNITION

R. Reena Rose1, A. Suruliandi 2 and K. Meena3

1
Department of Computer Applications, St. Xavier’s Catholic College of Engineering, India
E-mail: [email protected]
2
Department of Computer Science and Engineering, Manonmaniam Sundaranar University, India
E-mail: [email protected]
3
Department of Electronics and Communication Engineering, J. P. College of Engineering, India
E-mail: [email protected]

Abstract holistic information of face images is not effective under

Texture descriptors have an important role in recognizing face illumination variation, facial expression and partial occlusion
images. However, almost all the existing local texture descriptors use [9]. Feature based techniques such as shape based and texture
nearest neighbors to encode a texture pattern around a pixel. But in based are robust to variations in head orientation, scale and
face images, most of the pixels have similar characteristics with that
location of face in the image. But they are computationally more
of its nearest neighbors because the skin covers large area in a face
and the skin tone at neighboring regions are same. Therefore this expensive than holistic approaches [28, 35]. Hence an effort is
paper presents a general framework called Local Texture Description made in this paper to introduce a new texture description
Framework that uses only eight pixels which are at certain distance framework that produces better accuracy in face recognition with
apart either circular or elliptical from the referenced pixel. Local less computation burden.
texture description can be done using the foundation of any existing
local texture descriptors. In this paper, the performance of the 1.1 MOTIVATION AND JUSTIFICATION FOR
proposed framework is verified with three existing local texture THE PROPOSED APPROACH
descriptors Local Binary Pattern (LBP), Local Texture Pattern (LTP)
and Local Tetra Patterns (LTrPs) for the five issues viz. facial Textural feature extraction methods have an important role in
expression, partial occlusion, illumination variation, pose variation recognizing objects and scenes. They can be used to determine
and general recognition. Five benchmark databases JAFFE, Essex, uniformity, lightness, density, fineness, coarseness, roughness,
Indian faces, AT&T and Georgia Tech are used for the experiments. regularity, etc., of texture patterns as a whole [2, 21]. An ample
Experimental results demonstrate that even with less number of
number of methods have been proposed to extract facial texture
patterns, the proposed framework could achieve higher recognition
accuracy than that of their base models. features. There exist many models that are derived from Local
Binary Pattern (LBP). Ojala et al. [22] developed LBP in which
Keywords: a gray scale invariant texture pattern for a local neighborhood of
Face Recognition, Local Texture Description Framework, Nearest 3  3 is defined. A derivative of the LBP [23] was later
Neighborhood Classification, Chi-Square Distance Metric introduced by them to describe multi-resolution rotational and
gray scale invariant pattern on a circular neighborhood that
1. INTRODUCTION could represent salient micro-patterns of face images. It is a
very powerful method to analyze textures [28, 29]. Hence,
Face images are one among few biometric information that several researchers have been successfully applied it for face
can be captured from a distance without the knowledge of the recognition [1], facial expression analysis [37], background
subject. Moreover in some legal databases, face images are the modeling [10] etc. Owing to its low dimensionality and
only available biometric information. Hence face recognition efficiency in face recognition and texture classification, several
systems are the widely accepted means of extracting information variants of LBP was later introduced.
for visual surveillance, biometric authentication, computer- Advanced Local Binary Pattern (ALBP), which is an
human interaction etc. There exist several techniques for face extension of LBP, was invented by Liao et al. [17] to capture
recognition. However issues like complex lighting, expression micro information in face images. Later they introduced
variation, partial occlusion with objects, pose variation with Dominant Local Binary Pattern (DLBP) [18] for texture
large rotation angles and aging remain unsolved. Therefore, to classification which was extended to face recognition and
improve the effectiveness of today’s recognition systems, there detection. Suruliandi and Ramar [26] proposed a univariate
arises a need to enhance the systems performance. texture model called Local Texture Patterns (LTP) for image
Based on the property of the features extracted, face classification and proved that it is robust in terms of gray scale
recognition algorithms are classified into holistic and local variation and rotational variation. Heikkila et al. [11] introduced
feature based [6]. Holistic approaches such as Principal Center Symmetric Local Binary Pattern (CSLBP) that produces
Component Analysis (PCA) [15], Linear Discriminant Analysis 16 patterns. This descriptor is robust in flat areas and has
(LDA) [3], variants of LDA [4, 14, 31, 33, 34, and 38], Marginal tolerance against illumination changes. Zhang et al. [36]
Fisher Analysis (MFA) [32], Eigen feature Regularization and proposed Local Derivative Patterns (LDPs) in which LBP is
Extraction (ERE) [13] were extensively studied due to their good considered as a non-directional first-order local patterns
performance and low computational complexity. Even then, collected from the first-order derivatives and have extended it

773
R REENA ROSE et.al: LOCAL TEXTURE DESCRIPTION FRAMEWORK FOR TEXTURE BASED FACE RECOGNITION

for nth order LDPs. This operator has been successfully applied 1.3 ORGANIZATION OF THE PAPER
for face recognition. Guo et al. [8] proposed Local Binary
Pattern Variance (LBPV) that characterizes local contrast The latter part of the paper is organized as follows. A brief
information into a one-dimensional LBP histogram. Lei et al. review of texture descriptors LBP, LTP, and LTrPs are reported
[16] introduced a method that merges information obtained from in section 2. In section 3 the proposed LTDF is presented.
image space, scale and orientation and have proved that in face Section 4 gives the face recognition algorithm in detail.
recognition their method outperforms the one which considers Section 5 is devoted to the experimental results and discussions
the individual domain alone. In our early work [27], of the proposed LTDF model for five different conditions
performance of LBP, Multivariate Local Binary Pattern (MLBP) expression variation, illumination variation, partial occlusion
[19], LBPV, DLBP, Local Texture Pattern (LTP) and LDP are with spectacle, pose variation and general recognition. Finally,
evaluated for different face recognition issues, and have found the conclusion is given in section 6.
that LTP and LDP outperforms other descriptors.
Subrahmanyam Murala et al. [25] proposed Local Tetra Patterns 2. RELATED WORK
(LTrPs) for content based image retrieval and have proved that
their method has high discrimination power. 2.1 TEXTURE DESCRIPTION
Except Multi-scale Local Binary Pattern Histogram
(MLBPH) [5], almost all the existing local texture descriptors Texture is a term that characterizes the contextual property of
describe patterns by relating the closest neighbors around a an image. A texture descriptor can characterize an image as a
pixel. When pixels at certain distance apart are considered, it whole. Texture descriptor Grey Level Co-occurrence Matrix
may likely to acquire the features of different facial components (GLCM) [21] belongs to this category. Alternatively, it can also
like eyes, nose, mouth etc. This is the idea behind developing a characterize an image locally at the micro level and by global
general framework for describing a texture pattern over a local texture description at the macro level. In local description, the
region with pixels at certain distance apart. Both the face and relationship between a pixel and its neighborhood can be
the components of the face can be either circular or elliptical in expressed in terms of local texture patterns. The occurrence
nature. Hence proposing a new texture description that captures frequency of such patterns (PTN) will be collected in a
features along circular or elliptical neighborhood is expected to histogram (H) using (1) which describes the global feature of the
have high discrimination power even when all the face image. The texture descriptors LBP, LTP, and LTrPs follow the
recognition challenges are considered. Justified by these facts, a second approach.
framework LTDF is proposed for either circular or elliptical N M
neighborhood. H p    f PTN i, j , p ; (1)
i 1 j 1
1.2 OUTLINE OF THE PROPOSED APPROACH
p  1, P (2)
Overall process of face recognition is illustrated by Fig.1. At
1 if x  y
f  x, y   
first, all the images are converted into gray-scale images. Then
they are preprocessed to align into same canonical position. (3)
0 otherwise
Subsequently certain region of interest is cropped from the
images so as to prevent processing of unnecessary details. The where, N  M represents the size of the input image and P the
system is then trained by extracting texture features from gallery total number of patterns.
images by the proposed LTDF, and is stored separately for every
image in the database. While testing a probe image, texture 2.2 LOCAL BINARY PATTERN
features are extracted from that image, and are matched against
all the images in the database using nearest neighborhood Ojala et al. introduced LBP operator [23] for texture
classifier with chi-square dissimilarity metric. classification by which a texture pattern around a pixel in an
image can be computed by comparing its gray value with its
neighbors as demonstrated in Fig.2.

Gallery Images Nearest

Neighborhood
LTDF feature Feature Classifier
Preprocessing
extraction DB
Probe Image Texture feature
matching using Output
LTDF feature
Preprocessing chi-square
extraction

Fig.1. Face recognition process

774
ISSN: 0976-9102(ONLINE) ICTACT JOURNAL ON IMAGE AND VIDEO PROCESSING, FEBRUARY 2014, VOLUME: 04, ISSUE: 03

U is the uniformity measure and is computed as in Eq.(6).

20 90 199 Threshold = 21 0 1 1 Number of uniform patterns which can be obtained by LTP
scheme is 46 in total and can have values in the range 0 to 73
250 21 12 1 0 leaving few holes in between. Therefore patterns are relabeled to
18 9 212 0 0 1 form continuous numbers from 1 to 46.

Pattern String: 10010110

2.4 LOCAL TETRA PATTERN

Fig.2. Pattern string computation from a sample image Subrahmanyam et al. introduced this model in which a local
texture description of a pixel can be obtained using two things:
1) the direction of the pixel with its horizontal and vertical
LBP is deliberated as follows,
neighbors. 2) The magnitudes of horizontal and vertical first-
 p1t g p , g c 
 p 8 order derivatives. For every direction, a tetra pattern is first
 if U  2
LBPP, R   (4) obtained using Eq.(14) which is further divided into three binary
 9 otherwise pattern using Eq.(16). Therefore the total number of binary
pattern that an LTrPs can give is 13 including the magnitude
where, information. The detailed explanation with example is available
0 if g p  g c in [25].
 
t g p ,g c   (5) Given an image I, the first-order derivatives at a pixel gc for
1 if g p  g c direction one can be calculated as,
In the above equations gc, gp are the grey level of center pixel I 1 0 o  g c   I  g h   I g c  (10)
c and a vicinity pixel p respectively, P is the number of
neighbors and R is the radius of the neighborhood. The pattern I 190o g c   I g v   I g c  (11)
can be classified as either uniform or non-uniform. It is said to
where, gh and gv are the grey values of horizontal and vertical
be uniform if in the pattern string there are at most only two
neighbors. The direction of the center pixel can be written as,
transitions from 0 to 1 or vice versa. Usage of uniform patterns
reduces the total number of bins required. Image analysis 1, I 1 0 o g c   0 and I 1 90 o g c   0
requires only 9 bins for uniform patterns and one extra bin for all 
non-uniform patterns thus requires a total of 10 bins. The 2,
 I 1 0 o g c   0 and I 1 90 o g c   0
uniformity measure U is computed as follows, I 1Dir g c    (12)
3, I 1 0 o g c   0 and I 1 90 o g c   0
U  st g8 , g c , t g1, g c   i2 st gi , gc , t gi1, gc  
8
(6)
4, I 1 0 o g c   0 and I 1 90 o g c   0
where, The second-order LTrP2(gc) is expressed as,

1 if x  y  0
sx,y   (7)   
LTrP 2 g c   f I 1Dir g c , I 1Dir g1  , f I 1Dir g c , I 1Dir g 2  ,... 

0 otherwise f I Dir g , I Dir g | P  8
1
c
1
P

2.3 LOCAL TEXTURE PATTERN (13)

where,
Suruliandi and Ramar [26] have proposed LTP. In this,
0, I 1Dir g c   I 1Dir g P 
texture pattern around a pixel in an image is computed with the
pattern units P obtained for its eight neighbors as described  
f I 1Dir g c , I 1Dir g P     (14)
below.  I 1Dir g P , else

0 if g i  g c  Δg  Above equations yield an eight bit tetra pattern for every

 pixel. The tetra pattern is then converted into four parts
P g i , g c   1 if g c  Δg   g i  g c  Δg , i  1,2 ,...8 (8) depending upon the direction of center pixel. Each tetra pattern
 is converted to three binary patterns.
9 if g i  g c  Δg 
For direction “1” LTrP2 of any pixel can be segregated into
In the above equation, gc is the gray value of center pixel and three binary patterns using,

 p1 BP LTrP 2 g c  Direction2,3,4

gi is the gray value of 3  3 neighbors and ∆g is a small positive P
integer value that plays an important role in forming the uniform LTrP 2  (15)
Direction 2,3, 4
patterns. A pattern string is then formed by collecting the P
values of the eight neighbors starting from any position as 1, if LTrP 2 g c   
described below. 
BP LTrP g c 2

Direction

 (16)
 0, else
i1 Pgi , g c 
8
 if U  3
LTP   (9) where,  = 2,3,4.
 73 otherwise Similarly, tetra pattern for the other three directions of every
pixel are converted into binary patterns. Thus the total numbers

775
R REENA ROSE et.al: LOCAL TEXTURE DESCRIPTION FRAMEWORK FOR TEXTURE BASED FACE RECOGNITION

of binary patterns are 12 (4  3). The 13th binary pattern (LP) is where,
obtained by using the magnitudes of horizontal and vertical first-
order derivatives using, hr 2  vr 2
Ri  (20)
hr 2 sin 2  i  vr 2 cos 2  i
M I 1 g
p  I 0 g   I 90 g 
1 o
p
2 1 o
p
2
(17)
and

 p1t M I g   M I g   P8  360 

P
LP  1
p
1
c
(18) i   * i  1, i  1 to p (21)
 p 
In LTDFm, local pattern descriptions are made separately for
3. PROPOSED LOCAL TEXTURE each ring rj (j = 1 to n) and are then concatenated for n rings
DESCRIPTION FRAMEWORK (LTDF) which are several distances apart from the center pixel that
represent the texture as a whole. For a circular ring r1, which is
Local Texture Description Framework is a framework for closer to the center pixel, d is the radius of the ring from the
any texture descriptors that use nearest neighbors to describe a center pixel whereas for the other rings r2 to rn, it is the
texture pattern around a pixel in an image. It is either circular or difference in radius of the ring from its predecessor. For every
elliptical in shape and can have several rings that encircle a pixel in a ring, the corresponding pixel in the successor ring lies
center point as illustrated in Fig.3. LTDF can be described with in the same direction as in Fig.3(d).
a single ring (LTDFs) or with combination of more number of Once the neighborhood pixels on a ring are located, any local
rings (LTDFm). texture descriptors such as LBP, LTP, LTrPs etc. that encode
images using eight neighbors can be used to obtain local texture
description. For LTDFm, each pixel is represented by n pattern
values that correspond to n number of rings considered. LTDFcs
using LBP is similar as MLBPH [5], and LTDFcm is similar to
DAISY [28] descriptor when d is 1 for all the rings. Hence these
two approaches can be viewed as a subset of the proposed
framework and the proposed LTDFcm can also be referred as
(a) (b) (c) loosely coupled DAISY.

4. FACE RECOGNITION ALGORITHM

Input: Probe Image
Output: Recognized image
Training Phase:
For every image in the gallery, do the following,
a) Rotate the images in such a way that a line connecting
eye centers lies on a horizontal line.
b) Crop certain region from the image to avoid
computational complexity of using entire face by Eq.(22)
– Eq.(25).
(d) x1  x  p    p 2 (22)
Fig.3. Position of neighborhood pixels in LTDF for shapes y1  y  p (23)
a) circular b) horizontal ellipse c) vertical ellipse and
x2  3 * p (24)
d) Illustration of parameters in LTDF
y 2  3 * p   p (25)
For a pixel in an image, LTDFs is a joint distribution of
where, p is half the distance between two eye centers.
p(p=8) pixels that lie on a ring at radius d from that pixel. The
shape of the ring can be either circular (LTDFcs) or elliptical c) Compute local texture description for every pixel in the
(LTDFes). LTDFes has two radii: hr (horizontal radius) and vr image using LTDF and store it in a feature space.
(vertical radius). All the parameters used in this model are d) Divide the feature space into n  n equally sized sub
illustrated in Fig.3(d). The pixels considered from the ring are in regions.
eight directions (θ) from the center pixel. Position (gix,giy) of a e) Obtain global texture description in the form of histogram
vicinity pixel i in a ring is computed using Eq.(19) – Eq.(21). In as explained in section 3 for every sub region,
order to determine the coordinates of the pixels, bilinear concatenate them and store in a database. Fig.4 illustrates
interpolation can be used. For LTDFcs hr and vr are the same the feature extraction process.
(d=hr=vr).
g ix  Ri . cos i , g iy  Ri . sin  i (19)

776
ISSN: 0976-9102(ONLINE) ICTACT JOURNAL ON IMAGE AND VIDEO PROCESSING, FEBRUARY 2014, VOLUME: 04, ISSUE: 03

LTDFm using LBP alone. Fig.5 shows the LTDF coded faces of
a sample subject from AT&T database for the local texture
descriptors LBP, LTP and LTrPs. Sample images used from
different databases are displayed in Fig.6.

Fig.4. Illustration of the steps involved in computing texture

feature of an image

Testing Phase:
For a probe image, do the following,
a) Determine global texture description of the image using
steps a to e in the training phase. Fig.5. Feature maps obtained for an image from AT&T database
b) Find out the dissimilarity between the texture feature of
the probe image and texture feature of the gallery images
stored in the database using Chi-square statistic as
defined below,
H G i   H P i 2
i 1
n2m
 2 H G , H P   (26)
H G i   H P i 
where, HG(i) is the ith feature value of the gallery image
and HP(i) is the ith feature value of the probe image, m is
the number of patterns and n2 is the number of sub
(a) Sample images used for expression variation experiment
regions.
from JAFFE database
c) The gallery image which yields least dissimilarity
measure with the probe image is considered as the
recognized one.

5. EXPERIMENTAL RESULTS AND

DISCUSSIONS
In order to determine the effectiveness and feasibility of the
proposed framework in different issues including expression (b) Sample images used for illumination variation and partial
variation, illumination variation, general recognition, partial occlusion from ESSEX database
occlusion and pose variation, the framework is experimented
with local texture descriptors such as LBP, LTP and LTrPs.
Initially, an extensive experimental investigation is carried out
for single ring and multiple rings in circular shape.
Experimental results reveal that the single ring is sufficient in
most of the cases. Hence in the elliptical shape only single ring
is considered for further experimentation.
(c) Sample images used for pose variation experiment from
The following experimental setup and parameter settings are Indian Faces database
used. If any part of a ring falls outside the boundary then the ring
is neglected. For LTP, ∆g is assigned to have value 5. In
original LTrPs, positional weights (2P) are used but in this paper
it is not used in order to reduce the total number of patterns.
Further, for LTrPs the direction “1” alone is considered. The
input image is cropped and divided into 7 × 7 sub regions on
(d) Sample images used for general recognition experiment from
applying the algorithm in section 4. For all the experiments, the
AT&T database
recognition rate obtained for the best LTDF is tabulated and the
radius is given inside parenthesis. Owing to the usage of very Fig.6. Images used for different experiments from various
less number of patterns by LBP, experiments are carried out for databases

777
R REENA ROSE et.al: LOCAL TEXTURE DESCRIPTION FRAMEWORK FOR TEXTURE BASED FACE RECOGNITION

5.1 RESULTS ON EXPRESSION VARIATION comparison with the tested base models are reported in
Table.1(b) in terms of mean and standard deviation error. In 6-
Robustness in face recognition under different facial class face recognition, neutral expression images are not
expression is the most challenging issue. Facial expressions included whereas in 7-classs face recognition, all the expressions
result in temporally deformed facial features that lead to false are considered.
recognition. In order to test the effectiveness of the proposed
The results are evident for the effectiveness of the proposed
model, experiment is conducted for expression variation images
framework in recognizing faces with different expressions. The
by JAFFE database [20]. The database contains 213 frontal face
performance of the base models are enhanced when the proposed
images of 10 Japanese female models with seven different
framework is applied on them. The reason behind this might be
expressions. Two types of experiments are conducted on this
that the expression variations affect local regions and so when
database: one to recognize faces with varying expression and
pixels lie at certain region apart are used to form a texture
another to understand facial expression.
pattern, it has high discrimination power.
Initially the performance of the proposed LTDFc is
To analyze the ability of LTDFs on recognizing different
experimented for the standard LBP, by varying the number of
expressions, the confusion matrices are obtained for the base
rings and the difference in radius between the rings. Experiment
models, LTDFcs and LTDFes and are given in Table.1(c),
is conducted by setting one neutral expression image per subject
Table.1(d) & Table.1(e) respectively.
in the gallery set and the rest of the images in the probe set. The
results are tabulated in Table.1(a). From the results it is evident It is observed from the table results that both LTDFcs and
that the LTDFcs is capable of achieving a recognition accuracy of LTDFes outperform their base models to identify facial
94.08% which is greater than that of LTDFcm with three rings expression. Moreover LTDFes performs better in distinguishing
which yields an accuracy of 93.59% when d is 3. Higher the different expressions especially fear, happiness and surprise.
number of rings, greater being the total number of patterns
represented by the model. This shows the effectiveness of Table.1(b). Recognition rate (%) on the JAFFE database for
LTDFcs model. several methods

Table.1(a). Performance evaluation of LTDFcm (loosely coupled Methods 6-Class 7-Class

DAISY) using LBP on JAFFE database LBP 89.82±9.21 89.73±11.17
Recognition rate (%) LTP 96.91±4.28 97.68±4.44
1
 
No: LTrPs 96.16±5.69 97.63±3.35
2 3 4 5 6
Rings LTDFcs s
LTDFc _LBP(6) 100 100
Bins 10 20 30 40 50 60 s
LTDFc _LTP(6) 100 100
1
Distance between rings (d)

35.46 41.87 47.78 56.15 63.14 72.90 s

LTDFc _LTrPs(5) 100 100
(DAISY)
s
2 46.79 65.02 86.69 92.11 93.10 90.64 LTDFe _LBP(5,7) 100 100
3 71.92 90.64 93.59 90.64 91.13 90.64 LTDFes _LTP(5,8) 100 100
s
4 87.19 93.10 90.14 91.62 90.14 90.14 LTDFe _LTrPs(2,4) 100 100
5 92.61 91.62 91.13 91.62 90.64 88.17
5.2 RESULTS ON ILLUMINATION VARIATION
6 94.08 90.64 91.62 91.13 90.14 86.69
Recognition under different lighting condition is a
7 91.13 89.16 91.13 89.16 85.71 83.74 challenging problem in computer vision. This variation in
To further evaluate the generalization performance of LTDFs illumination affects the classification greatly. The performance
on novel subjects, a 10-fold cross-validation experiment scheme of the proposed LTDF is evaluated by conducting an experiment
is adopted. The dataset is randomly partitioned into ten groups. on images with different lighting condition. Frontal images of
Roughly equal number of subjects is kept in each group. Nine 27 persons with controlled illumination variation are taken from
groups are used as gallery set and the remaining group is used as Essex database [7] and in this paper the set is named as Essex-
probe set. This process is repeated for each group to have 10 illu. One exemplar per subject is kept in the gallery and 9 images
runs in total. The recognition rate for the best LTDFs in per individual are kept in the probe set. Experimental results are
given in Table.2.

Table.1(c). Confusion Matrix of 7-Class Facial Expression Recognition using LBP, LTP and LTrPs on JAFFE database

(%) Anger Disgust Fear Happiness Sadness Surprise Neutral

LBP 56.66 6.66 6.66 0 20 0 10
Anger LTP 60 10 13.33 0 6.66 0 10
LTrPs 73.33 6.66 6.66 0 10 3.33 0

778
ISSN: 0976-9102(ONLINE) ICTACT JOURNAL ON IMAGE AND VIDEO PROCESSING, FEBRUARY 2014, VOLUME: 04, ISSUE: 03

LBP 10.34 51.72 24.13 3.44 6.89 0 3.44

Disgust LTP 20.68 65.51 10.34 0 0 0 3.44
LTrPs 0 89.65 10.34 0 0 0 0
LBP 6.25 9.37 43.75 3.12 18.75 18.75 0
Fear LTP 3.12 9.37 75 0 0 6.25 6.25
LTrPs 0 6.25 78.12 0 0 9.37 6.25
LBP 0 0 0 80.64 6.45 3.22 9.67
Happiness LTP 0 0 3.22 74.19 6.45 16.12 0
LTrPs 0 3.22 0 74.19 6.45 0 16.12
LBP 3.22 3.22 6.45 3.22 67.74 3.22 12.90
Sadness LTP 3.22 0 6.45 3.22 70.96 3.22 12.90
LTrPs 3.22 3.22 3.22 3.22 77.14 3.22 6.45
LBP 6.66 0 13.33 3.33 3.33 66.66 6.66
Surprise LTP 3.33 0 10 6.66 3.33 73.33 3.33
LTrPs 3.33 0 3.33 3.33 3.33 73.33 13.33
LBP 6.66 0 13.33 3.33 6.66 3.33 66.66
Neutral LTP 0 6.66 3.33 0 13.33 6.66 70
LTrPs 0 0 0 0 3.33 3.33 93.33

Table.1(d). Confusion Matrix for Facial Expression Recognition obtained by LTDFcs for JAFFE database

(%) Anger Disgust Fear Happiness Sadness Surprise Neutral

LBP(6) 93.33 0 0 0 0 0 6.66
Anger LTP(6) 96.66 0 0 0 0 0 3.33
LTrPs(5) 96.66 0 0 0 3.33 0 0
LBP(6) 0 93.10 3.44 0 3.44 0 0
Disgust LTP(6) 0 96.55 0 0 3.44 0 0
LTrPs(5) 3.44 89.65 6.89 0 0 0 0
LBP(6) 0 0 96.87 0 0 3.12 0
Fear LTP(6) 0 0 96.87 0 0 3.12 0
LTrPs(5) 0 6.25 90.62 0 0 0 3.12
LBP(6) 0 0 0 83.87 9.67 0 6.45
Happiness LTP(6) 0 0 0 87.09 6.45 6.45 0
LTrPs(5) 0 0 0 83.87 3.22 3.22 9.67
LBP(6) 0 0 3.22 3.22 90.32 0 3.22
Sadness LTP(6) 0 3.22 0 3.22 90.32 0 3.22
LTrPs(5) 3.22 0 3.22 3.22 90.32 0 0
LBP(6) 0 0 3.33 3.33 0 93.33 0
Surprise LTP(6) 0 0 0 3.33 0 96.66 0
LTrPs(5) 0 0 0 10 3.33 76.66 10
LBP(6) 0 0 0 0 0 0 100
Neutral LTP(6) 0 0 0 0 0 0 100
LTrPs(5) 0 0 0 0 3.33 3.33 93.33

779
R REENA ROSE et.al: LOCAL TEXTURE DESCRIPTION FRAMEWORK FOR TEXTURE BASED FACE RECOGNITION

Table.1(e). Confusion Matrix for Facial Expression Recognition using LTDFes on JAFFE database

(%) Anger Disgust Fear Happiness Sadness Surprise Neutral

LBP(5,7) 96.66 0 0 0 0 0 3.33
Anger LTP(5,8) 96.66 0 0 0 0 0 3.33
LTrPs(2,4) 96.66 0 0 0 3.33 0 0
LBP(5,7) 0 93.10 3.44 0 0 0 3.44
Disgust LTP(5,8) 0 96.55 0 0 3.44 0 0
LTrPs(2,4) 0 89.65 10.34 0 0 0 0
LBP(5,7) 0 0 100 0 0 3.12 0
Fear LTP(5,8) 0 0 96.87 0 0 3.12 0
LTrPs(2,4) 0 3.12 96.87 0 0 0 0
LBP(5,7) 0 0 0 87.09 6.45 6.45 0
Happiness LTP(5,8) 0 0 0 90.32 3.22 0 6.45
LTrPs(2,4) 0 0 0 83.87 6.45 3.22 6.45
LBP(5,7) 3.22 0 0 3.22 90.32 0 3.22
Sadness LTP(5,8) 0 0 0 3.22 93.54 0 3.22
LTrPs(2,4) 3.22 3.22 3.22 3.22 80.64 0 6.45
LBP(5,7) 0 0 0 3.33 0 96.66 0
Surprise LTP(5,8) 0 0 0 3.33 0 96.66 0
LTrPs(2,4) 0 0 0 6.66 3.33 83.33 6.66
LBP(5,7) 0 0 0 0 0 0 100
Neutral LTP(5,8) 0 0 0 0 0 0 100
LTrPs(2,4) 0 0 0 0 3.33 3.33 93.33

As observed, LTDFcs using LBP is sufficient to attain a

Table.2(a). Performance evaluation of LTDFcm (loosely coupled maximum recognition accuracy of 98.35%. LTDFcs and LTDFcm
DAISY) using LBP for illumination variation produce similar results and so the system achieves high speed
when LTDFcs is used.
Recognition rate (%)
The best recognition rates obtained by the proposed LTDFs,
1
 
No: in comparison with the experimented base models are given in
2 3 4 5 6
Rings LTDFcs Table.2(b), in which LTDFs performs better than the base
Bins 10 20 30 40 50 60 models in face recognition with different lighting conditions.
Most of the pixels in face images lie on skin areas and are
1 similar in closest neighborhood. This causes the proposed
Distance between rings (d)

77.77 84.77 90.94 93 95.88 97.11

(DAISY) LTDF to gain high recognition rate by relating pixels that are at
2 88.06 95.06 97.11 97.11 97.53 97.11 certain distance apart.
3 95.06 97.53 97.94 97.53 97.94 97.94 5.3 RESULTS ON POSE VARIATION
4 97.94 97.94 97.53 97.94 97.11 95.88
In this section, effectiveness of the proposed LTDF on pose
5 98.35 97.94 97.94 97.53 95.06 95.06 variant images is evaluated with Indian Face database [30]. The
6 98.35 97.94 97.94 96.29 95.06 95.06 database constitutes two main directories separately for male and
female. There are eleven different images of 22 distinct females
7 97.94 98.35 97.11 95.47 95.06 95.06 in female directory. For every subject, there are seven images in
different poses. All the 22 female subjects’ images are used for
the experiment by placing one frontal pose image per subject in
the gallery set and the remaining subjects in the probe set.

780
ISSN: 0976-9102(ONLINE) ICTACT JOURNAL ON IMAGE AND VIDEO PROCESSING, FEBRUARY 2014, VOLUME: 04, ISSUE: 03

Table.2(b). Recognition rate (%) on the Essex-illu dataset for Table.3(b). Recognition rate (%) on the Indian Face database for
one sample training problem on several methods one sample training problem (one frontal image per subject) on
several methods
Recognition
Methods
accuracy Recognition
Methods
accuracy
LBP 77.77
LBP 19.69
LTP 93.82
LTP 36.36
LTrPs 93.41
LTrPs 29.54
LTDFcs _LBP(5) 98.35
s
LTDFc _LBP(6) 36.36
LTDFcs _LTP(5) 100
s
LTDFc _LTP(6) 47.72
LTDFcs _LTrPs(3) 97.53
s
LTDFc _LTrPs(6) 43.18
LTDFes _LBP(4,8) 98.35
s
LTDFe _LBP(9,7) 42.42
LTDFes _LTP(2,6) 100
s
LTDFe _LTP(5,1) 49.24
LTDFes _LTrPs(2,5) 97.53
s
It can be understood from Table.3 that the proposed LTDF is LTDFe _LTrPs(4,5) 45.45
more suited for pose variant images. For instant, the proposed
LTDFcs using LBP yields an accuracy of 36.36% whereas the
5.4 RESULTS ON PARTIAL OCCLUSION WITH
base model LBP gives an accuracy of 19.69%. This shows that OBJECTS
the proposed model performs better producing an accuracy of Occlusions appear as local distortion away from a common
17% greater than that of LBP and hence it has higher face representing human population [9]. In order to study the
discrimination power. capability of the model for recognizing faces occluded with
It is also noticed that LTDFcm and LTDFes using LBP objects, frontal face images of 13 persons with spectacles are
produce an accuracy of 47.92% and 42.42% respectively. collected from Essex database [7] and the image set is referred in
LTDFcm give highest result with four rings when d is 5 for all the this paper as Essex-po. One image per individual is randomly
rings. Both the models seem to be more efficient when chosen as gallery set and 12 images per person are kept in the
compared with LTDFcs for pose variant images. This is due to probe set. Table.4 gives the experimental results.
the fact that for pose variant images certain information can be From the experimental results in Table.4(a), it is observed
lost and so facial features are captured differently when these that the proposed LTDFcs is able to achieve a highest recognition
models are used. By this analysis, it is very well understood, that accuracy of 97.43% for faces partially occluded with spectacle.
LTDFes performs better for the pose variant images. The results By knowing the effectiveness of LTDFcs, the experiment is
prove the effectiveness of LTDFes. conducted with LTDFes. Experimental results reveal that the
Table.3(a). Performance evaluation of LTDFcm (loosely coupled recognition accuracy obtained by LTDFes is very similar to that
of LTDFcs. In addition it is noticed that the result produced by
DAISY) using LBP on pose variation
circular model using LBP is about 21% greater than that of its
Recognition rate (%) base model which produces an accuracy of 76.28%. This shows
1 the efficiency of LTDFs in recognizing images partially occluded
 
No:
2 3 4 5 6 with spectacle.
Rings LTDFcs
Bins 10 20 30 40 50 60 5.5 RESULTS ON GENERAL RECOGNITION
1 After observing the effects of the proposed LTDFcs and
Distance between rings (d)

19.69 17.42 20.45 29.54 35.60 34.84

(DAISY) LTDFes on various issues in face recognition, performance of the
2 15.15 28.78 37.12 37.12 42.42 42.42 models are experimented for general recognition using AT&T
[24] and Georgia Tech [12] databases. The AT&T database has
3 28.03 37.87 41.66 42.42 45.45 44.69 400 images of 40 subjects that demonstrate expression variation,
4 28.78 37.87 42.42 43.93 46.21 43.93 illumination variation, and partial occlusion with spectacle and
pose variation. The Georgia Tech database contains 750
5 32.57 43.93 43.18 47.92 44.69 45.45 different images of 50 subjects which demonstrate different
6 36.36 43.18 46.96 44.69 43.93 42.42 issues in face recognition like facial expression, occlusion,
different lighting condition, and change in pose. All the images
7 36.36 45.45 45.45 46.96 44.69 43.18 are used for the experiment. The results on one sample training
problem is obtained by keeping one frontal image per subject in
the gallery set and remaining in the probe set and is shown in
Table.5(a).

781
R REENA ROSE et.al: LOCAL TEXTURE DESCRIPTION FRAMEWORK FOR TEXTURE BASED FACE RECOGNITION

Table.4(a). Performance evaluation of LTDFcm (loosely coupled Table.5(b). N-fold Cross-Validation result on AT&T database
DAISY) using LBP on partial occlusion with spectacle for several methods

Recognition rate (%) Recognition

Methods
1 rate (%)
 
No:
2 3 4 5 6
Rings LTDFcs LBP 92.25±3.8
Bins 10 20 30 40 50 60 LTP 93.25±2.89
1
Distance between rings (d)

76.28 80.76 83.33 91.02 94.23 94.87 LTrPs 94±3.76

(DAISY)
s
2 81.41 93.58 95.51 96.15 96.15 96.79 LTDFc _LBP(4) 97.75±2.75
s
3 94.23 96.15 96.79 96.79 96.79 96.79 LTDFc _LTP(4) 98±2.83
s
4 96.15 97.43 96.79 96.79 96.79 95.51 LTDFc _LTrPs(4) 96.5±3.16
s
5 96.79 96.15 96.79 96.15 95.51 94.23 LTDFe _LBP(7,2) 99±1.74
s
6 96.79 96.79 96.79 95.51 94.23 93.58 LTDFe _LTP(7,2) 98.5±2.1
7 s
97.43 97.43 95.51 94.87 94.23 93.58 LTDFe _LTrPs(7,2) 96.75±2.8
It is noticed from Table.5(a) that the proposed LTDFcs and
Table.4(b). Recognition rate (%) on the Essex-po dataset for one LTDFes outperforms the tested base models even with one
sample training problem on several methods sample per subject in the gallery set. To prove the generalization
performance of face recognition with general conditions that
Recognition include all type of variations, an n-fold cross-validation test is
Methods
accuracy conducted on AT&T database by dividing the dataset into 10
LBP 76.28 groups. Each group has exactly one exemplar per 40 subjects
and for each run one group is used for probe set and the
LTP 88.46 remaining for the gallery set. The process is repeated 10 times
LTrPs 87.17 and the mean recognition rate with standard deviation error is
reported in Table.5(b). The results are evident for the proposed
s
LTDFc _LBP(7) 97.43 LTDFs and illustrates that the single ring circular and elliptical
models are capable of achieving high recognition rate than its
LTDFcs _LTP(4) 94.87
base model by using the same number of patterns. Thus with
s
LTDFc _LTrPs(3) 96.79 less number of bins the model is able to achieve better results.
LTDFes _LBP(4,7) 96.79
6. CONCLUSION
s
LTDFe _LTP(2,4) 97.43
Texture features capture the micro primitive patterns present
LTDFes _LTrPs(4,3) 95.51
in the face. This paper proposes a new texture description
method referred as Local Texture Description Framework for an
Table.5(a). Recognition rate (%) on the AT&T and Georgia
effective representation of face images. This descriptor encodes
Tech databases for one sample training problem (one frontal
images based on eight pixels that are certain distance apart from
image per subject) on several methods
the referenced pixel. The descriptor is analyzed for two different
Georgia shapes, circle and ellipse. It is also analyzed for various levels
Methods AT&T Methods of neighborhood. Any texture descriptors that use nearest
Tech
neighbors for describing a texture pattern around a pixel will fit
LBP 57.77 LBP 41.14 in this model. Therefore effectiveness of the model is verified
for standard LBP, LTP and LTrPs. Face recognition issues such
LTP 60.55 LTP 49.14
as illumination variation, expression variation and variation with
LTrPs 58.88 LTrPs 57.57 partial occlusion and pose variation are detailed in this work.
Experiments are also carried out for general cases.
LTDFcs _LBP(4) 70.55 LTDFcs _LBP(4) 52.42
Experimental results demonstrate that the proposed LTDF
s s
LTDFc _LTP(4) 68.61 LTDFc _LTP(5) 59.42 model based on any existing local texture descriptor provides
more accuracy of recognition than that of its conventional ones
LTDFcs _LTrPs(3) 60.27 LTDFcs _LTrPs(5) 59.85
for all the issues discussed. This is due to its ability in
s s
LTDFe _LBP(7,2) 71.11 LTDFe _LBP(4,8) 53.71 determining many number of important local texture primitives
that could discriminate the texture patterns present in different
LTDFes _LTP(5,3) 68.88 LTDFes _LTP(6,3) 60.28 face images. It is found from the experiment that LTDF s excels
LTDFe _LTrPs(1,2) s s
60.55 LTDFe _LTrPs(4,3) 60.14 LTDFm thereby producing better recognition accuracy with less

782
ISSN: 0976-9102(ONLINE) ICTACT JOURNAL ON IMAGE AND VIDEO PROCESSING, FEBRUARY 2014, VOLUME: 04, ISSUE: 03

number of bins. Moreover it is observed that for LTDF s, [11] M. Heikkilla, M. Pietikainen and C. Schmid, “Description
elliptical shape performs better than the circular shape. of interest regions with Local Binary Pattern”, Pattern
Computation of patterns using eight pixels at certain distance Recognition, Vol. 42, No. 3, pp. 425-436, 2009.
apart from a pixel causes many patterns to fall outside the [12] http://www.anefian.com/research/face_reco.htm.
boundary of an image. This decreases the number of patterns in
[13] X. Jiang, B. Mandal and A. Kot, “Eigenfeature
every bin which results in high speed.
regularization and extraction in face recognition”, IEEE
Proposed texture descriptor can be viewed as a new approach Transactions on Pattern Analysis and Machine
in describing texture pattern and hence it is applicable for all the Intelligence, Vol. 30, No. 3, pp. 383-394, 2008.
texture descriptors which use nearest neighbors of a pixel to
[14] Jiang X, “Asymmetric principal component and
describe a texture. In this paper the proposed method is
discriminant analyses for pattern classification”, IEEE
experimented for face recognition, but this can also be suitable
Transactions on Pattern Analysis and Machine
for other pattern recognition application such as fingerprint
Intelligence, Vol. 31, No. 5, pp. 931-937, 2009.
analysis, iris recognition etc. In this work, nearest neighborhood
classifier with chi-square distance metric is used. But [15] M. Kirby and L. Sirovich, “Application of Karhunen-loeve
performance of the model can be improved by using other procedure for the characterization of human faces”, IEEE
classifiers and distance metrics. Transactions on Pattern Analysis and Machine
Intelligence, Vol. 12, No. 1, pp. 103-108, 1990.
REFERENCES [16] Z. Lei, S. Liao, M. Pietikainen and S. Z. Li, “Face
recognition by exploring information jointly in space, scale
[1] Ahonen T, Hadid A and Pietikainen M, “Face description and orientation”, IEEE Transactions on Image Processing,
with local binary patterns: Application to face recognition”, Vol. 20, No. 1, pp. 247-256, 2011.
IEEE Transactions on Pattern Analysis and Machine [17] S. Liao, W. K. Law and A. C. S. Chung, “Combining
Intelligence, Vol. 28, No. 12, pp. 2037-2041, 2006. microscopic and macroscopic information for rotation and
[2] Andrzej Materka and Michal Strzelecki, “Texture Analysis histogram equalization invariant texture classification”,
Methods – A Review”, Institute of Electronics, Technical Proceedings of the 7th Asian Conference on Computer
University of Lodz, COST B11 report, pp. 1-33, 1998. Vision – Volume Part I, pp. 100-109, 2006.
[3] P. N. Belhumeur, J. P. Hespanha and D. J. Kriegman, [18] S. Liao, M. W. K. Chung and A. C. S. Chung, “Dominant
“Eigenfaces vs. fisherfaces: recognition using class specific Local Binary Patterns for texture classification”, IEEE
linear projection”, IEEE Transactions on Pattern Analysis Transactions on Image Processing, Vol. 18, No. 5,
and Machine Intelligence, Vol. 19, No. 7, pp. 711-720, pp. 1107-1118, 2009.
1997. [19] Lucieer A, Stein A and Fisher P, “Multivariate texture-
[4] Cevikalp H, Neamtu M, Wilker M and Barkana A, based segmentation of remotely sensed imagery for
“Discriminative common vectors for face recognition”, extraction of objects and their uncertainity”, International
IEEE Transactions on Pattern Analysis and Machine Journal of Remote Sensing, Vol. 26, No. 14, pp. 2917-
Intelligence, Vol. 27, No. 1, pp. 4-13, 2005. 2936, 2005.
[5] C. H. Chan, J. Kittler and K. Messer, “Multi-scale Local [20] M. Lyons, S. Akamatsu, M. Kamachi and J. Gyoba,
Binary Pattern Histograms for face recognition”, “Coding Facial Expressions with Gabor Wavelets”,
Proceedings of the International Conference on Advances Proceedings of the 3rd International Conference on
in Biometrics, pp. 809-818, 2007. Automatic Face and Gesture Recognition, pp. 200-205,
1998.
[6] Cong Geng and Xudong Jiang, “Fully Automatic Face
Recognition Framework Based on Local and Global [21] Mihran Tuceryan and Anil K. Jain, “Texture Analysis”, The
Features”, Machine Vision and Applications, Vol. 24, No. Handbook of Pattern Recognition and Computer Vision,
3, pp. 537-549, 2013. 2nd Edition, World Scientific Publishing Co., pp. 207-248,
1998.
[7] Face Recognition Data, University of Essex, UK, The Data
Archive, http://cswww.essex.ac.uk/mv/allfaces/index.html. [22] Ojala T, Pietikainen M and Harwood D, “A comparative
study of Texture Measures with Classification based on
[8] Z. Guo, L. Zhang and D. Zhang, “Rotation invariant texture
Featured Distributions”, Pattern Recognition, Vol. 29, No.
classification using LBP variance (LBPV) with global
1, pp. 51-59, 1996.
matching”, Pattern Recognition, Vol. 43, No. 3, pp. 706-
719, 2010. [23] Ojala T, Pietikainen M and Maenpaa T, “Multiresolution
gray-scale and rotation invariant texture classification with
[9] Hamidreza Rashidy Kanan and Karim Faez, “Recognizing
local binary patterns”, IEEE Transactions on Pattern
faces using Adaptively Weighted Sub-Gabor Array from a
Analysis and Machine Intelligence, Vol. 24, No. 7, pp.
single sample image per enrolled subject”, Image and
971–987, 2002.
Vision Computing, Vol. 28, No. 3, pp. 438-448, 2010.
[24] Samaria F. S and Harter A. C, “Parameterisation of a
[10] Heikkilla M and Pietikainen M, “A texture based method
stochastic model for human face Identification”,
for modeling the background and detecting the moving
Proceedings of the 2nd IEEE Workshop on Applications of
objects”, IEEE Transactions on Pattern Analysis and
Computer Vision, pp.138-142, 1994.
Machine Intelligence, Vol. 28, No. 4, pp. 657-662, 2006.

783
R REENA ROSE et.al: LOCAL TEXTURE DESCRIPTION FRAMEWORK FOR TEXTURE BASED FACE RECOGNITION

[25] Subrahmanyam Murala, Maheshwari R. P and for dimensionality reduction”, IEEE Transactions on
Balasubramanian R, “Local Tetra Patterns: A New Feature Pattern Analysis and Machine Intelligence, Vol. 29, No. 1,
Descriptor for Content-Based Image Retrieval”, IEEE pp. 40-51, 2007.
Transactions on Image Processing, Vol. 21, No. 5, pp. [33] Yang J, Frangi A, Yang J, Zhang D and Jin Z, “KPCA plus
2874-2886, 2012. LDA: a complete kernel fisher Discriminant framework for
[26] Suruliandi A and Ramar K, “Local Texture Patterns – A feature extraction and recognition”, IEEE Transactions on
Univariate Texture Model for Classification of Images”, Pattern Analysis and Machine Intelligence, Vol. 27, No. 2,
Proceedings of the International Conference on Advanced pp. 230-244, 2005.
Computing and Communications, pp. 32-39, 2008. [34] Ye J, Janardan R, Park C and Park H, “An optimization
[27] Suruliandi A, Meena K and Reena Rose R, “Local binary criterion for generalized discriminant analysis on
pattern and its derivatives for face recognition”, IET undersampled problems”, IEEE Transactions on Pattern
Computer Vision, Vol. 6, No. 5, pp. 480-488, 2012. Analysis and Machine Intelligence, Vol. 26, No. 8, pp. 982-
[28] Tola Engin, V. Lepetit and P. Fua, “DAISY: An efficient 994, 2004.
dense descriptor applied to Wide-Baseline Stereo”, IEEE [35] Yousra Ben Jemaa and Sana Khanfir, “Automatic Local
Transactions on Pattern Analysis and Machine Gabor features extraction for face recognition”,
Intelligence, Vol. 32, No. 5, pp. 815-830, 2010. International Journal of Computer Science and
[29] Topi Maenpaa and Matti Pietikainen, “Texture Analysis Information Security, Vol. 3, No. 1, 2009.
with Local Binary Patterns”, Handbook of Pattern [36] Zhang B, Gao Y, Zhao S and Liu J, “Local derivative
Recognition and Computer Vision, 3rd Edition, World pattern versus local binary pattern: Face recognition with
Scientific, 197-216, 2005. higher-order local pattern descriptor”, IEEE Transactions
[30] Vidit Jain, Amitabha Mukherjee, 2002, The Indian Face on Image Processing, Vol. 19, No. 2, pp. 533-544, 2010.
Database, http://vis_www.cs.umass.edu/~vidit/Indian Face [37] Zhao G and Pietikainen M, “Dynamic texture recognition
Database/. using local binary patterns with an applications to facial
[31] Wang X and Tang X, “A unified framework for subspace expressions”, IEEE Transactions on Pattern Analysis and
face recognition”, IEEE Transactions on Pattern Analysis Machine Intelligence, Vol. 29, No. 6, pp. 915-928, 2007.
and Machine Intelligence, Vol. 26, No. 9, pp. 222-1228, [38] Zheng W and Tang X, “Fast algorithm for updating the
2004. Discriminant vectors of dual-space IDA”, IEEE
[32] Yan S, Xu D, Zhang B, Yang Q, Zhang H and Lin S, Transactions on Information Forensics and Security,
“Graph embedding and extensions: a general framework Vol. 4, No. 3, pp. 418-427, 2009.

784