A NOVEL FACE RECOGNITION ALGORITHM BASED ON PCA

Rajkiran Gottumukkal and Vijayan K. Asari

VLSI Systems Laboratory, Department of Electrical and Computer Engineering
Old Dominion University, Norfolk, VA 23529
ABSTRACT and a common performance benchmark in face
The research in this paper is driven by the need to design recognition [6].
a face recognition method, which would be able to cope
with unconstrained variations in illumination, and head 2. REVIEW OF FEATURE EXTRACTION
pose between images of the same individual. The main BASED ON PCA
novelty of this work is the idea that some of the local PCA is a conceptual approach for feature extraction,
facial features of the same individual do not vary even whereby compression of the discriminatory information is
when the facial expression, illumination and pose vary. achieved by approximating the representation by a small
This idea is applied to the principal component analysis number of terms of the expansion. For any given number
(PCA) method to arrive at a new method called composite d of terms, the mean square error between the
PCA. The accuracy of the conventional PCA method and approximation and the original pattern is minimal.
composite PCA method are evaluated under the Let I1 , I 2 , … , I M be the set of face images, size of each
conditions of varying expression, illumination and pose
using standard databases. The composite PCA technique image being L×L, in the face database represented as row
is found to be more accurate than the conventional PCA vectors of size L2. Then the average face can be computed
method under these conditions. Another advantage of the as
proposed method is the significant reduction in training 1 M
A= ∑ Ii (1)
time for the computation of principal components. M i =1

Each face differs from the average face by the vector

1. INTRODUCTION
Automatic recognition of the human faces has been an Yi = I i − A . The covariance matrix C is constructed from
area of active research for the past few years. Although these vectors as
recognition of faces does not seem to present any 1 M T
difficulties for most human observers, computerized face C= ∑ Yi ⋅ Yi (2)
M i =1
recognition systems fail to achieve satisfactory
performance due to the generally similar shape of the where the dimension of C will be L2× L2. The
faces combined with the numerous variations in eigenvectors of the covariance matrix are computed and
expression, illumination and pose between images of the the M’ significant eigenvectors are chosen as those with
same face. This task, therefore, still presents a significant the largest corresponding eigenvalues. The eigenvectors
challenge and is considered as one of the fundamental are represented as E1 , E2 , … , E M ' , where each eigenvector
problems in computer vision and pattern analysis.
is a row vector of size L2. The weight vector for each
There are two main approaches for face recognition [1-3].
image in the face database is computed from the
The first approach is the feature-based matching approach
eigenvectors as shown below:
using the relationship between facial features such as
eyes, mouth and nose [3]. The second approach is the
template matching approach using the holistic features of
W pnr = E
r
T
(
⋅ I pn − A ) ∀p , n, r (3)

face images [3-4][7]. In this approach the salient regions where Er ' s are the eigenvectors corresponding to the M’
of the facial image are extracted and directly compared on
largest eigenvalues of C and r varies for 1 to M’; n varies
a pixel-by-pixel basis with an image database of known
from 1 to Γ , Γ being the number of images per
individuals.
individual and p varies from 1,2… P, P being the number
Template based techniques often follow the subspace
of individuals in the training set. The mean weight vector
method called eigenface originated by Turk and Pentland
for each individual is computed as shown below:
[4]. This technique is based on the Karhunen-Loève
transformation, which is also referred to as PCA, and was 1 Γ
Tpr = ∑ W pnr ∀p , r (4)
introduced into face processing by Kirby and Sirovich [5]. Γ n =1
It has gained great success and become a de facto standard
Next the weight vector is computed for the test image images are considered to be matrices of size L×L. Hence
using the eigenvectors obtained previously as shown the size of each sub-image will be L2 N 2 . These sub-
below:
images can be represented as a function of the original
T
(
Wtest r = Er ⋅ Itest − A ) ∀r (5)
image, I i as shown below:
To classify the test image, the city block distance of the L L 
weight vector of the test image from the weight vectors of I ijk ( x , y ) = I i 
( j − 1) + x , ( k − 1) + y  ∀i , j , k (7)
the individuals in the face database is computed as shown N N 
below: where i varies from 1 to M, M being the number of images
in the training set; j and k vary from 1 to N, N2 being the
1 M'
number of sub-images and x and y vary from 1 to L / N .
Dp = ∑ Tpr − Wtest r ∀p (6)
M ' r =1 An average image is obtained from the test sub-images as
th 1 M N N
The minimum value of D p corresponds to the p A= 2
∑ ∑ ∑ I ijk (8)
M ⋅N i =1 j =1 k =1
individual in the training set whose face resembles closest
The next step is to normalize each training sub-image by
to the face in the test image. The minimum distance is
subtracting it from the average as
compared with a threshold value, and if it is less than the
threshold it is considered as recognition. On the other Yijk = I ijk − A ∀i , j , k (9)
hand, if it is greater than or equal to the threshold value From the normalized images the covariance matrix is
the test image is rejected. There will be some images, computed as shown below:
which will be falsely rejected and falsely recognized, and 1 M N N T
to minimize this the threshold should be set optimally. C= 2
∑ ∑ ∑ Yijk ⋅ Yijk (10)
The technique used by us to compute the threshold is M ⋅N i =1 j =1 k =1

described in the next section. C will be a matrix of size L2 N 2 × L2 N 2 ; hence the size
of the covariance matrix is reduced by a factor of N4
3. FACE RECOGNITION BASED ON compared to the covariance matrix computed for the
COMPOSITE PCA conventional PCA method. Next the eigenvalues and
The PCA based face recognition method is not very eigenvectors of C are computed, and M ' eigenvectors
effective under the conditions of varying pose and corresponding to the M ' largest eigenvalues are
illumination, since it considers the global information of considered to compute the weights of the training sub-
each face image and represents them with a set of weights. images as
Under these conditions the weight vectors will vary
considerably form the weight vectors of the images with
T
W pnjkr = Er ⋅ I pnjk − A ( ) ∀p , n, j , k , r (11)
normal illumination and pose, hence it is difficult to where r varies from 1 to M ' ; n varies from 1 to Γ , Γ
identify them correctly. On the other hand if the face being the number of images per individuals and p varies
images were divided into smaller regions and the weight from 1 to P, P being the number of individuals in the
vector are computed for each of these regions, then the training set. From these weight vectors the mean weights
weights will be more representative of the local of sub-images belonging to each individual are computed
information of the face. When there is a variation in the as shown below:
illumination or pose, only some of the face regions will 1 Γ
vary and rest of the regions will remain the same as the Tpjkr = ∑ W pnjkr ∀p , j , k , r (12)
face regions of a normal image. Hence weights of the Γ n =1

face regions not affected by varying illumination and pose The concept of computing the threshold for a particular
will closely match with the weights of the same class (individual) is based on the minimum of the
individuals face regions under normal conditions. maximum distances obtained by finding the city block
Therefore it is expected that better positive recognition distances between the nth weight value with respect to P
rates can be obtained by following this approach and we individual’s mean weights.
refer to this approach as composite PCA. We expect that The city block distances of all the image weights from the
if the face images are divided into very small regions the mean weights obtained in equation (12) are computed as
global information of the face may be lost and the 1 N N M' '
accuracy of this method may deteriorate. D = ∑ ∑ ∑ T p' jkr −W pnjkr ∀p , n, p (13)
pnp' N 2
j =1 k =1 r =1
3.1. Training phase where p’ is indicating the number of individuals and
The first step in this phase is to divide all the images in p’ = 1, 2, .., P. Equation (13) represents the distance of
the database into N2 smaller regions, and as before all the the pth individual’s nth image’s weight from the mean
weight of the p’th individual in the database. From this the shown in Fig. 1 and the results obtained are shown in
threshold for classification is set for each of the pth class Table 1.
as
'
H p = min(max( D )) ∀p , n, p (14)
pnp'
This threshold is used to classify the test images. Fig. 1. Images of an individual from the ODU database with
varying facial expressions

3.2. Testing phase Table 1. Comparison of results obtained under the condition of varying
facial expressions for PCA and composite PCA
The testing phase starts with dividing the test image into
N2 smaller regions using equation (7). The weights Composite PCA
PCA
corresponding to the N2 sub-images are computed as N =42
N2=16 N2=64
shown below:
Positive recognition rate 100.0 100.0 100.0 100.0
T
(
Wtest jkr = Er ⋅ Itest jk − A ) ∀j , k , r (15)
Negative recognition rate 0.0 0.0 0.0 0.0
The mean distance of the test image weights from the False rejection rate 0.0 0.0 0.0 0.0
mean weights of all the classes is computed as Training time in seconds 115.31 40.44 38.33 81.14
1 N N M'
Dp = ∑ ∑ ∑ T pjkr −Wtest jkr ∀p (16) Testing time in seconds 0.11 0.16 0.23 0.63
N2 j =1 k =1r =1
The second database used for comparing the two methods
The minimum value of Dp is considered and if it is less was the Yale database and was obtained from [8]. The
than the corresponding threshold Hp then it is considered Yale database has images of 15 individuals, each
as recognition. It is considered as a positive recognition if individual having 11 images. The face images vary with
it is recognized as the correct individual, else it is respect to facial expression and illumination. For our
considered as a negative recognition. If the minimum experiments we only used the images with varying
value Dp is greater than or equal to the corresponding illumination. Out of the four images of a person, only one
threshold Hp, the test image is rejected. was used for training and to test the recognition rates all
the images were used. The images of an individual from
4. CONVENTIONAL PCA VS. COMPOSITE PCA the Yale database used for training and testing are shown
The performance of the composite PCA method is in Fig. 2 and the results obtained are shown in Table 2.
evaluated with respect to that of the conventional PCA
method by testing them with three face databases. All
images in the databases are matrices of size 64×64. Only
the first 20 eigenvectors were considered for the tests, i.e. (a) (b)
M ' =20. When the size of the sub-images is less than or Fig. 2. Images of an individual from the Yale database with
equal to 4×4 (N2=256), the number of eigenvectors that varying illumination. Image in 2(a) was used for training and
images in both 2(a) and 2(b) were used for testing
can be obtained from the covariance matrix will be less
than 20, since the size of the covariance matrix is less than Table 2. Comparison of results obtained under the condition of varying
illumination for PCA and composite PCA
or equal to 16×16. It was observed that the recognition
rate deteriorates for N2 ≥ 256 since some of the local Composite PCA
PCA
information of the face is lost when the face images are N =42
N2=16 N2=64
divided into very small sub-images. Hence in this paper
we only analyze the results obtained for N2 ≤ 256. All the Positive recognition rate 63.33 73.33 86.67 81.67
timing results were obtained on a 1.5GHz Dell Negative recognition rate 36.67 18.33 13.33 18.33
workstation powered by Intel® Xeon processor. False rejection rate 0.0 8.33 0.0 0.0
The first database used for testing was the ODU database;
it has face images of 15 individuals, each individual Training time in seconds 132.47 17.94 18.03 44.41
having 8 images with varying expressions. Of the 8 Testing time in seconds 0.13 0.16 0.25 0.64
images of each person in the face database only 4 images
per person are chosen randomly to train the algorithm. The third database used for comparing the two methods
The algorithm is tested with all the images in the database, was the UMIST database and was obtained from [9]. We
which will ensure that the algorithm is tested thoroughly used 120 images of 15 individuals, each individual having
with the images in the training set as well as new images. 6 images for our experiments. Each image of a person was
The images of an individual from this face database are taken at a different pose, with a normal expression. Out of
the six images of a person, only four were used for
training and to test the recognition rates all the images recognition time for composite PCA since the weight
were used. The images of an individual from the UMIST vectors of the sub-images of the test image are computed
database used for training and testing are shown in Fig. 3 serially and the distances of these weights from the mean
and the performance of the two methods with UMIST weights are also computed serially. We were not able to
database is shown in Table 3. utilize the parallelism available in composite PCA using
our programs written in Matlab on a desktop computer.
The testing (recognition) stage of the composite PCA can
be made faster than the conventional PCA by using
(a) (b)
special purpose hardware to utilize the parallelism
Fig. 3. Images of an individual from the UMIST database with available in composite PCA.
varying pose. Images in 3(a) were used for training and
images in both 3(a) and 3(b) were used for testing
5. CONCLUSION
Table 3. Comparison of results obtained under the condition of varying The concept of face recognition based on PCA has been
pose for PCA and composite PCA
modified to arrive at a new method called composite PCA.
Composite PCA We showed that the new method is more accurate than the
PCA conventional PCA method and has more inherent
N2=4 N2=16 N2=64
parallelism. By using the composite PCA method the size
Positive recognition rate 83.33 87.78 90.0 91.11 of the covariance matrix is reduced hence the training
Negative recognition rate 16.67 12.22 10.0 8.89
phase can be done faster. The recognition time can also be
improved by designing special purpose hardware to utilize
False rejection rate 0.0 0.0 0.0 0.0 the parallelism available in composite PCA. To perform
Training time in seconds 108.56 40.59 32.14 64.80 face recognition in unconstrained environments composite
PCA is better suited than the conventional PCA as
Testing time in seconds 0.11 0.16 0.24 0.63
demonstrated by the results obtained under the conditions
of varying illumination and pose.
4.1. Discussion of the results 6. REFERENCES
From the results we note that the conventional PCA [1] R. Chellappa, C.L. Wilson, and S. Sirohey, “Human and
method performs well under the condition of varying machine recognition of faces: a survey,” Proc. of IEEE,
expressions for a small database. Since the positive vol. 83, No. 5, pp 705–740, May 1995.
recognition rate is 100% for both the methods we cannot
establish which method is more accurate. Using a larger [2] A. Samal and P.A. Iyengar, “Automatic recognition and
analysis of human faces and facial expressions: a survey,”
database would be a better method to evaluate the
Pattern Recognition, 25 (1), 65–77, 1992.
performances of both the algorithms. In the case of Yale
database, which is subjected to varying illumination, [3] R. Brunelli and T. Poggio, “Face recognition: feature
significant improvement in positive recognition rate is versus templates,” IEEE Trans. Pattern Anal. Machine
observed at N2=16. The negative recognition rate is also Intell, 15 (10), 1042–1052, 1993.
reduced at N2=16. In the case of UMIST database, which [4] M. Turk and A. Pentland, “Face recognition using
is subjected to varying pose, a good improvement in eigenfaces,” Proc. IEEE Conf. on Computer Vision and
positive recognition rate is observed at N2=16 and the Pattern Recognition, pp. 586–591, 1991.
negative recognition rate is also decreased. Hence the
optimum value of N2 could be 16, where best [5] M. Kirby and L. Sirovich, “Application of the Karhunen–
Loeve procedure for the characterization of human
improvement in the recognition rates can be obtained faces,” IEEE Transactions on Pattern Analysis and
using composite PCA. Machine Intelligence, 12 (1), 103–108, 1990.
Here it is worth noting that the computations required to
obtain the eigenvectors will be fewer in the case of [6] B. Moghaddam, T. Jebara, A. Pentland, “Bayesian face
composite PCA method compared to the conventional recognition,” Pattern Recognition, 33 (11), 1771–1782,
PCA method, since the size of the covariance matrix is 2000.
reduced by a factor of N4 in this method. Hence we see [7] D.B. Graham and N.M. Allinson, “Characterizing Virtual
that the training time is less for composite PCA. On the Eigensignatures for General Purpose Face Recognition,”
other hand the number of weight vectors computed for Face Recognition: From Theory to Applications, NATO
each image increases by a factor of N4. But the number of ASI Series F, Computer and Systems Sciences, Vol. 163,
computations required to obtain the weight vectors pp 446-456, 1998.
remains the same since the size of the eigenvectors is [8] http://cvc.yale.edu/projects/yalefaces/yalefaces.html
reduced by a factor of N4. But we see an increase in the
[9] http://images.ee.umist.ac.uk/danny/database.html