International Journal of Advanced Science and Technology

Vol. 52, March, 2013

A New Approach for Face Image Enhancement and Recognition

Niloofar Amani1, Asadollah Shahbahrami2 and Manoochehr Nahvi1
1

Department of Electrical Engineering, 2Department of Computer Engineering

Faculty of Engineering, University of Guilan, Rasht, Iran
[email protected], {shahbahrami, nahvi}@guilan.ac.ir
Abstract

A new approach is presented to improve the face recognition accuracy. This approach is
based on the contrast enhancement using high-frequency emphasize filtering and histogram
equalization. In the presented method, image contrast and the global (or local) visualization
are enhanced using digital filtering and equalizing the histogram of the pixel values over
entire image. For this, first the face images are transformed into a high-frequency domain
and then the global thresholding technique, by Otsu method, is applied to the image. Then,
the values lower than threshold has only been considered. For dimension reduction and also
feature extraction purpose the linear method such as two dimensional principle component
analysis (2DPCA) and two dimensional linear discriminant analysis (2DLDA) are adopted.
In the last stage of the algorithm, the simple minimum distance method is exploited for the
classification. Experimental trials demonstrate that the presented method is leading to the
promising recognition rates and noticeable improvement in the face recognition system.
Keywords: face recognition, face image enhancement, high-frequency emphasis filtering,
2DPCA, 2DLDA

1. Introduction
Face recognition is one of the most outstanding abilities of human vision, and is probably
the biometric method easier to understand. This is the reason that the features of individual
face are considered in security and access control systems, law enforcement, surveillance.
Many of scientists and research groups are involved in face recognition methods and systems.
With the expectation to make the computer like the humans, having the ability to recognize
people, and perceive in close contact [1-4]. Building an automated system that accomplishes
such objective is very challenging. The challenges mainly come from the large variations in
the visual stimulus due to illumination conditions, facial expressions, aging, and disguises
such as facial hair, glasses [4-13].
During the past two decades, many face recognition systems have been proposed as
reviewed in [13-23]. An important part of these systems is feature extraction. In this stage, a
proper face representation is needed. This would be computationally feasible and also robust
to possible intrinsic and extrinsic facial variations [13]. Several feature extraction methods
have been exploited in face recognition systems including Eigenfaces [8], Fisherfaces [6],
Laplacian faces, nearest feature line-based subspace analysis, neural networks, elastic bunch
graph matching, wavelets, and kernel methods [1, 2, 4], Principal Component Analysis (PCA)
[13, 15] and Linear Discriminant Analysis (LDA) have successfully been exploited.

International Journal of Advanced Science and Technology

Vol. 52, March, 2013

In face recognition literature, there are various face representation methods based on
global features, including a large number of subspace-based methods and some spatialfrequency techniques [13]. In face recognition, persons are identied by use of a stored set of
face images. In traditional appearance-based models, the intensity of each pixel in a face
image is used as an input feature. Since there are more than tens of thousands of pixels in a
face image [18], so facial image data are always high-dimensional and considerable
computational time is required for the classification purpose. Thus the subspace methods, by
projecting patterns to a lower dimensional space, are widely exploited. In practical situations,
when the image dimension is prohibitively large, one is often forced to use linear techniques.
Two important linear techniques for extracting discriminative feature and also dimension
reduction are PCA and LDA. The focuses of some researchers are based on projective maps.
The main aspect of these methods is generating feature vector for each face image, then
classify the input face image in the database. Generating feature vector also has the
advantages of reducing dimension of the input images [21]. The PCA method performs
dimension reduction by projecting the original data onto lower dimensional subspace spanned
by the leading eigenvectors of the covariance matrix. The LDA method searches for the
projective axes on which the data points of different classes are far from each other
(maximizing between class scatter), while constraining the data points of the same class to be
as close to each other as possible (minimizing within class scatter) [5]. Many researchers
propose methods based on spatial-frequency techniques, such as Fourier transform [1, 24] and
Discrete Cosine Transform (DCT) [2, 3]. In these methods, face images are transformed to a
lower frequency domain bands contain most facial discriminating features and ignoring high
bands containing noise [1, 13, 19].
An important issue to improve the performance of face recognition system is enhancing
face image. The aim of image enhancement is that the images have better visual quality. By
enhancing the brightness, contrast and resolution of image can be improved. This is a part of
pre-processing stage that can influence the feature extraction and therefore the final
recognition performance. For instance in [22, 23], the image enhancement has been
considered in face recognition system. Song et al. [23], calculates, before feature extraction
stage, the illumination difference between right and left part of face. If there is a large amount
of difference than take the mirror of average illuminated part.
In this paper the impact of image enhancement procedure on the face recognition are also
considered. For feature extraction stage, we used the combination of 2DPCA and 2DLDA.
The simulation results show that our image enhancement procedure noticeably increases the
face recognition accuracy.
In what follows general background information and also pre-processing stages are given
in the next section. Then the proposed arrangement and finally experimental results are
presented in the Sections 4 and 5, respectively.

2. Background
A brief conceptual block diagram of face recognition and verification system is illustrated
in the Figure 1. This diagram illustrates the training stage at top and the test stage at the top of
the figure. The pre-processing includes image size conversions, image histogram equalization
and some other enhancement process. Then the weight vector constructed in the feature
extraction stage is compared with the of every face database member. If there is at least one
face in database similar to the acquired image, then the input face is classified as "known",
otherwise unknown".

International Journal of Advanced Science and Technology

Vol. 52, March, 2013

Figure 1. Block Diagram of Recognition/Verification System

2.1 Pre-processing
In this section, various processing including high-frequency emphasis filtering, histogram
equalization, adaptive thresholding and intensity-level slicing are described.
2.1.1 High-Frequency Emphasis Filtering
High-frequency emphasis [25, 26], has a filter transfer function given by eq. (1)

Hhe (u, v) = a + bHhp (u, v)

(1)

Where a 0 and b > a, typical values of a are in the range of 0.25 to 0.5 and typical
values of b are in the range 1.5 to 2.0. Hhp (u, v) is the transfer function of the corresponding
high-pass filter in this paper. The transfer function of the Butterworth High-Pass Filter
(BHPF) of order n is adopted, and the transfer function is given by eq. (2)

Hhp (u, v) = {

Where
is the specified nonnegative quantity, and
to the center of the frequency rectangle.

(2)
is the distance from point

Let f and g denote two discrete functions in the two dimensional discrete space, and let
f(x, y) denote the gray level of point (x, y) in image, g(x, y) denote the gray level of point (x,
y) in enhanced image, F and G denote Fourier transform of corresponding image. The
expression of high-frequency emphasis follows directly from (1) and (2)

G (u, v) = Hhe (u, v) F (u, v) =

(3)

The convolution theorem tells us that the corresponding process in the spatial domain is
(4)
Where h

is the inverse Fourier transform of the filter transform function H(u, v).

2.1.2 Histogram Equalization

Histogram equalization based on the probability theory is a common method for enhancing
the low contrast and the appearance of a digital image by effectively spreading out the most
frequently occurred intensity values through a nonlinear transformation function and is
commonly histogram modification approach [15]. The main propose of histogram
equalization is to allow pixels in areas of lower contrast to gain a higher contrast [26-31].
2.1.3 Thresholding
Thresholding is a non-linear operation converting gray-level images into binary images by
selecting an appropriate decision. By this operation pixels are assigned to two levels below or

International Journal of Advanced Science and Technology

Vol. 52, March, 2013

above the specified threshold value [32, 33]. If the pixel lies above the threshold, it will be
marked as foreground, and if it is below or equal to the threshold as background. The
threshold value may adaptively be determined according to image content, intensity or color
value.
2.1.4 Intensity-Level Slicing
Intensity-level slicing or gray level slicing methods belong to the category of point
operations and function by changing the pixel value, or gray level, by a mapping process. The
mapping equation can be typically linear or nonlinear. Typical applications of intensity-level
slicing include contrast enhancement and feature enhancement [25, 34].

3. Proposed Method
This paper is focused on the second and third stages of the Figure 1 which are two key
parts of a successful face recognition system. The pre-processing stage includes image
resizing, high-frequency emphasis filtering, histogram equalization, adaptive thresholding and
intensity-level slicing, as shown in Figure 2.
In the feature extraction block the 2DPCA is performed first and then 2DLDA is used for
the second feature extraction in the 2DPCA transformed space. This has three important
advantages easier and accurate evaluation of the covariance matrices, reduction the time
required for the determination of the corresponding eigenvectors and finally significant
increase in the face recognition accuracy [35]. In the classification, we used the Nearest
Neighbor (NN) classifier due to its simplicity and performance in our experiments. Its
training is very fast, robustness to noisy training data and effectiveness for the large training
data set [36].

Figure 2. Block-diagram of pre-processing stage

In the following, each part of the system is examined.
High-Frequency Emphasis Filtering:
In an image, edges and other abrupt changes in gray levels are associated with highfrequency components. Image sharpening can be achieved in the frequency domain by a highpass filtering process in the Fourier Transform. In this case, we multiply a high-pass filter
function by a constant value and add offset so that high-frequency components of image are
enhanced and the gray-level tonality due to the low frequency components are retained.
As it is discussed above, the texture of face image is enhanced by the algorithm of highfrequency emphasis filtering, in our simulation a=0.5 and b=1.5. In order to further
improvement in contrast of face and background, histogram equalization is used [26].
Choosing the high-pass filter for pre-processing envolves a lot of trial and error. Experiments
were conducted with Butterworth and Gaussian filters and improvement in performance was
observed with Butterworth filter. To study the impact of the High-frequency emphasis with
Butterworth filter, in Figure 3, their maximum recognition rates were compared with the
Gaussian filter. The figure shows the impact of Butterworth and Gaussian filter in recognition
rate in the left and right, respectively.

International Journal of Advanced Science and Technology

Vol. 52, March, 2013

Figure 3. Comparison of Butterworth Filter and Gaussian Filter

The result of simulation is shown in Figure 4. The left picture is a faces in the utilized
database after resizing and the right picture is the result of high-frequency emphasis filtering.

High-frequency
Emphasis Filtering

Figure 4. Original Image and the Result of High-frequency Emphasis

Histogram Equalization:
Automatic face recognition performance is also affected by illumination [37, 38]. The
problem is that the variation between images of different faces can be smaller than the
variations between images of the same face under different illumination. It can even be shown
that illumination causes larger variation in face images than pose [39]. At the very beginning
of modern machine face recognition, it became quite clear that different illumination in
various images will be a problem [15, 40]. Image pre-processing and normalization is
significant part of face recognition systems. Changes in lighting conditions lead to dramatic
decrease in the recognition performance [41]. Histogram equalization is an effective method
in stretching the range of gray levels and enlarging the contrast of image. It also makes the
change of the order of gray levels of the original image completely controllable. Therefore, it
can enhance images effectively [26].
Figure 5 is the result of applying histogram equalization on the Figure 4. Between these
two figures, it can be seen that the contrast of face image and background become more, and
the outline of the face image is clearer.

International Journal of Advanced Science and Technology

Vol. 52, March, 2013

Figure 5. Face image after high-frequency emphasis filtering and the result of
histogram equalization
Thresholding:
Thresholding is an essential task in the image processing and pattern recognition. In
general, automatic thresholding techniques are classied into two basic groups: global and
local methods. Global thresholding uses only one threshold value estimated based on statistics
or heuristics on global image attributes while local methods exploit threshold values that
change dynamically in the image. Many techniques for global image thresholding have been
proposed in the literature [32, 42]. In this work Otsus method is exploited for the global
thresholding [43]. The threshold is individually calculated for each image so we called it
adaptive thresholding.
Intensity-Level Slicing:
Image enhancement techniques are used to emphasize and sharpen image features for the
purpose of display and analysis. Highlighting a specific range of gray levels in certain ranges
is often desired. In this paper we use a type of intensity-level slicing shown in Figure 6. This
transformation preserves intensities less than threshold and reduces all other intensities to a
lower level.

Output Intensity Level,s

L -1

Threshold
Input Intensity Level,r

L -1

Figure 6. Intensity-level slicing that used in this paper

In the face recognition whole face plays an essential role. This overall appearance is
determined based on the darker areas. For instance, the overall border of a face can be
characterized by hair, and the dark areas of the face such as the eyes, nose, mouth etc. give us
useful information about the face. It is obvious that the fine or pale lines on the face play
unimportant role in this type of recognition. In human face there are areas with lower

International Journal of Advanced Science and Technology

Vol. 52, March, 2013

intensity level (darker), showing important features of the face, and operate efficiently in face
recognition. By emphasizing on this regions and ignoring the greater intensity level (brighter
areas), we can continue the recognition process. This kind of procedure allows us keeping the
essential features, less memory requirements and smaller processing time. Our criterion to
distinguish between the light from dark areas is Otsus thresholding method. This method
calculates the threshold for each image individually. Since this kind of thresholding is binary
and has value between 0 and 1. It can therefore be used in gray scale by its multiplying by
255. Finally, we keep the regions with equal or higher than threshold times 255 and those
with lower than threshold with their original values. Figure 7 shows the result of face image
after intensity level slicing. The threshold calculated by Otsu method was 0.4941.
Multiplying the threshold by 255, the threshold for gray scale image will be 126,
approximately.

Figure 7. Face image after histogram equalization and the result of face image
after pre-processing
Feature Extraction: 2DPCA + 2DLDA:
The 2DPCA method is an appropriate technique for 2D data set such as images [44, 45].
There is another method called two Dimensional Linear Discriminant Analysis (2DLDA)
which is also a well-known method for feature extraction and dimension reduction. These
methods have been used in many applications such as [46, 47] and [48]. In contrast to the
covariance matrix of PCA, the size of the image covariance matrix of 2DPCA is much
smaller [45]. In both PCA and 2DPCA the eigenvectors (Eigenfaces) can be calculated by
using the Singular Value Decomposition (SVD) techniques [49, 50]. The process of
generating the covariance matrix is actually avoided [44]. This technique is effective for
reducing computation when the training sample size is much smaller than the dimensionality
of the images [15, 49].
In this paper, we combine 2DPCA and 2DLDA to reduce the dimension, which is also
called two Dimensional Fisher Principal Component Analysis (2DFPCA) [35].
In this paper, we used 20 feature vectors for 2DPCA. Our results show that the number of
2DLDAs Eigen vectors required to optimize this algorithm is only 10. The number of Eigen
vectors of 2DPCA and 2DLDA are selected in such a way that the performance of 2DPCA +
2DLDA is optimized.

4. Experimental Results
Two databases were used to validate the effectiveness of our method. We conduct
experiments on the AT&T (Olivetti) database and Yale face database. AT&T database
consists of 400 images (112 92 Pixels), 10 different images from 40 individuals. The Yale
database contains 165 images of 15 individuals (11 images per individual) (100 100 pixels),

International Journal of Advanced Science and Technology

Vol. 52, March, 2013

the images were taken at different times, varying the lighting, facial expressions (open /
closed eyes, smiling / not smiling) and facial details (glasses / no glasses). All the images
were taken against a dark homogeneous background with the subjects in an upright, frontal
position (with tolerance for some side movement). The images were resized to 95 105
pixels in this experiment.
In Figure 8, maximum recognition rate achieved by 2DLDA plus 2DPCA were compared
with 2DPCA, LDA and PCA. The recognition rate on the testing set for this case is
95.7565%.

Figure 8. Comparison of proposed method with 2DPCA, LDA and PCA

We designed a series of experiments for different 2DLDA selected eigenvectors. Table 1
show that the performance of 2DPCA + 2DLDA is optimized at d = 10. The Maximum
recognition rate was 95.7576% obtained using twenty 2DPCA eigenvectors.
The results of performing the presented method on the two database show that the
proposed method has low computational time. It can be used as a real time face recognition
process in places that the time of recognition is important. The average recognition time is
0.5s, approximately. All experiments were performed on a PC (Intel processor, Core i7, 1.73
GHz, 4 GB RAM) and the algorithms was coded in Matlab 2009b, 64 bits [17].
Table 1. Comparison of the top recognition accuracy (%) 2DPCA + 2DLDA for
varying 2DLDAs eigenvectors and 20 2DPCAs eigenvectors

Number of 2DLDAs
selected eigenvectors
Top recognition %

10

12

93.9394 93.3333 93.9394 95.7576 93.9394

5. Conclusion
We have presented a new approach to improve the face recognition accuracy. This method
uses linear phase high-frequency and emphasizes for digital filtering and histogram
equalization for contrast enhancement to achieve improvements. The results obtained using
this method indicate sharper edges and give more details. The novelty of this paper is that the
face recognition is based on the high-frequency emphasis filtering, global thresholding and

International Journal of Advanced Science and Technology

Vol. 52, March, 2013

the combine of 2DPCA and 2DLDA. At the first step, the face images are transformed into
high-frequency emphasis domain. After that, the global thresholding apply to the images,
after we choose the threshold by Otsu's method, for each image and then we multiply it with
255 to find the suitable threshold for our gray scale image. The 2DPCA + 2DLDA are used to
reduce the dimension of face images, which can speed up the recognition algorithm and
minimum distance is used for classifying them. Experiments are conducted on the Yale face
database and AT&T (Olivetti) database (ORL) results demonstrate that the maximum
recognition rate of 95.7576 is achieved.

References
[1] Z. Liu and X. Lu, IEEE Conference on Cross Strait Quad-Regional Radio Science and Wireless Technology,
vol. 2, (2011), pp. 1406.
[2] X. Tan and B. Triggs, IEEE Transactions On Image Processing, vol. 19, (2010), pp. 1635.
[3] B. Zhang, Y. Gao, S. Zhao, J. Clerk Maxwell and Z. Liu, IEEE Transactions On Image Processing, vol. 19,
(2010), pp. 533.
[4] X. Tan and B. Triggs, IEEE Transactions On Image Processing, vol. 18, (2009), pp. 2599.
[5] K. R. Soundar and K. Murugesan, IET Computer Vision, vol. 4, (2010), pp. 173.
[6] X. He, S. Yan, Y. Hu, P. Niyogi and H. Zhang, IEEE Transaction on Pattern Analysis and Machine
Intelligence, vol. 27, (2005), pp. 328.
[7] A. U. Batur and M. H. Hayes, Proceedings IEEE International Conference on Computer Vision and Pattern
Recognition, (2001).
[8] P. N. Belhumeur, J. P. Hespanha and D. J. Kriegman, IEEE Transaction on Pattern Analysis and Machine
Intelligence, vol. 19, (1997), pp. 711.
[9] M. Belkin and P. Niyogi, Laplacian Eigenmaps and Spectral Techniques for Embedding and Clustering,
Proceedings Conference on Advances in Neural Information Processing System, (2001).
[10] M. Belkin and P. Niyogi, Using Manifold Structure for Partially Labeled Classication, Proceedings
Conference on Advances in Neural Information Processing System, (2002).
[11] M. Brand, Charting a Manifold, Proceedings Conference on Advances in Neural Information Processing
Systems, (2002).
[12] Y. Chang, C. Hu and M. Turk, Manifold of Facial Expression, Proceedings IEEE International Workshop
on Analysis and Modeling of Faces and Gestures, (2003).
[13] Y. Su, S. Shan, X. Chen and W. Gao, IEEE Transactions On Image Processing, vol. 18, (2009), pp. 1885.
[14] W. Zhao, R. Chellappa, P. Phillips and A. Rosenfeld, ACM, Computing Survey, vol. 35, (2003), pp. 399.
[15] M. Turk and A. Pentland, Journal of Cognitive Neuroscience, vol. 3, (1991), pp. 71.
[16] A. J. Colmenarez and T. S. Huang, Face Detection and Recognition, Springer-Verlag, (1998), pp. 174-185.
[17] R. Chellapa, C. L. Wilson and S. Sirohey, Human and MachineRecognition of Faces, A Survey,
Proceedings of the IEEE, vol. 83, (1995).
[18] C. Kim and C. H. Choi, Elsevier Pattern Recognition Letters, vol. 40, (2007), pp. 1592.
[19] X. Yuan Jing, H. San Wong and D. Zhang, Elsevier Pattern Recognition Letters, vol. 27, (2006), pp. 1465.
[20] S. Anila and N. Devarajan, European Journal of Scientific Research, vol. 57, (2011), pp. 556.
[21] C. Gurel, Development of a Face Recognition System, Master of science thesis, Atilim University, (2011).
[22] G. Wang and Z. Ou, Face Recognition Based on Image Enhancement and Gabor Features, Proceedings 6th
World Congress on Intelligent Control and Automation, (2006), pp. 9761-9764.
[23] Y. Song, Y. Kim, U. Chang and H. B. Kwon, Pattern Recognition, vol. 39, (2006), pp. 1542.
[24] J. K. Anil, B. M.Ruud, P. Sharath, et al., Personal Identification in Networked Society, Kluwer Academic
Publishers, (1999).
[25] R. C. Gonzalez and R. E. Woods, Digital Image Processing (third edition), Prentice Hall, Upper Saddle
River, NJ, (2008).
[26] J. Zhao, H. Tian, W. Xu and X. Li, IEEE Conference on Intelligent Computation Technology and
Automation, vol. 1, (2009), pp. 499.
[27] B. Y. M. Kwan and H. K. Kwan, World Academy of Science, Engineering and Technology, (2011), pp. 590.
[28] J. C. Russ, The Image Processing Handbook, CRC Press, Boca Raton, FL, (1992).
[29] X. Wen and M. Luan, Journal of Beihua University (Natural Science), vol. 8, (2007), pp. 185.
[30] X. Liu and J. Liu, Computer Engineering and Applications, vol. 44, (2008), pp. 171.
[31] R. Maini and H. Aggarwal, Journal of Computing, vol. 2, (2010), pp. 8.
[32] Y. Bazi, L. Bruzzone and F. Melgani, Elsevier Pattern Recognition Letters, vol. 40, (2007), pp. 619.

International Journal of Advanced Science and Technology

Vol. 52, March, 2013

[33] C. H. Bindu, International Journal of Recent Trends in Engineering, vol. 2, (2009), pp. 88.
[34] S. R. Sulistiyanti, A. Susanto, T. S. Widodo and G. B. Suparta, World Academy of Science, Engineering and
Technology, vol. 48, (2010), pp. 781.
[35] R. M. Mutelo, L. C. Khor, W. L. Woo and S. S. Dlay, IEEE International Symposium on Circuits and
Systems, (2006), pp. 4325.
[36] N. Bhatia and B. Vandana, International Journal of Computer Science and Information Security, vol. 8,
(2010), pp. 302.
[37] W. Zhao, R. Chellappa, P. J. Phillips and A. Rosenfeld, ACM Computing Surveys, vol. 35, (2003), pp. 399.
[38] P. J. Phillips, H. Moon, S. A. Rizvi and P. J. Rauss, IEEE Transaction on Pattern Analysis and Machine
Intelligence, vol. 22, (2000), pp. 1090.
[39] Y. Adini, Y. Moses and S. Ullman, IEEE Transaction on Pattern Analysis and Machine Intelligence, vol. 19,
(1997), pp. 721.
[40] K. Delac, M. Grgic and T. Kos, Sub-Image Homomorphic Filtering Technique for Improving Facial
Identification under Difficult Illumination Conditions, International Conference on Systems, Signals and
Image Processing, (2006), Budapest, Hungary.
[41] V. V. Starovoitov, D. I. Samal and D. V. Briliuk, Image Enhancement for Face Recognition, International
Conference on Iconics, (2003), St. Petersburg, Russia.
[42] H. M. Som, J. M. Zain and A. J. Ghazali, An International Journal on Advanced Computing, vol. 2, (2011),
pp. 61.
[43] N. Otsu, A Threshold Selection Method Form Gray-Level Histograms, Proceedings of the IEEE
Transactions Systems, Man and Cybernetics, (1979), pp. 62-66.
[44] J. Yang, D. Zhang, A. F. Frangi and J.-Y. Yang, IEEE Transactions on Pattern Analysis and Machine
Intelligence, vol. 26, (2004), pp. 131.
[45] J. Yang and C. Liu, IEEE Transactions on Information Forensics and Security, vol. 2, (2007), pp. 781.
[46] J. Ye and Q. Li, A Two-Stage Linear Discriminant Analysis via QR-Decomposition, IEEE Transactionson
Pattern Analysisand Machine Intelligence, vol. 27, (2005), pp. 929.
[47] J. Yang and J. -Y. Yang, Elsevier Pattern Recognition Letters, vol. 35, (2002), pp. 1997.
[48] J. Yang, D. Zhang, X. Yong and J. -Y. Yang, Elsevier Pattern Recognition Letters, vol. 38, (2005), pp. 1125.
[49] L. Sirovich and M. Kirby, Low-Dimensional Procedure for Characterization of Human Faces, Journal
Optical Society of America, vol. 4, (1987), pp. 519.
[50] M. Kirby and L. Sirovich, IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 12, (1990),
pp. 103.

10