Blur and Motion Blur Influence on Face

Recognition Performance
Katarina Knežević, Emilija Mandić, Ranko Petrović, and Branka Stojanović

and deblurred images (blur and motion blur enhancement).

Abstract—Face recognition is still one of the most popular The paper is organized as follows. Section II describes
biometric recognition techniques. It is widely used, both the face recognition algorithm, while Section III describes
online and offline. Performance of such a system is directly blur and motion blur problem space. Sections IV and V
connected to face image quality. Since blur and motion blur
describe the experimental work, including experimental
are common imagery problems, this paper explores the
influence of such disturbances on the face recognition setup, results, and discussion. Section VI lists conclusions
performance. The research described in this paper compares and indicates directions for future work in this research
the performance of the face recognition algorithm based on area.
the Haar features and Local Binary Patterns Histograms
when it uses face images of a good quality, images with added II. FACE RECOGNITION ALGORITHM
Gaussian blur and motion blur, as well as enhanced images. The face recognition process can be divided into three
subcategories: face detection, features extraction, and
Keywords— Face recognition, Gaussian blur, Motion blur, recognition (Fig.1).
Haar feature, LBPH algorithm.

I. INTRODUCTION

F ACE recognition, beside fingerprint recognition, is one

of the most popular biometric recognition techniques.
It is non-invasive, relatively easy to implement from the
system point of view, and very useful both in online Fig.1 Face recognition procedure
applications (real-time face recognition) and offline
applications (for example, search engines for recognizing The system workflow is as follows. The first step, face
persons from images). Nowadays it is widely used in detection [1], separates the person(s) from other image
social networks, like Facebook, Snapchat, Instagram, etc. parts. Features detection afterwards extracts matching
The algorithms for face recognition, the heart of every features that are used for recognition [2]. The last step,
face recognition system, are a well-studied topic in the face recognition, identifies the person.
research community. In order to achieve automatic recognition, a database of
the known face data is required (gallery database) - for
A face image captured and processed both in real-time
each person that needs to be recognized by the system, at
and in offline applications suffers from common
least one face image should be stored in the gallery
disturbances – blur and motion blur. These issues
database. The recognition process includes finding the best
represent a big challenge for scientists today. match (similarity) between the tested image and the
This paper explores the influence of such disturbances images stored in the gallery database.
on the face recognition performance. The experimental setup is based on Python with
The research described in this paper compares the OpenCV library. OpenCV has two pre-trained and ready-
performance of the face recognition algorithm (based on to-use face recognition classifiers: the Haar classifier and
Haar features for face detection and LBPH - Local Binary the LBPH classifier, for person detection and recognition.
Patterns Histograms algorithm for face recognition), when
it uses face images of good quality (original images), A. Face detection algorithm
images with added noise (Gaussian blur and motion blur) Face detection algorithm used in this research is based
on built-in OpenCV face detection function, in the
Cascade Classification Class, which has already been
Katarina Knežević is with the School of Electrical Engineering,
University of Belgrade, 73 Bulevarkralja Aleksandra, 11020 Belgrade, trained to find a face in the image. This function uses Haar
Serbia (e-mail: [email protected]) features [3]. The most common Haar characteristics are
Emilija Mandić is with the School of Electrical Engineering, represented by combining binary variables, which have
University of Belgrade, 73 Bulevar kralja Aleksandra, 11020 Belgrade, been calculated by using several functions.
Serbia (e-mail: [email protected]).
Ranko Petrović is with the Vlatacom Institute of High Technologies, The Haar feature templates shown in Fig.2 are used. For
Blvd. Milutina Milankovića 5, 11070 Belgrade, Serbia (e-mail: calculating features one by one, each window is placed in
[email protected]). the image. Each feature is represented by one value
Dr. Branka Stojanović is with the Vlatacom Institute of High obtained by subtracting pixels at the white rectangle
Technologies, Blvd. Milutina Milankovića 5, 11070 Belgrade, Serbia (e-
mail: [email protected]). location from pixels at the black rectangle location.

978-1-5386-6974-7/18/$31.00 ©2018 IEEE

 comparing each pixel with its neighborhood. First, a
selected window is divided into blocks (eg. 3x3 pixels for
each block), as can be seen in Fig.4. Each pixel in the
block is compared with each of 8 neighbors. Pixels are
arranged along the circle, e.g. in the clockwise direction.
Where the pixel center value is greater than the neighbor's
value, the result is "0", otherwise it is "1". Therefore, from
the 8 surrounding pixels, one can obtain 28 possible
combinations, called Local Binary Patterns or LBP. After
that, the updated pixel values are read clockwise. They are
treated as a number in the binary system. Then, this
number is converted to a decimal number, which
represents a new pixel center value. The procedure is
Fig.2 Common Haar features repeated for each pixel in the image.

Fig.3 Example of Haar features [4]

Fig.3 presents two templates of Haar features, as

example. The first one focuses on the fact that the region
around the eyes is usually darker than the area of the nose
and the face. The second feature relies on the fact that the
eyes are darker than the area of the nose area between the
eyes. Generally, the features are calculated for the image Fig.4 LBPH example [4]
blocks. Most of the features that are obtained are
irrelevant. For example, when features are calculated on The obtained values in blocks are encapsulated in a
the cheek area, the window becomes useless, because none histogram so that each block corresponds to a histogram.
of these surfaces is darker or brighter than other regions on Finally, all histograms are connected to form one vector. It
the cheeks. Consequently, useless features are quickly is shown in Fig.5.
discarded and only meaningful ones are retained. This
technique is called AdaBoost. AdaBoost is a face detection
training process that selects only those characteristics that
are known to improve classification (face / non-face).
Finally, the algorithm takes into account the fact that
most of the region in a picture is a non-face region. With
this in mind, it is a better idea to check if the window
belongs to the face region, and if not, it is immediately
discarded and not processed again, so that the focus is on
the area where the face is. Fig.5 LPBH [4]
B. Face recognition algorithm Histograms define a final vector of labels [7]. Finally,
Face recognition algorithms (classifiers) require to be the resulting histogram connects these block histograms to
trained with known faces images (gallery database). The form one vector of features for one image, which contains
training process extracts recognition features from images all the features of interest.
of known faces (subjects) and labels them.
OpenCV has three following built-in face recognition C. Facial image database
algorithms [5]: In this study, a publicly available database of facial
1. EigenFaces Face Recognizer images has been used in order to test the face detection
2. FisherFaces Face Recognizer and recognition algorithm. It was provided by the authors
3. Local Binary Patterns Histograms (LBPH) Face [8]. The database contains face images of 30 persons. All
Recognizer images have the same dimensions and the same
The research described in this paper uses LBPH (Local orientation. Color (visible light) images subset of this
Binary Patterns Histograms) [6] algorithm. The basic idea database contains 11 images per rotation with different
of this algorithm is reducing the structure in the image by poses for each expression (neutral, smiling, boring and
open mouth) and illumination (daylight, darkness and C. Motion Blur Enhancement Algorithm
three different light sources, frontal, left and right lateral) Motion blur effects are manifested as visible lines
of every individual. This database can be used for different generated by fast movement of an object in front of the
experimental setup in face recognition area [9]. recording device.
Different subsets of this database were used for training A test subset with added motion blur was created by 2D
and testing. Both subsets contain 30 visible light, frontal motion blur filter. One dimension is linear motion of
images (one per each person in the database), taken with a camera (len), and the other is the angle of camera motion
different subject pose - these subsets contain images with (theta). Parameters used for this experiment were len 7 and
neutral and smiling expressions, randomly divided into theta 12.
testing and training subset. A test subset with artificially removed motion blur was
Test subset is used in five different forms: generated in the same manner as the previously described
• original, de-blurring algorithm [12], [13].
• with added Gaussian blur,
• with enhanced Gaussian blur, IV. EXPERIMENTAL WORK
• with added motion blur, A. Experiment Description
• with enhanced motion blur. The algorithm for face recognition tested in this paper
detects faces by the Haar features and then do the
III. BLUR AND MOTION BLUR
recognition using the LBPH algorithm. The algorithm first
A. Common imagery problems was tested with original images, then with images
Image noise is an unwanted information on a digital containing artificially added motion blur and Gaussian
image in the form of a random variation of color or blur. Afterwards it was also tested with image subsets with
brightness. Image noise is an undesirable by-product of enhanced blur (Fig.6).
image capture that obscures the desired information. It can
be produced by the sensor, or when a photographer moves
accidentally his hand while taking a picture [10]. One of
the image noise effects is occurrence of a blur. Image blur
is very common in natural photos, originating from
different factors, like atmospheric disturbances, object
motion, out of focus camera and camera shaking. Motion
blur is caused by sudden movement of a sensor, or fast
movement of an object, during exposure time.
B. Blur Enhancement Algorithm
In image processing field, the most common blur
representation is the Gaussian blur or Gaussian smoothing. Fig.6Original image, image with blur(motion, Gaussian) and de-
It is the result of blurring an image by a Gaussian function blurred images
[11].
Test subset with added blur (blur subset) is created B. Evaluation Methodology
using the 2D Gaussian smoothing kernel with standard The results are presented with ROC (Receiver
deviation of 2. operating characteristic) and CMC (Cumulative match
A test subset with enhanced Gaussian blur has been characteristic) curves.
generated from the blur subset with the method for ROC is a graphic representation illustrating the
recovering the blur kernel [12], which is based on diagnostic ability of the binary classifier system [14]. It is
statistical irregularities. created by representing the true positive rate against the
This model is used together with an accurate spectral false positive rate. In addition, it can be considered as the
whitening formula to estimate the power spectrum of the number of wrong recognized samples.
blur. The blur kernel is then recovered using a phase CMC curve is a graphic representation illustrating the
retrieval algorithm with improved convergence and performance of the algorithm based on cumulative results
disambiguation capabilities. Unlike methods that rely on – Rank n accuracy [15]. The test image face is recognized
the presence and the identification of well separated edges correctly at Rank n if the database face of the same person
in the image, this statistical approach copes well with is between the first n faces in the sorted database. Rank 1
images containing under-resolved texture and foliage is the most important performance indicator for the
clutter. The described method do not reconstruct the latent recognition on CMC curve.
image repeatedly and access the input image only once to This paper presents results for all five test subsets,
extract a small set of statistics - the core of this technique described previously – ROC and CMC curves, and area
depends only on the blur kernel size and does not scale under the ROC curve (AUC) and Rank 1 accuracy.
with the image dimensions. Therefore, it is not
computationally complex. V. RESULTS AND DISCUSSION
Accurate estimation of blur kernel and minimization Fig.7 presents ROC curve and Fig.8 presents CMC
problem is solved by the linearized Bergman iteration curve calculated on subsets with Gaussian and motion
[13]. blur.
 Fig.7 ROC curves comparison for Gaussian and motion blur Fig.9 ROC curves comparison for experiment with Gaussian blur

Fig.8 CMCcurves comparison for Gaussian and motion blur Fig.10 CMC curves comparison for experiment with Gaussian
blur

The CMC and ROC curves indicate that the system The area under the ROC curve (AUC) and Rank 1
shows the best performance in tested cases for subset with accuracy for Gaussian blur enhanced images are presented
original face images, as expected. The algorithm shows in Table II. All performance indicators are presented for
lower accuracy in the presence of the Gaussian blur and three different subsets: (i) original images, (ii) Gaussian
motion blur. For these two subsets of images, although the blur and (iii) enhanced images.
results are quite similar, the Gaussian blur is detected as
the harder problem. TABLE II: AUC AND RANK1 ACCURACY FOR GAUSSIAN BLUR
The area under the ROC curve (AUC) and Rank 1 EXPERIMENT
accuracy are shown in Table I as the most important Original Gaussian Enhanced
indicators of the face recognition algorithm performance. images blur Gauss. blur
These results give the same conclusions as in the case of AUC 0.9434 0.9204 0.9071
ROC and CMC curves. Rank1 0.9333 0.5600 0.7467

TABLE I: ACCURACY FOR DIFFERENT BLUR – AUC AND RANK1 The algorithm performance for motion blur experiment
is presented on Fig.11 with ROC curve and on Fig.12 with
Original Gaussian Motion blur CMC curve.
blur The area under the ROC curve and Rank 1 accuracy for
AUC 0.9434 0.9204 0.9281 motion blur experiment are shown in Table III.
Rank1 0.9333 0.5600 0.7467
TABLE III: AUC AND RANK1 ACCURACY FOR MOTION BLUR
Algorithm performance was then tested on enhanced EXPERIMENT
(improved) Gaussian blur images subset. The results are Original Motion blur Enhanced
presented on Fig.9 with ROC curve and on Fig.10 with images Motion blur
CMC curve. AUC 0.9434 0.9281 0.9093
Rank1 0.9333 0.7467 0.8400
 problem for face recognition compared to motion blur
presence on images. The reason for this is the fact that
edges are more visible in images with motion blur,
comparing to Gaussian.
The second conclusion is that blur and motion blur
negatively influence the face recognition performance, as
expected.
The unique conclusion from CMC and ROC
performance indicators is that the enhanced images more
likely will be recognized by biometric system, comparing
to blurred images, with the cost of more false alarms.
Fig.11ROC curves comparison for experiment with motion blur Our future work in this area will include more image
disturbances types and new techniques for image
enhancement.

REFERENCES
[1] P. Viola and M. Jones, "Robust real-time face detection,"
International journal of computer vision, vol. 57, no. 2, pp. 137-154,
2004.
[2] P. Kalaiselvi and S. Nithya, "Face Recognition System under
Varying Lighting Conditions," IOSR Journal of Computer
Engineering (IOSRJCE), vol. 14, no. 3, pp. 79-88, 2013.
[3] I. P. Wilson and J. Fernandez, "Facial Feature Detection Uusing
Haar Classifiers," in CCSC: South Central Conference, 2006.
[4] K. Knezevic, "The usage of projection mapping and augmented
reality for communication through creative design of clothes," BSc
Fig.12CMC curves comparison for experiment with motion blur Thesis, ETF, University of Belgrade, Belgrade, 2018.
[5] "OpenCV," [Online]. Available:
All the presented results indicate that blur and motion https://docs.opencv.org/2.4/index.html. [Accessed 19 09 2018].
blur negatively influence the face recognition [6] J. Chang-yeon, "Face Detection using LBP features," CS 299 Final
performance, as expected. Project Report, 2008.
The CMC curves and Rank 1 accuracy for both [7] T. Ahonen, A. Hadid and M. Pietikäinen, "Face description with
local binary patterns: Application to face recognition," IEEE
experiments (Gaussian and motion blur experiments) Transactions on Pattern Analysis & Machine Intelligence, vol. 12,
indicate that the tested enhancement method [12] [1], pp. 2037-2041, 2006.
improves the face recognition performance, compared to [8] "IEEE OTCBVS WS Series Bench; DOE University Research
the performance in the case of blurred images. Program in Robotics under grant DOE-DE-FG02-86NE37968;
DOD/TACOM/NAC/ARC Program under grant R01-1344-18;
The ROC curves and AUC give, at the first glance, FAA/NSSA grant R01-1344-48/49; Office of Naval Research under
contradictory results –the face recognition performance grant #N000143010022.," [Online]. Available: http://vcipl-
seems to decrease after blur enhancement. One should okstate.org/pbvs/bench/.
have in mind that these two types of indicators (CMC and [9] M. Pavlovic, R. Petrovic, B. Stojanovic and S. Stankovic, "Facial
expression and lighting conditions influence on face recognition
ROC) are related to different face recognition aspects. performance," in TELFOR, Belgrade, 2018.
CMC and Rank 1 accuracy indicate that the enhanced [10] A. K. Boyat and B. K. Joshi, "A Review Paper: Noise Models in
images will more likely be recognized by the biometric Digital Image Processing," Signal & Image Processing: An
system, compared to the blurred images. International Journal (SIPIJ), vol. 6, pp. 63-75, April 2005.
ROC and AUC indicate that the cost for this [11] S. Sharma and B. Sharman, "Identification of Face across Random
Motion Blur, Illumination, and Pose," International Journal of
improvement will be an increase in the number of false Computer Applications(0975-8887), vol. 154, no. 2, pp. 17-20,
alarms. 2016.
The CMC curves, for these reasons, are commonly [12] A. Goldstein and R. Fattal, "Blur-Kernel Estimation from Spectral
referred to as the most important indicators for biometric Irregularities," in European Conference on Computer Vision
recognition systems’ performance. (ECCV), Berlin, 2012.
[13] J. Cai, S. Osher and Z. Shen, "Linearized Bregman iterations for
VI. CONCLUSION frame-based image deblurring," SIAM Journal on Imaging Sciences,
vol. 2, no. 1, pp. 226-252, 2009.
Face recognition performance in biometric systems [14] R. M. Bolle, J. H. Connell, S. Pankanti, N. K. Ratha and A. W.
directly depends on face image quality. This paper Senior, "The relation between the ROC curve and the CMC,"
explores the influence of blur and motion blur on face Automatic Identification Advanced Technologies. Fourth IEEE
Workshop, pp. 15-20, 2005.
recognition performance. Face recognition algorithm used
[15] S. Paisitkriangkrai, C. Shen and A. V. D. Hengel, "Learning to rank
in the experimental part of the work is based on the Haar in person re-identification with metric ensembles," in Proceedings
features and LBPH. of the IEEE Conference on Computer Vision and Pattern
Two sets of experiments – Gaussian blur and motion Recognition, 2015.
blur experiments have shown similar outcome and led to
following conclusions.
The first conclusion is that Gaussian blur is a harder