2014 4th International Conference on Engineering Technology and Technopreneuship (ICE2T)

A Comparative Study between LBP and Haar-like

features for Face Detection Using OpenCV
Kushsairy Kadir #1, Mohd Khairi Kamaruddin *1.Haidawati Nasir#2, Sairul I Safie#3,
Zulkifli Abdul Kadir Bakti#1
#1
British Malaysian Institute, Universiti Kuala Lumpur, Selangor
#2
Malaysian Institute of Information Technology, Universiti Kuala Lumpur, Kuala Lumpur
#3
Malaysian Institue of Industrial Technology, Universiti Kuala Lumpur, Johor

[email protected]

Abstract—Face Detection is an important step in any face which allows background regions of the image to be quickly
recognition systems, for the purpose of localizing and extracting discarded while spending more computation on promising
face region from the rest of the images. There are many face-like regions.
techniques, which have been proposed from simple edge Local Binary Patterns proposed by Timo Ojala et al [4] has
detection techniques to advance techniques such as utilizing
been extensively used for the face detection algorithm. Hadid
pattern recognition approaches. This paper evaluates two
methods of face detection, her features and Local Binary Pattern et al [5] was the first to apply LBP histogram features in face
features based on detection hit rate and detection speed. The detection, showing exceptionally high discriminative power in
algorithms were tested on Microsoft Visual C++ 2010 Express face or non-face classification.
with OpenCV library. The experimental results show that Local This paper focused on comparative studies of two face
Binary Pattern features are most efficient and reliable for the detection algorithm. The remainder of this paper is organized
implementation of a real-time face detection system. as in Section II discusses the methodology of the algorithms
and Section III shows the result of the testing. For the testing
I. INTRODUCTION the same training set and samples are used.
Face detection is a technique of using any various methods
to detect a face in an image or video. Nowadays there are II. METHODOLOGY
many devices and websites which use face detection Every face detection algorithm has different operation and
technology to detect a face such as Facebook that offer to tag procedures to detect a face. This section explains the
friends on uploaded images and then will automatically tags components of both algorithms and step by step towards its
the face whenever the face appear in any of the photos. function. First, is the face detection using Haar-like features,
Currently, we can find that most laptops come with webcams and second, is the face detection using Local Binary Pattern
that utilize software that has a face detection function. features.
Therefore, face detection is not new in the computer vision
industry. A. Haar-like Features
Face detection is a computer technology that determines the The features that Viola and Jones used are based on Haar
locations and sizes of human faces in arbitrary (digital) wavelets [6]. Haar wavelets are single wavelength square
images. Face detection can be regarded as a specific case of waves (one high interval and one low interval). In two
object-class detection. In object-class detection, the task is to dimensions, a square wave is a pair of adjacent rectangles, one
find the locations and sizes of all objects in an image that light and the other is dark. This method is quite similar to
belongs to a given class. binary method that separates the obvious different:
One of the classical techniques for face detection was
presented by Paul Viola and Michael Jones [1] which use
weak classifier cascade for face detection. There are four
components in the technique proposed by Viola and Jones.
First, is to generate Haar-like features [2] that are digital
image features used in object recognition. Second, is the
introduction of a new image representation called the Integral
Image [1] which allows the features used by the detector to be
computed very quickly. The third component is a simple and
efficient classifier which is built using AdaBoost [3] learning
algorithm to select a small number of critical visual features Figure 1: Haar-like Features[7]
from a very large set of potential features. The fourth
component is a method for combining classifiers in a cascade

2014 4th International Conference on Engineering Technology335

and Technopreneuship (ICE2T)
 2014 4th International Conference on Engineering Technology and Technopreneuship (ICE2T)

The actual rectangle combinations used for visual object

detection are not true Haar wavelets. Instead, they are the
modified rectangle combinations better suited to visual
recognition tasks. Because of that difference, these features
are called Haar features or Haar-like features [7].
The presence of a Haar-like feature is determined by
subtracting the average dark-region pixel value from the
average light-region pixel value [8]. If the difference is above
the threshold (set during learning), that feature is said to be
present. Examples of Haar-like features are shown in Figure 1.

Figure 3: Integral image

The blue matrices represent the original images, while the

purple ones represent the images after the integral
transformation. If we were to compute the shaded area in the
first image, we would have to sum all pixels individually,
reaching the answer of 20 after about six memory accesses.
Using the integral image, what is needed is a single access
(but this only because we were in the border). In case we are
Figure 2: Haar-like features application. not in the border, all it would require would be at max four
array accesses, independently of the size of the region. So, for
To detect a face by using Haar-like features, it can come the scanning process, we no longer use every tiny pixel, we
one feature per image or ten features or even hundreds will make the process using integral images.
features per image as shown in Figure 2. The number of
features depends on the structure, occlusion and skin tone of
the face in an image.
To determine the presence or absence of hundreds of Haar-
like features at every image location and at several scales
efficiently, Viola and Jones used a technique called an Integral
Image [1]. In general, integrating means adding small units
together, which in this case the small units are pixel values. Figure 4: Integral Image
The integral value for each pixel is the sum of all the pixels
above it and to its left. Starting from the top left, and The value at each pixel location, (x,y), contains the sum of
traversing to the right and down, the entire image can be all pixel values within a rectangular region that has one corner
integrated with a few integer operations per pixel [8]. at the top left of the image and the other at a location (x,y). To
The idea is to compute all sums in the very beginning and find the average pixel value in this rectangle, divide the value
save them for future computations. This can be done by at (x,y) by the rectangle’s area. Conveniently, A+B+C+D is
forming a summed area table for the frame being processed, the Integral Image’s value at location 4, A+B is the value for
also known as computing its integral image representation. In location 2, A+C is the value for location 3 and A is the value
an integral image, the area of any rectangular region in the for location 1. So, with an Integral Image, the sum of pixel
image can be computed by using only four array accesses [9]. values for any rectangle in the original image with three
The Figure 3 shows the example. integer operations: (x4,y4) – (x2,y2) – (x3,y3) + (x1,y1) or
D=A+B+C+D-(A+B)-(A+C)+A.
B. Local Binary Patterns
Local Binary Patterns (LBP) [4] [10] is a simple yet very
efficient texture operator which labels the pixels of an image
by thresholding the neighbourhood of each pixel with the
value of the centre pixel and considers the result as a binary
number. Due to its discriminative power and computational
simplicity, LBP texture operator has become a popular
approach to various applications.
The LBP texture method has provided excellent results in
various applications. Perhaps the most important property of

336
 2014 4th International Conference on Engineering Technology and Technopreneuship (ICE2T)

the LBP operator in real-world applications is its robustness to

monotonic gray-scale changes caused, for example, by
illumination variations. Another important property is its
computational simplicity, which makes it possible to analyse
images in challenging real-time settings.
The LBP texture analysis operator, introduced by Ojala et
al [4], is defined as a grayscale invariant texture measure,
derived from a general definition of texture in a local
neighbourhood. The original LBP operator forms labels for
the image pixels by thresholding the 3 x 3 neighbourhood of
each pixel with the centre value and considering the result as a
binary number. The histogram of these 28 = 256 different
labels can then be used as a texture descriptor [11]. Figure 7: Example of texture primitives [13]

Each bin (LBP code) can be regarded as a micro-texton

[13]. Local primitives which are codified by these bins include
different types of curved edges, spots, flat areas, line end or
corners.

III. RESULTS
The algorithms were tested on Microsoft Visual C++ 2010
Express with OpenCV library. This testing running on
Windows 7 64bit with Intel Core i3 M380 (2.53GHz), 6GB
Figure 5: The basic LBP operation RAM and SATA Hard Disk. The settings for the algorithm
testing are using 1.1 scale factor with four minimum
neighbours threshold and not using any flag, with 25x25
minimum detection scale. Hear and LBP had already been
tested by using multiple sets of face database:
i) The Color FERET Database, NIST USA [14] with
1127 images.
ii) MIT CBCL Database [15] with 2000 images
Figure 6: Neighbourhood set for different (P,R). The pixel iii) Taarlab Face Database, Brown University [16] with
values are bilinear interpolated whenever the sampling point is 759 images.
not in the centre of a pixel.
All data sets that have been selected are focus on frontal
The operator has been extended to use neighbourhoods of image and with 30 degree occlusion to right and left.
different sizes. Using a circular neighbourhoods and bilinear
interpolating values at non-integer pixel coordinates would TABLE 1
allow any radius and number of pixels in the neighbourhood Haar Result
[11]. In the following, the notation (P,R) will be used for pixel Algorithm Dataset Detected Hit Rate Detection
neighbourhood, which means P sampling points on a circle of Faces Speed (ms)
radius of R. Another extension of the original operator is the Haar Color 976/1127 86.6% 235.4117
definition of so called uniform patterns [12]. This extension FERET
was inspired by the fact that some binary pattern occurs more MIT 1670/2000 83.5% 255.5048
commonly in texture images than others. A LBP is called Taarlab 647/759 85.2% 231.5865
uniform [10] if the binary pattern contains at most two bitwise
Overall 3293/3886 84.7% 241
transitions from 0 to 1 or vice versa when the pit pattern is
traversed circularly. For example, the patterns 00000000 (0
TABLE 2
transitions), 01110000 (2t transitions) and 11001111 (2
LBP Result
transitions) are uniform whereas the patterns 11001001 (4
Algorithm Dataset Detected Hit Rate Detection
transitions) and 01010011 (6 transitions) are not. In the
Faces Speed (ms)
computation of the LBP labels, uniform patterns are used so
that there is a separate label for each uniform pattern and all LBP Color 1004/1127 89% 95.44924
the non-uniform patterns are labelled with a single label [11]. FERET
For example, when using (8,R) neighbourhoods, there are a MIT 1779/2000 89% 101.8864
total of 256 patterns, 58 of which are uniform, which yields in Taarlab 674/759 88.8% 104.9335
59 different labels. Overall 3457/3886 89% 101

337
 2014 4th International Conference on Engineering Technology and Technopreneuship (ICE2T)

The color FERET data set is a mix database that consists of The above two experiments result show that two algorithms
different skin tone faces, faces with glasses, various racial of face detection can detect face accurately and reach the
faces and 30 degrees of face left and right occlusion with requirement if there are faces in video. In other words, the
semi-controlled environment. effectiveness of both algorithms almost are the same if the
Table 1 and Table 2 shows the results of Haar and LBP data sets of sequential static image are used to detect faces.
algorithms tested on the three databases. The result shows for Although LBP detected 4.3% more faces than Haar, the
Haar algorithm the detection rate for Color FERET database is result outputs for both algorithms are almost the same which
84.7% compare to LBP detection rate, which is 89%. The both algorithms detected almost all subjects in these three data
result shows that LBP detected more faces by the difference of sets. Both algorithms do not have detection problems with
3.4% more hit rate than Haar. Haar and LBP have the same racial faces, hairy faces, difference skin tone faces or faces
result in number of subjects detected which both algorithm with glasses.
missed one subject totally. Half of Haar undetected images are Based on the results of the experiment, the LBP face
faces with 30 degree left occlusion which means that Haar detection algorithm is the most reliable algorithm to find faces
vulnerable in face with left occlusion detection. in images because LBP detected 4.3% more faces than Haar
MIT face database is a grayscale face database with a wide on hit rate aspect and LBP detection speed is 140% faster than
range of background light. It contains face images of 10 has detection speed on the overall result. This could be a huge
subjects (200 for each subject). LBP reaches the same hit rate advantage in the implementation on embedded system for
(89%) as it had been used on Color FERET database while real time face detection.
Haar has a lower hit rate percentage (83.5%) on MIT data set
compared to it’s detection on Color FERET database. The IV. CONCLUSION
result from this data set means that LBP is more robust than This paper discusses two methods or algorithms that have
Haar in a wide range of environment such as background light. been used in face detection, Haar features and Local Binary
Patterns (LBP). Comparative study has been made between
these two algorithms by using three face databases as samples
which are Color FERET database, MIT CBCL database and
Taarlab database. All three databases provide single face
images with various types of faces. Results have been
evaluated based on detection speed for every image and
detection’s hit rate which leads to accuracy.
Based on overall results, it can be concluded that LBP has
overall faster detection speed compared to Haar by 140%.
LBP also detected more faces than Haar by extra 4%. From
here, we can say that LBP is better than Haar in the aspect of
detection accuracy and detection speed.

ACKNOWLEDGMENT
This study is supported by Majlis Amanah Rakyat (MARA)
under Skim Geran Penyelidikan Institusi MARA.

REFERENCES
[1] Paul Viola and Michael Jones, Rapid Object Detection using a Boosted
Cascade of Simple Features. Accepted Conference on Computer
Vision and Pattern Recognition, 2001.
[2] Constantine P., M. Orent T. Poggio. A General Framework for Object
Figure 8: Sample results taken from Color FERET data set
Detection. International Conference on Computer Vision, 1998.
testing using LBP algorithm. [3] Y. Freund and R.E. Schapire. A Decision-Theoretic Generalization of
on-Line and an Application to Boosting. Journal of Computer and
Figure 8 shows the sample of result that was taken from System Sciences, 1997.
[4] T. Ojala, M. Pietikainen, and D. Harwood, A comparative study of
Color FERET data set testing by using LBP algorithm. Figure
texture measures with classification based on feature distributions,
8 also shows that LBP algorithm is capable to detect faces Pattern Recognition, 1996.
with different skin tone, hairy face, faces with glasses and [5] A. Hadid, M.Pietikainen and T.Ahonen. A discriminative feature space
different racial face’s structure. for detecting and recognizing aces. In Proceedings of the 2004 IEEE
Taarlab database includes multiple images for over 200 computer society conference on Computer vision and pattern
recognition, Washington, DC, USA, 2004, CVPR ’04, pp. 797-804,
individuals of many different races with majority of Asian IEEE Computer Society.
faces. This database has a consistent lighting with various [6] A. Haar, For Orthogonal Function Systems Theory, German
emotions and even wearing sunglasses. LBP still lead the Mathematical Research Journal, 1910.
detection hit rate percentage by 3.6% more than Haar [7] R. Lienhart and J. Maydt. An Extended Set of Haar-like Features for
Rapid Object Detection, 2004.
detection hit rate in testing with Taarlab data set. [8] R. Hewitt, 2007 “Seeing with OpenCV: How Face Detection Works”,
webpage on Cognotics, Resources for Cognitive Robotics [Online].

338
 2014 4th International Conference on Engineering Technology and Technopreneuship (ICE2T)

Available: : [12] T. Ojala and M. Pietikainen, Multiresolution Gray-Scale and Rotation

http://www.cognotics.com/opencv/servo_2007_series/part_2/sidebar.ht Invariant Texture Classification with local Binary Patterns, Pattern
ml Analysis and Machine Intelligence, 2002.
[9] C. Souza, 2012, “Haar-feature Object Detection in C#”, webpage on [13] J. Chang-yeon, Face Detection using LBP features, CS 229 Final
CODE PROJECT [Online]. Available: Project Report, 2008.
http://www.codeproject.com/Articles/441226/Haar-feature-Object- [14] “The Color FERET Database”, webpage on NIST [Online]. Available:
Detection-in-Csharp http://www.nist.gov/itl/iad/ig/colorferet.cfm/.
[10] T. Ahonen, A. Hadid, and M. Pietikainen, Face description with local [15] 2004, “Stimulus images courtesy” of M. J. Tarr, Center for the Neural
binary patterns: Application to face recognition, IEEE Transactions on Basis of Cognition and Department of Psychology, Carnegie Mellon
Pattern Analysis and Machine Intelligence, 2006. University (Brown University), webpage on The CNBC Wiki [Online].
[11] A. Hadid, G. Zhao, T. Ahonen, and M. Pietikainen, Face analysis Available:http://cbcl.mit.edu/software-datasets/heisele/facerecognition-
using Local binary Patterns, World Scientific Volume, 2008, pp.347- database.html/.
350. [16] 2008, “The MIT CBCL Face Recognition Database”, webpage on MIT
CBCL [Online]. Available: http://wiki.cnbc.cmu.edu/Face_Place/

339