Face recognition system on Raspberry Pi

Olegs Nikisins, Rihards Fuksis, Arturs Kadikis, Modris Greitans

Institute of Electronics and Computer Science,
14 Dzerbenes Street, Riga, LV 1006, Latvia

Abstract
An embedded face recognition system based on the Raspberry Pi single-board
computer is proposed in this paper. Face recognition system consists of face
detection and face localization using Haar feature-based cascade classifier. Face
features are extracted using weighted Local Binary Pattern algorithm. Developed
system performs one full face analysis in 110 ms. Comparison of two biometric
samples is performed in 2 ms. The proposed embedded face recognition system
was tested on FERET database and achieves accuracy of CMC: 99.33% and
EER: 1%.
Keywords: Face recognition, Raspberry Pi, Local Binary Pattern.

1. Introduction
Real time human identification systems are important for security, surveillance
and biometric applications. Usually it is desirable to detect, track and recognize
persons in public areas such as airports, shopping centres, in areas with restricted
access such as private offices, houses etc. Human identification can be
performed by analysing its biometric information, such as fingerprints, face, iris,
palm prints, palm veins etc. However, for fast and convenient person recognition,
still the most suitable biometric parameter is facial information.
Identification of humans by using facial biometrics is still challenging task, due
to the variable illumination, changing facial expressions according to mood
changes, head orientation and pose. Over the years, various face detection
algorithms have been developed. Some face recognition methods analyse the
geometric features of facial images, such as location and distance between nose,
eyes, and mouth [5] [3]. However, these methods are sensitive to the changes in
illumination and facial expression. Because of this drawback, most of the face
recognition systems try to extract some holistic features from the original face
images for matching. By using holistic methods face is recognized using
descriptions based on the entire image rather than on local features of the face [6].
Many subspace learning based holistic feature extraction methods have been
developed, including Eigenfaces [12], Fisherfaces [2], 2D PCA [14] and others.
In this paper we describe the holistic method called local binary pattern (LBP)
[9]. In this paper we propose an embedded face recognition system that can be
used as a system to control electromagnetic door lock of the doors, to recognize
persons at boarder or elsewhere.
 Fig.1: Face recognition steps

The paper is divided in two main parts: section 2 explains the face detection and
recognition algorithm and section 3 explains the implementation and the
information about how the system was built. At the end we introduce the
experimental setup and the results and give conclusions about this work.

2. Face recognition algorithm

Fig.2: The sum of the pixels within rectangle D can be computed with four array
references. (A+B +C +D)+(A) - ((A + B) + (A + C))

Fig.3: Coefficients for face region extraction

The proposed algorithm for face recognition can be divided into several steps.
The sequence of steps of the proposed algorithm used for face recognition is
shown in Fig. 1. The first step is to acquire the image. Next, face detection has to
be performed, to find whether the face appears in the captured image or not. The
next step is to locate the position of the face in the image. Face detection and
face localization is per- formed by using Haar feature-based cascade classifier [8]
[9]. The rectangular features needed for Haar classifier are computed using an
intermediate representation for the image that is called an integral image [8].
 Fig.4: LBP example of 3x3 neighborhood, (P = 8, R = 1)

Fig.5: Labelling principle

The integral image at location x; y contains the sum of the pixels above and to
the left of x; y, inclusive:
i( x, y) = f ( x, y)
x x, y y
(1)

, where i(x,y) is the integral image and f(x,y) is the original image. Using the
following pair of recurrences:
s( x, y) = s( x, y - 1) + f ( x, y) (2)
i( x, y) = i( x - 1, y) + s( x, y) (3)

(where s(x, y) is the cumulative row sum, s(x,-1) = 0, and i(-1; y) = 0) the
integral image can be computed in one pass over the original image [8].
Using the integral image any rectangular sum can be computed by referencing
four array locations (shown in Fig. 2). Difference between two rectangular sums
can be computed in eight references. Since the two-rectangle features defined
above involve adjacent rectangular sums they can be computed in six array
references, eight in the case of the three- rectangle features, and nine for four-
rectangle features [8].
If the image contains a face, the algorithm returns a rectangle with coordinates
where face was found. However, it is not the final region of interest (ROI) that
we use. To calculate the necessary ROI, we use the coordinates of a rectangle
and recalculate the ROI position. We use FERET face image database for
algorithm evaluation that contains many frontal face images with their respective
eye coordinates. Using the obtained rectangle from the Haar classifier and the
known eye positions, we developed a method to statistically determine where in
the image the eyes are most likely to be located.
 Fig.6: Example of LBP transform and formation of
face feature histogram from 6x6 regions

Fig.7: Hardware block scheme

We use this method to determine the approximate location of the persons eyes
to calculate the ROI based of the distance between the eyes d. Distance to each
of the sides is calculated by coefficients shown in Fig. 3. However, our approach
does not localize the position of the eyes and therefore cannot correct the
rotations of the face.
After the ROI is found, we perform LBP transformation. LBP labels each pixel
by thresholding the neighbourhood pixels with central pixel value and represents
the result as a bi- nary number. An example of the labelling procedure for 3x3
region is shown in Fig. 4. Binary value usually is written as a decimal number
and stored in the centre pixel position of the output image. After LBP transform
a histogram of labels is used as a descriptor of the image. LBP operator can be
per- formed by using different amount of pixels P on the sampling circle and
different radius R from the central pixel. Example of radius R = 1 and R = 2 with
P = 8 is shown in Fig. 5. A histogram of the labelled image fl(x; y) can be
calculated as follows:

H i = I { fl ( x, y) = i}, i = 0,K, n - 1,
x, y
(4)

where n = 2P is the number of different labels and

1, A = true
I { A} = (5)
0, A = false
In order to save the spatial information about the object, LBP transformed image
is divided into several sub-regions R0,R1,,Rm1. A spatially enhanced histogram
is calculated by concatenating the region histograms into a single feature
histogram:
 H i , j = I { fl ( x, y) = i}, I {( x, y) R j } (6)
x, y
An example of LBP transform and face feature histogram forming with 6x6
regions of LBP image is shown in Fig. 6.
Feature histogram is used as a descriptor for each of the given images. To
compare two feature histograms many methods can be used. For example,
correlation of the histograms, Chi-Square [10], histogram intersection [11],
Bhattacharyya distance [12] and other methods. In our algorithm we chose
histogram intersection for two histogram comparison. Research on using LBP
transform for face recognition from which this implementation was derived is
published in [13].

3. Face recognition system

Fig.8: Developed face recognition system

In the heart of the embedded face recognition system is a Raspberry Pi single-

board computer, which controls all of the peripherals. The board has a 700MHz
ARM CPU and 512 MB of RAM. For image acquisition we use a USB web
camera with a 320x240 resolution. To visualize the information and provide user
friendly identification procedure, we use mini LCD. A control PCB is developed
for voltage con- version and also for electrical door magnet control. Block
diagram of the system is shown in Fig. 7. As an enclosure we have developed a
custom-made metal enclosure for Raspberry Pi, web camera, control board and
mounting of LCD. Developed system is shown in Fig. 8. Raspberry Pi runs the
Raspbian OS which is a Linux operating system derived from Debian. Our
program is written in C++ and uses the OpenCV library for image acquisition
and face detection. The developed recognition software can be used as a
standalone or a part of a multimodal biometric system. We have developed the
data exchange interface so the system could connect to other biometric system to
ex- change biometric data. When connected with another biometric system, face
recognition system can work as a slave module in capture-send mode. In slave
mode system waits for the start signal, performs image acquisition and
processing and then sends the acquired data to the master system. Biometric
research group has developed the palmprint and palm vein biometric system that
acts as a master and controls the face recognition system [14].
4. Experimental setup and results

Fig.9: Cumulative Match Characteristic

Fig.10: Receiver Operating Characteristic

For evaluation of performance we use two parameters. One is equal error rate
(ERR). The rate at which both accept and reject errors are equal. The value of the
EER can be easily obtained from the receiver operating characteristic (ROC)
curve. The EER is a quick way to compare the accuracy of devices with different
ROC curves. The lower the rate, the lower the error of the system. The other one
is cumulative match characteristic (CMC), a CMC curve plots the probability of
identification against the rank. The greater the rate at rank 1, the higher the
accuracy of the system.
To evaluate the performance of the system we use publicly available FERET
database. Probe set and gallery set form the FERET database are combined. The
combined data contains data from 992 persons, 2 samples from each person, for
a total of 1986 images. 25 % of persons are used for the algorithm training and
the other 75 % are used for testing. During the training process, the optimal
values for three parameters were found by performing the exhaustive search. The
first parameter is the resolution of the ROI, the second one is the radius for the
LBP transform and the third one is the amount of regions for the histogram
calculation. Optimal parameters are ROI width = 110, R = 4, m = 5.
Using the FERET database we achieved the results of CMC: 99.33% and EER:
1%. CMC curve is shown in Fig. 9 and ROC curve is shown in Fig. 10.
5. Conclusion
Embedded face recognition system based on Raspberry Pi single-board computer
is introduced in this paper. Paper is divided in two parts - the software part and
the hardware part. Software part describes the algorithms for face detection,
localization, feature extraction and recognition. Hardware part describes how the
system was built and what modules does it use. System was built with an option
to connect with other biometric systems such as palmprint and palm vein
biometric system, which is described in [14]. By combining several biometric
parameters we can obtain higher accuracy of identification of authentication and
also higher resistance to counterfeiting, because it is always harder to falsify 3 or
more parameters than just one. We have also developed a smart card applet that
can store and compare biometric data. If biometric data is compared on the card
(Match on Card principle) and it is also encrypted we can ensure that the original
data is always kept on the card and wont be set outside, therefore, eliminating
the chance of data interception.

References
[1] Ingemar J. Cox, J. Ghosn, and P.N. Yianilos, Feature- based face
recognition using mixture-distance, in Com- puter Vision and Pattern
Recognition, 1996. Proceed- ings CVPR 96, 1996 IEEE Computer Society
Confer- ence on, 1996, pp. 209216.
[2] Paola Campadelli and Raffaella Lanzarotti, A face recognition system based
on local feature characteri- zation., in Advanced Studies in Biometrics,
Massimo Tistarelli, Josef Bign, and Enrico Grosso, Eds. 2003, vol. 3161 of
Lecture Notes in Computer Science, pp. 147152, Springer.
[3] Keun-Chang Kwak and W. Pedrycz, Face recognition using an enhanced
independent component analysis ap- proach, Neural Networks, IEEE
Transactions on, vol. 18, no. 2, pp. 530541, 2007.
[4] M. Turk and A. Pentland, Eigenfaces for recognition, in Journal of
Cognitive Neuroscience, 1991, vol. 3, pp. 7186.
[5] P.N. Belhumeur, J.P. Hespanha, and D. Kriegman, Eigenfaces vs.
fisherfaces: recognition using class spe- cific linear projection, Pattern
Analysis and Machine Intelligence, IEEE Transactions on, vol. 19, no. 7, pp.
711720, 1997.
[6] Jian Yang, D. Zhang, A.F. Frangi, and Jing-Yu Yang, Two-dimensional pca:
a new approach to appearance- based face representation and recognition,
Pattern Analysis and Machine Intelligence, IEEE Transactions on, vol. 26, no.
1, pp. 131137, 2004.
[7] Timo Ojala, Matti Pietikinen, and David Harwood, A comparative study of
texture measures with classifica- tion based on featured distributions,
Pattern Recogni- tion, vol. 29, no. 1, pp. 51 59, 1996.
[8] P. Viola and M. Jones, Rapid object detection using a boosted cascade of
simple features, in Computer Vision and Pattern Recognition, 2001. CVPR
 2001. Proceed- ings of the 2001 IEEE Computer Society Conference on,
2001, vol. 1, pp. I511I518 vol.1.
[9] R. Lienhart and J. Maydt, An extended set of haar-like features for rapid
object detection, in Image Process- ing. 2002. Proceedings. 2002
International Conference on, 2002, vol. 1, pp. I900I903 vol.1.
[10] Ofir Pele and Michael Werman, The quadratic-chi his- togram distance
family, in Computer Vision ECCV 2010, Kostas Daniilidis, Petros Maragos,
and Nikos Paragios, Eds. 2010, vol. 6312 of Lecture Notes in Com- puter
Science, pp. 749762, Springer Berlin Heidelberg.
[11] A. Barla, F. Odone, and A. Verri, Histogram intersec- tion kernel for image
classification, in Image Process- ing, 2003. ICIP 2003. Proceedings. 2003
International Conference on, 2003, vol. 3, pp. III51316 vol.2.
[12] Xi Chen and Tat-Jen Cham, Discriminative distance measures for image
matching, in Pattern Recognition, 2004. ICPR 2004. Proceedings of the
17th International Conference on, 2004, vol. 3, pp. 691695 Vol.3.
[13] Olegs Nikisins, Weighted multi-scale local binary pat- tern histograms for
face recognition, in International Conference on Applied Mathematics and
Computational Methods, AMCM 2013, 2013, pp. 7681.
[14] M. Pudzs, R. Fuksis, R. Ruskuls, T. Eglitis, A. Kadikis, and M. Greitans,
Fpga based palmprint and palm vein biometric system, in Biometrics
Special Interest Group (BIOSIG), 2013 International Conference of the, 2013,
pp. 14.