Real-Time Facial

Recognition

MATCH!

EECE 279: Real-Time Systems Design

Vanderbilt University
Ames Brown & Jason Cherry
 Topics Of Discussion
• Why real-time face recognition?
• What is difficult about real-time face recognition?
• In general how is face recognition done?
• Eigenfaces
• Other face recognition algorithms
• Opposition
• Future of face recognition
 Why Real-Time Face
Recognition?
• Security
– Fight terrorism
– Find fugitives
• Personal information access
– ATM
– Sporting events
– Home access (no keys or passwords)
– Any other application that would want personal identification
• Improved human-machine interaction
• Personalized advertising
• Beauty search
 Real-Time Face Recognition
System Requirements
• Want the system to be inexpensive enough to use
at many locations
• Match within seconds
– Before the person walks away from the advertisement
– Before the fugitive has a chance to run away
• Ability to handle a large database
• Ability to do recognition in varying environments
 What Is Difficult About Real-
Time Face Recognition
• Lighting variation
• Orientation variation (face angle)
• Size variation
• Large database
• Processor intensive
• Time requirements
 General Image Types
• Still image (digital photograph)
– AmIU
• Dynamic image
– Video camera
 FERET database
• Contains images of 1,196 individuals, with up to
5 different images captured for each
individual
• Often used to test face recognition algorithms
• Information on obtaining the database can be
found here:
http://www.itl.nist.gov/iad/humanid/feret/
General Face Recognition
Steps

• Face Detection
• Face Normalization
• Face Identification
 Face Detection
In General Different Approaches
• Locate face in a given • Motion detecting and head
image tracking
• Separate it from the scene
• “Face Space” distance
Face Detection: “Face Space”
 Face Detection: Motion
Detecting and Head Tracking
 Face Detection: Motion
Detecting and Head Tracking
Face Normalization

• Adjustment
– Expression
– Rotation
– Lighting
– Scale
– Head tilt
• Eye location
 Face Normalization: FERET
face2norm
1. Image is rotated to align the eyes (eye coordinates must
be known).
2. The image is scaled to make the distance between the
eyes constant. The image is also cropped to a smaller size
that is nearly just the face.
3. A mask is applied that zeros out pixels not in an oval that
contains the typical face. The oval is generated
analytically.
4. Histogram equalization is used to smooth the distribution
of gray values for the non-masked pixels.
5. The image is normalized so the non-masked pixels have
mean zero and standard deviation one.
 Face Identification

• Application of a face recognition algorithm

 PCA Algorithms
• Principle Component Analysis
• Look at the principal components of the
distribution of faces, or the eigenvectors of the
covariance matrix of the set of face images
• Eigenfaces
 Eigenfaces Algorithm
Eigenfaces Initialization
1. Acquire an initial set
of face images (the
training set)
 Eigenfaces Algorithm
2. Calculate the eigenfaces
from the training set,
keeping only the M images
that correspond to the
highest eigenvalues. These
M images define the face
space. As new faces are
experienced, the eigenfaces
can be updated or
recalculated
 Eigenfaces Algorithm
3. Calculate the corresponding distribution in M-dimensional
weight space for each known individual, by projecting
their face images onto the “face space.”
 Eigenfaces Algorithm
Eigenfaces Recognition 3. If it is a face, classify the
1. Calculate a set of weights weight pattern as either a
based on the input image known person or as
and the M eigenfaces by unknown.
projecting the input image
onto each of the 4. (Optional) Update the
eigenfaces. eigenfaces and/or weight
2. Determine if the image is patterns.
a face at all by checking
to see if the image is
sufficiently close to “face
space.”
 Eigenfaces Problems
• Recognition performance decreases quickly as the
head size, or scale, is misjudged. The head size in
the input image must be close to that of the
eigenfaces for the system to work well
• In the case where every face image is classified as
known, a sample system achieved approximately
96% correct classification averaged over lighting
variation, 85% correct averaged over orientation
variation, and 64% correct averaged over size
variation
 Parameter Based Facial
Recognition
• Facial image is analyzed and reduced to small set
of parameters describing prominent facial features
• Major features analyzed are: eyes, nose, mouth
and cheekbone curvature
• These features are then matched to a database
• Advantage: recognition task is not very expensive
• Disadvantage: the image processing required is
very expensive and parameter selection must be
unambiguous to match an individual’s face
 Template Based Facial
Recognition
• Salient regions of the facial image are extracted
• These regions are then compared on a pixel-by-
pixel basis with an image in the database
• Advantage is that the image preprocessing is
simpler
• Disadvantage is the database search and
comparison is very expensive
 Real-Time System using
Template Recognition
• Implemented on a IBM PC w/ a video camera,
image digitizer, and custom VLSI image
correlator chip (340 Mop/second).
• Needed single frontal facial image under semi-
controlled lighting conditions
• Took the system 2 to 3 seconds to identify a user
from 173 images of 34 persons
• 88% recognition rate
How the System Worked
 Artificial Neural Networks in
Real-Time Face Recognition
• Use many of the same algorithms described
before but with back propagation ANN’s
• Disadvantages: Complex and difficult to train
• Difficult to implement
• Sensitive to lighting variation
 There are many face
recognition algorithms
• LDA (Linear Discriminant Analysis)
• Bayesian Classifier
• Gabor Wavelet Algorithm
• Elastic graphs

Get more information and source code at

http://www.cs.colostate.edu/evalfacerec/index.html
 Not Everyone Loves Face
Recognition
• Critics say it produces too many false positives
• Invasion of privacy
• To easy to misuse for wrong purposes
• Technology is not accurate enough given the
current technology and algorithms
 Future Of Face Recognition

• Some consider the problem impossible

• No standard way of approaching the problem
• Advancements in hardware and software
• Slow integration into society in limited
environments
• Very large potential market
Questions?
 References
• M. Turk and A. Pentland. Eigenfaces for Recognition.
Journal of Cognitive Neuroscience, 3(1), 1991
• C. Nastar and M. Mitschke. Real-Time Face Recognition
Using Feature Combination. In Proceedings of the Third
IEEE International Conference on Automatic Face and
Gesture Recognition, Nara, Japan, April 1998
• J. Gilbert and W. Yang. A Real-Time Face Recognition
System using Custom VLSI Hardware. Harvard
Undergraduate Honors Thesis in Computer Science, 1993.
 Topics Of Discussion
• Why real-time face recognition?
• What is difficult about real-time face recognition?
• In general how is face recognition done?
• Eigenfaces
• Other face recognition algorithms
• Opposition
• Future of face recognition
 Real-Time Facial
Recognition

MATCH!

EECE 279: Real-Time Systems Design

Vanderbilt University
Ames Brown & Jason Cherry