Pattern Recognition
Text Books
• Pattern Recognition
• S. Theodoridis & K. Koutrumbas
• Pattern Classification
• R. Duda et al.
• Pattern Recognition Statistical, Structural and Neural
Approaches
• R. Shalkoff
Pattern Recognition:
What is it?
The act of taking as input sensed
data(measurements)
and taking an action
based on the “category” or “class”
ofthe pattern.
What It Does
• Build a machine that can recognize patterns:
• The task: Assign unknown objects – patterns – into the
correct class. This is known as classification.
What It Does
• Areas:
– Machine vision Image Data Base retrieval
– Character recognition (OCR) Data mining
Biometrics
– Computer aided diagnosis
Fingerprint identification
– Speech recognition
Iris Recognition
– Face recognition
DNA sequence identification
– Bioinformatics
Representation of patterns
• Features:
• measurable quantities from the patterns
• determines the classification task
• Feature vectors: A number of features
x1 ,..., xl ,
constitute the feature vector
x x1 ,..., xl R l
T
Feature vectors are treated as random vectors.
Example 1:
Example 1:
Issues in Pattern Recognition
• How are features generated?
• What is the best number of features?
• How are they used to design a classifier?
• How good is the classifier?
Example 2
• “Sorting incoming Fish on a conveyor
according to species using optical sensing”
Sea bass
Species
Salmon
• Problem Analysis
– Set up a camera and take some sample images to extract
features
• Length
• Lightness
• Width
• Number and shape of fins
• Position of the mouth, etc…
• Preprocessing
– isolate fishes from one another and from the
background
• Feature Extraction
– send isolated fish image to feature extractor
– it reduces the data too
• Classification
– pass the features to a classifier
• Classification
– Select the length of the fish as a possible feature
for discrimination
x*
The length is a poor feature alone!
Select the lightness as a possible feature.
• Threshold decision boundary and cost relationship
– Move our decision boundary toward smaller values of
lightness in order to minimize the cost (reduce the
number of sea bass that are classified as salmon!)
Task of decision theory
• Adopt the lightness and add the width of the
fish
Fish xT = [x1, x2]
Lightness Width
• adding correlated feature does not improve
anything and is thus redundant
• too many features may lead to curse of
dimensionality
still there are some misclassifications
perhaps the best one, but too complex
decision boundary
• satisfaction is premature
– cause: aim of a classifier is to correctly classify unknown
input
Issue of generalization!
A compromise between training and testing
Pattern Recognition Systems
• Sensing
– Use of a transducer (camera or microphone)
– PR system depends of the bandwidth, the
resolution sensitivity distortion of the transducer
• Segmentation and grouping
– Patterns should be well separated and should not
overlap
• Feature extraction
– Discriminative features
– Invariant features with respect to translation, rotation and
scale.
• Classification
– Use a feature vector provided by a feature extractor to assign
the object to a category
• Post Processing
– error rate
– risk
– use context
The Design Cycle
• Data collection
• Feature Choice
• Model Choice
• Training
• Evaluation
• Computational Complexity
• Data Collection
– How do we know when we have collected an
adequately large and representative set of
examples for training and testing the system?
• Feature Choice
– Depends on the characteristics of the problem
domain.
– Requirement
• simple to extract
• invariant to irrelevant transformation
• insensitive to noise.
• Model Choice
– too many classification models?
– which one is best?
• Training
– Use data to determine the classifier. Many
different procedures for training classifiers and
choosing models
• Evaluation
– Measure the error rate (or performance and
switch from one set of features to another one
• Computational Complexity
– What is the trade-off between computational
ease and performance?
Learning and Adaptation
• Supervised learning
– A teacher provides a category label or cost for
each pattern in the training set
• Unsupervised learning
– The system forms clusters or “natural groupings”
of the input patterns
Unsupervised Learning
Unsupervised Learning
x1
x2
Unsupervised Learning
x1
x2
Unsupervised Learning
x1
x2
