Computer Vision and Pattern Recognition

L11. Shape Representation (II)

Dr John Collomosse
[email protected]

Centre for Vision, Speech and Signal Processing

University of Surrey
Learning Outcomes
After attending this lecture, and doing the reading and
labwork you should be able to:

 Describe how 2D moments can be used to represent shape

 Understand how shape descriptors can be integrated into a standard
supervised classification framework for shape recognition
Describe basic morphological filtering operations like erosion, dilation,
opening and closure.
Construct and use chain codes
Discuss the invariance properties of different shape descriptors
Work with both region based shape descriptors and contour based shape
descriptors
Recall: Statistical Moments (1D)
 Central moments are moments computed about the mean
 Written 1 for short
th
 In general, the  central moment is:

 2 is the variance of the data (measures “spread”= divergence from

mean)

 3 is the skew of the data (measures how data is “balanced” about mean)
Statistical Moments (2D)
 The 2D equation for a moment about zero is:
m10 – x component of mean
m01 – y component of mean

 The 2D equation for a central moment is:

 As before, 00 is by definition = 1

 It is the 2D central moments that are mainly useful in shape description
 Taking successive higher order moments decomposes a shape in a
conceptually similar way to Fourier analysis decomposing a signal
Useful applications of Moments
 The orientation of a shape can be computed using ratio of moments

 The seven Hu moments (affine invariant shape descriptors)

 Originally proposed for character recognition

No, you don’t have to

remember these for the exam...
Hu Moments - Example
 Computing Hu moments for this image under various transformations

Results in these values of the 7 Hu moments

Shape Descriptors
A shape descriptor is a feature representing the shape of a region or
contour.
 Just like any descriptor (remember image descriptors?) a shape
descriptor should be disciminative and compact
 So far we met two shape descriptors:-

Shape factors Moments

These were suitable for describing

regions
= region based shape descriptors
 Later on we will also look at some contour based shape descriptors
What use are Shape Descriptors?
Shape descriptors give us a vector of numbers describing the shape
 As with all features, we consider these numbers as a point in an n-D space
 So each shape maps to a point in the space

Feature space
H7 H1
H2
In this example, each shape
H3
maps to a point in 7D space H4
H5
H6
H7

Shape descriptor
H6
H5
H4 H3
H2
H1
Shape Descriptors for Classification

Different shapes map to points in different parts of the space

H7 H1
H2
H3
H4
H5
H6
H7

Shape descriptor
H6
H5
H4 H3
Feature space
H2
H1
Shape Descriptors for Classification
If we model each class of shape we can build a supervised classifier
• Given a new shape, we calculate it descriptor (point in space) =
• The model it’s nearest to tells us the category of the shape

H7 H1
H2
H3
H4
H5
H6
H7

Shape descriptor
H6
H5
H4 H3
Feature space
H2
H1
Shape Descriptors for Classification
All the supervised classification techniques covered before, apply
here
• Can use Eigenmodels (or even SVMs) to classify
• Use a confusion matrix to evaluate

H7 H1
H2
H3
H4
H5
H6
H7

Shape descriptor
H6
H5
H4 H3
Feature space
H2
H1
 Over to You
You win a military contract to design an air defence system
A camera will watch the sky and decide which kind of plane is overhead

MIG? or Stealth Bomber?

How will you do it?

 Over to You - Solution
1) Separate the plane from the blue sky
- Use a colour classifier to decide if each pixel is sky or plane
2)Calculate a shape descriptor from the region
3)Feed the shape descriptor into a supervised classifier

( You need to first train the classifier with examples of each class of
plane that needs to be recognised )
Shape Descriptors for Classification
Plane spotting with shape descriptors

Is it a...

Bird?
Plane?
Superman?
H7

bird plane man

BIRD 1.0 0 0

H6
H5 PLANE 0 1.0 0
H4 H3
H2 MAN 0 0.5 0.5
H1
Cleaning up the binary mask
The shape descriptors we have looked at so far are based on regions
Regions are not always cleanly separable from the background
This creates noise in the binary mask
Binary Erosion
A simple form of non-linear filter can help - Erosion

0 0 0 0 1 1
Has speckles
of noise Structuring
0 1 0 0 1 1
Region
0
Element
0 0 1 1 1

1 0 1 1 0 1
Has unwanted holes
0 0 1 1 1 1

0 0 1 1 1 1

0 1 1 1 1 1
 Binary Erosion
A simple form of non-linear filter can help - Erosion

0 0 0 0 1 1 0 0 0 0 1 1

0 1 0 0 1 1 0 0 0 0 1 1

0 0 0 1 1 1 0 0 0 1 1 1

1 0 1 1 0 1 Erode 0 0 1 1 0 1

0 0 1 1 1 1 0 0 1 1 1 1

0 0 1 1 1 1 0 0 1 1 1 1

0 1 1 1 1 1 0 1 1 1 1 1

Input Output

Count the number of ON pixels in structuring element. If > n, then

output is ON else it’s OFF.
 Binary Dilation
Dilation is the “conceptual” opposite of erosion

0 0 0 0 1 1 0 0 0 0 1 1

0 0 0 0 1 1 0 0 0 0 1 1

0 0 0 1 1 1 0 0 0 1 1 1

0 0 1 1 0 1 Dilate 0 0 1 1 1 1

0 0 1 1 1 1 0 0 1 1 1 1

0 0 1 1 1 1 0 0 1 1 1 1

0 1 1 1 1 1 0 1 1 1 1 1

Input Output

Count the number of OFF pixels in structuring element. If > n, then

output is OFF else it’s ON.
Morphological Operations
Dilation and erosion are called “morphological operators”
Because they are non-linear they can’t be achieved via convolution
Combinations of dilation and erosion can be very useful
 A dilation followed by an erosion is a “closure”
 An erosion followed by a dilation is an “opening”

Erode Dilate

Opening
 Binary Opening and Closure
These operations illustrate why dilation is not the inverse of erosion
The effect of one does not reverse the other, and this is useful!

Erode Dilate

Opening

Dilate Erode

Closure
Obtaining boundary of a region
A different combination of erosion and dilation can be used here too

Boundary(I) = Dilate(I) – Erode(I)

Contour based shape descriptors

The shape descriptors we have looked at

so far are based on regions
We compute them from connected
components in the image

We will now look at a way of

computing a shape descriptor from
the contour (boundary) of a region
Chain code
The chain (Freeman) code was invented in the early 80s as one of
the first contour based shape representations.

Chain codes can represent open or closed shapes

They can also be used for compression (e.g. 3x2 +1x3...)
Common uses for chain codes are in gesture recognition due to
powerful string matching algorithms that can be applied.
Creating a chain code
Over to you
Can chain codes represent all shapes?

As described, are they invariant to:-

• Translation?
• Rotation ?
• Scale?
Over to you
Can chain codes represent all shapes?
Yes
As described, are they invariant to:-
• Translation? Yes
• Rotation ? No
• Scale? No
Start-point Invariant Chain codes
We can character-rotate the chain code string to a known point, to
make it invariant to the start of the coding point.

Do this by computing the “first difference” of a chain code

Difference of adjacent points, anti-clockwise –
e.g. CC=10103322 FD=3133030

Try every rotation of the chain code to find smallest integer FD

Note this is not the same as rotational invariance.
Summary
After attending this lecture, and doing the reading and
labwork you should be able to:

 Describe how 2D moments can be used to represent shape

Understand how shape descriptors can be integrated into a standard
supervised classification framework for shape recognition
Describe basic morphological filtering operations like erosion, dilation,
opening and closure.
Construct and use chain codes
Discuss the invariance properties of different shape descriptors
Work with both region based shape descriptors and contour based shape
descriptors
Further Reading
 “Feature extraction and image processing”
-pp.103 – (Morphological ops e.g. Erosion, Dilation)
-pp. 311 – (Moments)