Image Features

Lecture 3
By: Sarah M. Ayyad

Back Next
What are Features?

Features in the context of computer vision are more technical in nature!

Back Next
What are Features?
● Features of an image are defined as regions where some unusual action happens.
● For example, the brightness changes drastically to form an edge; or gradients of edges
change drastically to form corner.

● Image features could identify and describe patterns in images which can be used to
perform a variety of tasks such as:

○ Image Registration

○ Stitching images together

○ Classifying images into different categories

Back Next
Image registeration
● Used to align and compare images

Back Next
Stitching Images together
● To form a wider panoramic shot

Back Next
Classifying Images into different
Categories

Back Next
Characteristics of good features

Distinctiveness Quantity
1 Each feature
00 has a distinctive
description.
2 Hundreds or thousands
in a single image
Can differentiate a large database
of objects.

Locality Compactness
3 Features occupies a small area 4 Many fewer features than
image pixels
of the image.
Robust to clutter and “ we should have enough,
occlusion. but not too many”.
In the next lectures, we will learn
about
Feature Detection
• To identify the locations of features (identify interest points)

Feature Extraction
• To extract vector feature descriptor around each interest point

Feature Matching
• Important for many workflows
oImage registration
oImage stitching
oTracking
Back oAutomated labeling Next
 Feature Detection
They are many algorithm used to detect features.
They fall into 3 categories:

Blobs
SURF algorithm

01 02 03

Corners Regions
Harris-stephans
MSER algorithm
algorithm
1. Corner Detection
● Corner detection is used as the first step of many vision tasks such as tracking,
localization, image matching and recognition.
● Harris Corner Detector is a corner detection operator that is commonly used in
computer vision algorithms to extract corners and infer features of an image.
● A corner is a point whose local neighborhood stands in two dominant and
different edge directions.
● In other words, a corner can be interpreted as the junction of two edges, where
an edge is a sudden change in image brightness.
● Corners are the important features in the image, and they are generally termed as
interest points
Back Next
Finding interest points in an image
● Suppose we only consider a small window of pixels
• What defines whether a feature is a good or bad candidate?

Back Next
Finding interest points in an image
• How does the window change when you shift it?

flat region edge corner

no change in all no change along the significant change in all
directions edge direction directions, i.e., even the
minimum change is large
Back Find locations such that the minimum change caused by Next
shifting the window in any direction is large
 Corner Detection
Shift in any direction would result in a significant change at a corner.

Steps: (General Idea)

- Shift in horizontal, vertical, and
diagonal directions by one pixel.
- The similarity is measured by
calculating the absolute value of the
square error for each shift.
- A lower number indicates more
similarity flat region edge corner

Back Next
Finding interest points (Corner detection):
The math
Consider shifting the window W by (u, v)

𝐸(𝑢, 𝑣) = σ𝑥,𝑦 𝑤 𝑥, 𝑦 [𝐼 𝑥 + 𝑢, 𝑦 + 𝑣 − 𝐼(𝑥, 𝑦)]2

Window Shifted Intensity

function Intensity

Window function 𝑤 𝑥, 𝑦 =
Back Next
moves fast moves slowly
Finding interest points (Corner detection):
The math

𝐸(𝑢, 𝑣) = σ𝑥,𝑦 𝑤 𝑥, 𝑦 [𝐼 𝑥 + 𝑢, 𝑦 + 𝑣 − 𝐼(𝑥, 𝑦)]2

For nearly constant patches, this will be near 0.

For very distinctive patches, this will be larger.
Hence, the required patches are those with larger 𝐸(𝑢, 𝑣)

Back Next
Finding interest points (Corner detection):
The math

𝐸(𝑢, 𝑣) = σ𝑥,𝑦 𝑤 𝑥, 𝑦 [𝐼 𝑥 + 𝑢, 𝑦 + 𝑣 − 𝐼(𝑥, 𝑦)]2

We want to find out this function behaves for small shift

(𝑢, 𝑣, near 0,0 )

Second-order Taylor expansion of 𝐸 𝑢, 𝑣 about 0,0

(local quadratic approximation for small 𝑢, 𝑣)
Back Next
Applying Harris-stephens algorithm
using Matlab
Using Harris-stephens algorithm
 Using Harris-stephens algorithm
Plotting the corners features over the image.
The Location property
The Metric property
The Count property
Detecting only strongest features