CS436/CS5310 


Computer Vision Fundamentals

Murtaza Taj
[email protected]

Lecture 15: Corner Detection

Mon, 11th Mar 2019
Why Image Features
! Motivation
Object Detection Object Tracking

Panorama 3D Reconstruction
What Are Image Features?
! Global Features
! A global property of an image, e.g. the average gray level

! Local Features
! A part of an image with some special property, e.g. circle, line,
specific texture etc.
Local Features - Definition
! (Good) Image features are local, meaningful, detectable
parts of an image and should be invariant to normally
expected changes in the scene

! Meaningful
! Features associated to “interesting” scene elements via the image
formation process, e.g. intensity variations created by contours of
an object in the scene
! # of pixels with pixel-level 134 is a feature which is not meaningful
(why?) and also not invariant to light changes

! Detectable
! Means that algorithms should exist to detect the location of that
feature, otherwise it is of no use
Detecting Point Features (Corners)
http://en.wikipedia.org/wiki/Corner_detection
Corners
! How do we define a corner?
! Qs: How does a corner differ from an edge?

! A corner may be interpreted as an intersection of two

edges, or as a region where strong derivatives exist in
more than one direction
 What Is a Corner?
! Corners are local image features characterised by
locations where variations of intensity function f(x, y) in
both X and Y directions are high
! Intersection points between two or more edge segments

www.cse.cuhk.edu.hk/~lyu/seminar/05fall/Wyman.ppt
Detecting Corners: The Basic Idea
! We should easily recognize the point by looking through a
small window
! Shifting a window in any direction should give a large
change in intensity

www.wisdom.weizmann.ac.il/~deniss/vision_spring04/files/InvariantFeatures.ppt
 Basic Idea

“flat” region:
 “edge”:
 “corner”:


no change in all no change along significant change
directions the edge direction in all directions

www.wisdom.weizmann.ac.il/~deniss/vision_spring04/files/InvariantFeatures.ppt
 +
- >0

Demo of a point + with well distinguished neighbourhood.

 +
- >0

Demo of a point + with well distinguished neighbourhood.

 +
- >0

Demo of a point + with well distinguished neighbourhood.

 +
- >0

Demo of a point + with well distinguished neighbourhood.

 +
- >0

Demo of a point + with well distinguished neighbourhood.

 +
- >0

Demo of a point + with well distinguished neighbourhood.

 +
- >0

Demo of a point + with well distinguished neighbourhood.

 +
- >0

Demo of a point + with well distinguished neighbourhood.

Building a Simple Corner Detector
! Taking the derivatives of an image window I, sliding it over
itself and taking the squared difference: Sum of Squared
Differences of gradient

where is the offset of the sliding window

and is a pixel location in the image window

Taylor Series
! Taylor expansion of f(x) about a
Building a Simple Corner Detector

! Using Taylor expansion of about

where is the gradient vector (recall from

edge detectors)

! Hence the error term can be written as:

Building a Simple Corner Detector
! From previous slide, the sum of squared error for an
image window sliding over itself can approximately be
written as:

where
KLT Corner Detector
! The sum of squared differences depends on the following
matrix (up to 1st order Taylor Approximation)

! This matrix encodes the ‘structure’ of the gray-levels

[why?]
 KLT Corner Detector

⎡ ∑ I x2 ∑I I
x y
⎤
What form will A=⎢ 2 ⎥ take in each case?
⎢⎣∑ I x I y ∑I y ⎥⎦

           
KLT Corner Detector

⎡ ∑ I x2 ∑ I I ⎤⎥
x y
! ⎢
The matrix ⎢⎣∑ I x I y ∑ I ⎥⎦ is the covariance of the gradient
2
y

vectors in a window
! Because we can assume gradients to be zero mean

! This matrix can be used to distinguish between corners,

edges and plain patches.
! Recall: For a set of 2D vectors s.t. 

covariance
2
! We can write C as 2

which is why the matrix at the top of the slide is covariance of

gradient vector
Some Properties of this Covariance Matrix

! Is positive-semidefinite ⎡ ∑ I x2 ∑I I
x y
⎤
⎢ 2 ⎥
! Consequently, its eigen values will not be 
 ⎢⎣∑ I x I y ∑I y ⎥⎦
negative
! If the x values do not tell me anything
about the y values, then the off-diagonal
terms are zero
! Rotation of the image by R changes
covariance A by RART
Proof: Before rotation, covariance
After rotation covariance
Some Properties of this Covariance Matrix

! Consider the eigenvectors of A

! Both equations can be combined as

(1)

where
is the matrix of eigen vectors

is the matrix of eigen values

! From (1)

! Implies: rotating the image by diagonalizes

covariance
 KLT Corner Detector

⎡ ∑ I x2 ∑I I
x y
⎤
What form will A=⎢ 2 ⎥ take in each case?
⎢⎣∑ I x I y ∑I y ⎥⎦

           
Corner Detection
! Hence, we do not need to rotate the image in any case

! We just need to find the eigen values of A, which, when

arranged as give us the covariance of the rotated
image!

! This leads us to the KLT corner detector algorithm

KLT Corner Detector

! Let λ1 , λ2 be eigen values

of the gradient covariance
matrix
! Smooth patch: 

λ1 = λ2 = 0
! Edge

λ1 > 0, λ2 = 0

⎡λ1 0 ⎤
! Corner 

λ1 > 0, λ2 > 0

! KLT Criterion

⎢0 λ ⎥
min[λ1 , λ2 ] > T ⎣ 2⎦
Harris Corner Detector
! Approximation

! k effects scale of the corner

Haris detector, step-by-step
Haris detector, step-by-step
Haris detector, step-by-step
Haris detector, step-by-step
Haris detector, step-by-step
Haris Detector Properties
Haris Detector Properties
The Problem of Scale
The Problem of Scale
Solution to Scale Problem
! Many ‘scale invariant’ corner detectors have been
proposed
! They essentially find corners at multiple scales
! A good corner detector, invariant to intensity and
geometric distortions is part of ongoing research

! Solutions
! Scale-invariant feature transform (SIFT) [5]
! PCA-SIFT, GSIFT, CSIFT, ASIFT [4]
! Speeded Up Robust Features (SURF) [7]
! Oriented FAST and Rotated BRIEF (ORB)[1]
! STAR[8], FAST[6], BRIEF[2], FREAK[3]
References
! [1] E. Rublee, V. Rabaud, K. Konolige, and G. R. Bradski, “ORB: An efficient alternative to SIFT or SURF”, in
Proc. ICCV, 2011, pp. 2564–2571.

! [2] S. Leutenegger, M. Chli, and R. Siegwart, “Brisk: Binary robust invariant scalable keypoints,” in Proc. Int.
Conf. Computer Vision, 2011, pp. 2548–2555.

! [3] A. Alahi, R. Ortiz, and P. Vandergheynst, “FREAK: Fast Retina Keypoint”, In Proc. IEEE Conference on
Computer Vision and Pattern Recognition, 2012, pp. 510-517.

! [4] Jian Wu, Zhiming Cui, Victor S. Sheng, Pengpeng Zhao, Dongliang Su, Shengrong Gong, “A Comparative
Study of SIFT and its Variants”, Measurement Science Review,Jun 2013

! [5] D. Lowe, “Object recognition from local scale-invariant features”, in: Proceedings of the International
Conference on Computer Vision ICCV, Corfu, 1999, pp. 1150–1157.

! [6] E. Rosten, and T. Drummond, “Machine learning for highspeed corner detection”, in Proc. of European Conf.
on Computer Vision, 2006, pp. 430–443.

! [7] H. Bay, A. Ess, T. Tuytelaars, L. Van Gool, “SURF: Speeded Up Robust Features”, Computer Vision and
Image Understanding, vol. 110(3), 2008, pp. 346–359.

! [8] M. Agrawal, K. Konolige, and M.R. Blas, “CenSurE: Center surround extremas for real time feature detection
and matching”, Lecture Notes in Computer Science, vol. 5305, 2008, pp. 102–115.

! [9] M. Calonder, V. Lepetit, C. Strecha, and P. Fua, “BRIEF: Binary Robust Independent Elementary Features”, in
Proceedings of ECCV 2010, pp. 778–792.