CS436/CS5310
Computer Vision Fundamentals
Murtaza Taj
[email protected]
Lecture 15: Corner Detection
Mon, 11th Mar 2019
Why Image Features
! Motivation
Object Detection Object Tracking
Panorama 3D Reconstruction
What Are Image Features?
! Global Features
! A global property of an image, e.g. the average gray level
! Local Features
! A part of an image with some special property, e.g. circle, line,
specific texture etc.
Local Features - Definition
! (Good) Image features are local, meaningful, detectable
parts of an image and should be invariant to normally
expected changes in the scene
! Meaningful
! Features associated to “interesting” scene elements via the image
formation process, e.g. intensity variations created by contours of
an object in the scene
! # of pixels with pixel-level 134 is a feature which is not meaningful
(why?) and also not invariant to light changes
! Detectable
! Means that algorithms should exist to detect the location of that
feature, otherwise it is of no use
Detecting Point Features (Corners)
http://en.wikipedia.org/wiki/Corner_detection
Corners
! How do we define a corner?
! Qs: How does a corner differ from an edge?
! A corner may be interpreted as an intersection of two
edges, or as a region where strong derivatives exist in
more than one direction
What Is a Corner?
! Corners are local image features characterised by
locations where variations of intensity function f(x, y) in
both X and Y directions are high
! Intersection points between two or more edge segments
www.cse.cuhk.edu.hk/~lyu/seminar/05fall/Wyman.ppt
Detecting Corners: The Basic Idea
! We should easily recognize the point by looking through a
small window
! Shifting a window in any direction should give a large
change in intensity
www.wisdom.weizmann.ac.il/~deniss/vision_spring04/files/InvariantFeatures.ppt
Basic Idea
“flat” region:
“edge”:
“corner”:
no change in all no change along significant change
directions the edge direction in all directions
www.wisdom.weizmann.ac.il/~deniss/vision_spring04/files/InvariantFeatures.ppt
+
- >0
Demo of a point + with well distinguished neighbourhood.
+
- >0
Demo of a point + with well distinguished neighbourhood.
+
- >0
Demo of a point + with well distinguished neighbourhood.
+
- >0
Demo of a point + with well distinguished neighbourhood.
+
- >0
Demo of a point + with well distinguished neighbourhood.
+
- >0
Demo of a point + with well distinguished neighbourhood.
+
- >0
Demo of a point + with well distinguished neighbourhood.
+
- >0
Demo of a point + with well distinguished neighbourhood.
Building a Simple Corner Detector
! Taking the derivatives of an image window I, sliding it over
itself and taking the squared difference: Sum of Squared
Differences of gradient
where is the offset of the sliding window
and is a pixel location in the image window
Taylor Series
! Taylor expansion of f(x) about a
Building a Simple Corner Detector
! Using Taylor expansion of about
where is the gradient vector (recall from
edge detectors)
! Hence the error term can be written as:
Building a Simple Corner Detector
! From previous slide, the sum of squared error for an
image window sliding over itself can approximately be
written as:
where
KLT Corner Detector
! The sum of squared differences depends on the following
matrix (up to 1st order Taylor Approximation)
! This matrix encodes the ‘structure’ of the gray-levels
[why?]
KLT Corner Detector
⎡ ∑ I x2 ∑I I
x y
⎤
What form will A=⎢ 2 ⎥ take in each case?
⎢⎣∑ I x I y ∑I y ⎥⎦
KLT Corner Detector
⎡ ∑ I x2 ∑ I I ⎤⎥
x y
! ⎢
The matrix ⎢⎣∑ I x I y ∑ I ⎥⎦ is the covariance of the gradient
2
y
vectors in a window
! Because we can assume gradients to be zero mean
! This matrix can be used to distinguish between corners,
edges and plain patches.
! Recall: For a set of 2D vectors s.t.
covariance
2
! We can write C as 2
which is why the matrix at the top of the slide is covariance of
gradient vector
Some Properties of this Covariance Matrix
! Is positive-semidefinite ⎡ ∑ I x2 ∑I I
x y
⎤
⎢ 2 ⎥
! Consequently, its eigen values will not be
⎢⎣∑ I x I y ∑I y ⎥⎦
negative
! If the x values do not tell me anything
about the y values, then the off-diagonal
terms are zero
! Rotation of the image by R changes
covariance A by RART
Proof: Before rotation, covariance
After rotation covariance
Some Properties of this Covariance Matrix
! Consider the eigenvectors of A
! Both equations can be combined as
(1)
where
is the matrix of eigen vectors
is the matrix of eigen values
! From (1)
! Implies: rotating the image by diagonalizes
covariance
KLT Corner Detector
⎡ ∑ I x2 ∑I I
x y
⎤
What form will A=⎢ 2 ⎥ take in each case?
⎢⎣∑ I x I y ∑I y ⎥⎦
Corner Detection
! Hence, we do not need to rotate the image in any case
! We just need to find the eigen values of A, which, when
arranged as give us the covariance of the rotated
image!
! This leads us to the KLT corner detector algorithm
KLT Corner Detector
! Let λ1 , λ2 be eigen values
of the gradient covariance
matrix
! Smooth patch:
λ1 = λ2 = 0
! Edge
λ1 > 0, λ2 = 0
⎡λ1 0 ⎤
! Corner
λ1 > 0, λ2 > 0
! KLT Criterion
⎢0 λ ⎥
min[λ1 , λ2 ] > T ⎣ 2⎦
Harris Corner Detector
! Approximation
! k effects scale of the corner
Haris detector, step-by-step
Haris detector, step-by-step
Haris detector, step-by-step
Haris detector, step-by-step
Haris detector, step-by-step
Haris Detector Properties
Haris Detector Properties
The Problem of Scale
The Problem of Scale
Solution to Scale Problem
! Many ‘scale invariant’ corner detectors have been
proposed
! They essentially find corners at multiple scales
! A good corner detector, invariant to intensity and
geometric distortions is part of ongoing research
! Solutions
! Scale-invariant feature transform (SIFT) [5]
! PCA-SIFT, GSIFT, CSIFT, ASIFT [4]
! Speeded Up Robust Features (SURF) [7]
! Oriented FAST and Rotated BRIEF (ORB)[1]
! STAR[8], FAST[6], BRIEF[2], FREAK[3]
References
! [1] E. Rublee, V. Rabaud, K. Konolige, and G. R. Bradski, “ORB: An efficient alternative to SIFT or SURF”, in
Proc. ICCV, 2011, pp. 2564–2571.
! [2] S. Leutenegger, M. Chli, and R. Siegwart, “Brisk: Binary robust invariant scalable keypoints,” in Proc. Int.
Conf. Computer Vision, 2011, pp. 2548–2555.
! [3] A. Alahi, R. Ortiz, and P. Vandergheynst, “FREAK: Fast Retina Keypoint”, In Proc. IEEE Conference on
Computer Vision and Pattern Recognition, 2012, pp. 510-517.
! [4] Jian Wu, Zhiming Cui, Victor S. Sheng, Pengpeng Zhao, Dongliang Su, Shengrong Gong, “A Comparative
Study of SIFT and its Variants”, Measurement Science Review,Jun 2013
! [5] D. Lowe, “Object recognition from local scale-invariant features”, in: Proceedings of the International
Conference on Computer Vision ICCV, Corfu, 1999, pp. 1150–1157.
! [6] E. Rosten, and T. Drummond, “Machine learning for highspeed corner detection”, in Proc. of European Conf.
on Computer Vision, 2006, pp. 430–443.
! [7] H. Bay, A. Ess, T. Tuytelaars, L. Van Gool, “SURF: Speeded Up Robust Features”, Computer Vision and
Image Understanding, vol. 110(3), 2008, pp. 346–359.
! [8] M. Agrawal, K. Konolige, and M.R. Blas, “CenSurE: Center surround extremas for real time feature detection
and matching”, Lecture Notes in Computer Science, vol. 5305, 2008, pp. 102–115.
! [9] M. Calonder, V. Lepetit, C. Strecha, and P. Fua, “BRIEF: Binary Robust Independent Elementary Features”, in
Proceedings of ECCV 2010, pp. 778–792.
