Computer Vision I

CSE 252A, Winter 2007

David Kriegman

Homography Estimation∗

1. From 3D to 2D Coordinates
Under homography, we can write the transformation of points in 3D from camera 1 to camera 2 as:

X2 = HX1 X1 , X2 ∈ R 3 (1)

In the image planes, using homogeneous coordinates, we have

λ1 x 1 = X 1 , λ2 x2 = X 2 , therefore λ2 x2 = Hλ1 x1 (2)

This means that x2 is equal to Hx1 up to a scale (due to universal scale ambiguity). Note that x2 ∼ Hx1 is a
direct mapping between points in the image planes. If it is known that some points all lie in a plane in an image 1 , the
image can be rectified directly without needing to recover and manipulate 3D coordinates.

2. Homography Estimation
To estimate H, we start from the equation x2 ∼ Hx1 . Written element by element, in homogenous coordinates we
get the following constraint:

    
x2 H11 H12 H13 x1
 y2  =  H21 H22 H23   y1  ⇔ x2 = Hx1 (3)
z2 H31 H32 H33 z1

In inhomogenous coordinates (x02 = x2 /z2 and y20 = y2 /z2 ),

H11 x1 + H12 y1 + H13 z1
x02 = (4)
H31 x1 + H32 y1 + H33 z1
H21 x1 + H22 y1 + H23 z1
y20 = (5)
H31 x1 + H32 y1 + H33 z1
Without loss of generality, set z1 = 1 and rearrange:

x02 (H31 x1 + H32 y1 + H33 ) = H11 x1 + H12 y1 + H13 (6)

y20 (H31 x1 + H32 y1 + H33 ) = H21 x1 + H22 y1 + H23 (7)

We want to solve for H. Even though these inhomogeneous equations involve the coordinates nonlinearly, the
coefficients of H appear linearly. Rearranging equations 6 and 7 we get,

aTx h = 0 (8)
aTy h = 0 (9)
∗ Adapted from lecture notes from CSE252b Spring 2004 with permission from Serge Belongie.
1 For cameras related by a pure rotation, every scene point can be thought of as lying on a plane at infinity.

1
where

(10)
T
h = (H11 , H12 , H13 , H21 , H22 , H23 , H31 , H32 , H33 )
(11)
T
ax = (−x1 , −y1 , −1, 0, 0, 0, x02x1 , x02 y1 , x02 )
(12)
T
ay = (0, 0, 0, −x1, −y1 , −1, y20 x1 , y20 y1 , y20 ) .

Given a set of corresponding points, we can form the following linear system of equations,

Ah = 0 (13)

where
aTx1
 
 aTy1 
..
 
A= . . (14)
 
 
 aTxN 
aTyN

Equation 13 can be solved using homogeneous linear least squares, described in the next section.

3. Homogeneous Linear Least Squares

We will frequently encounter problems of the form

Ax = 0 (15)

known as the Homogeneous Linear Least Squares problem. It is similar in appearance to the inhomogeneous linear
least squares problem

Ax = b (16)

in which case we solve for x using the pseudoinverse or inverse of A. This won’t work with Equation 15. Instead we
solve it using Singular Value Decomposition (SVD).
Starting with equation 13 from the previous section, we first compute the SVD of A:
9
(17)
X
A = U ΣV > = σi ui vi>
i=1

When performed in Matlab, the singular values σi will be sorted in descending order, so σ9 will be the smallest. There
are three cases for the value of σ9 :
• If the homography is exactly determined, then σ9 = 0, and there exists a homography that fits the points exactly.
• If the homography is overdetermined, then σ9 ≥ 0. Here σ9 represents a “residual” or goodness of fit.
• We will not handle the case of the homography being underdetermined.
From the SVD we take the “right singular vector” (a column from V ) which corresponds to the smallest singular
value, σ9 . This is the solution, h, which contains the coefficients of the homography matrix that best fits the points.
We reshape h into the matrix H, and form the equation x2 ∼ Hx1 .

4. Homogeneous Linear Least Squares Alternate Derivation

Starting again with the equation Ah = 0, the sum squared error can be written as,
1
(18)
T
f (h) = (Ah − 0) (Ah − 0)
2
1
(19)
T
f (h) = (Ah) (Ah)
2
1 T T
f (h) = h A Ah. (20)
2

2
Taking the derivative of f with respect to h and setting the result to zero, we get

d 1 T
A A + (AT A)T h (21)


f =0 =
dh 2
0 = AT Ah. (22)

Looking at the eigen-decomposition of AT A, we see that h should equal the eigenvector of AT A that has an eigenvalue
of zero (or, in the presence of noise the eigenvalue closest to zero).
This result is identical to the result obtained using SVD, which is easily seen from the following fact,
Fact 1 Given a matrix A with SVD decomposition A = U ΣV T , the columns of V correspond to the eigenvectors of
AT A.