What is Computer Vision?

The science of
? facilitating
human-like
capability of
seeing and
understanding
the environment
to a machine or
computer.

1
http://www.bmva.org/visionoverview
 Computing structure from
binocular vision

Given two views of the scene

recover 3D scene structure

Courtesy: www.cs.gmu.edu/~kosecka/lect4.ppt
 4

2
1

1 2 3 4

Approach: unwarp then measure

What kind of warp is this?
CSE 576, Spring 2008 Projective Geometry 3
Outline
 Image formation
 Projective Geometry
 Camera Geometry
 Stereo Geometry
 Depth estimation
Image formation in optical
camera

P  p : Projection

Center of projection

O
p

Projection plane
Projective Geometry
Real Space and Projective
Space (2D)
p(kx,ky,k)

p(x,y) p(x,y,1)
y

x
o

y
z
x
o
Homogeneous
p(x,y) p(kx,ky,k) Coordinate system
Homogeneous Representation

Singular point in the projective space.

 In projective geometry, a singular point refers to a
point where a curve or surface becomes degenerate,
often leading to non-smooth behavior.

 The projective space is an extension of Euclidean

space that includes points at infinity, and it is used to
study properties that remain invariant under
projective transformations (collinearity, cross-ratios,
etc.)
Homogeneous Representation

Does not belong to P2.

 Homogeneous representation
of a line in a plane
l(ka,kb,kc)
ax+by+c=0

p(x,y) l(a/c,b/c,1)
y

x
o

A point in P2 A line in P2 b
c
a
o

Point containment in P2
 Points and lines in P2

l:ax+by+c=0 l1 l2
x1(x1,y1) p(x,y)
y y
x2(x2,y2)

x x
o o

Exactly one line through two points.

Exactly one point at intersection of two lines.
 Examples
1. Compute the line passing through (3,5) and (5,0) in
a plane.

2. Compute the point of intersection of the lines:

5x-2y+4=0 and 6x-7y-3=0.
 Duality
x l
x Tl  0 lT x  0
x  l l' l  x x'
Duality principle:
To any theorem of 2-dimensional projective geometry
there corresponds a dual theorem, which may be
derived by interchanging the role of points and lines
in the original theorem.
Points and lines in a plane
l(ka,kb,kc)

ax+by+c=0
p(x,y) l(a/c,b/c,1)

y b
z c
o x
a
o
x Tl  0 lT x  0
x  l l' l  x x'
 In projective geometry, a scaling factor refers to a value that
scales or resizes a point, line, or other geometric object while
preserving its projective properties.

 Projective geometry is concerned with the study of geometric

properties that remain invariant under projective
transformations, which include rotations, translations, scalings,
and perspective transformations.
 Intersection of parallel lines

l1:ax+by+c1=0

y X(x,y,1)

x
o l2:ax+by+c2=0
(b, -a, 0)
y
c
x
o

Ideal Point
 In projective geometry, points at infinity cannot be
assigned Cartesian coordinates like regular points in
Euclidean geometry. Instead, ideal points are
introduced to handle these cases.

 These ideal points are not located within the

Euclidean plane but are associated with specific
directions or limits of perspective.
 Here's why ideal points are connected to the notion of direction:

 Parallel Lines: In Euclidean geometry, parallel lines never meet.

However, in projective geometry, parallel lines can be thought of as
intersecting at an ideal point at infinity. This ideal point represents
the direction of the parallel lines.

 Intersecting Lines: When two lines intersect in Euclidean geometry,

they do so at a single point. In projective geometry, two lines
intersect in a single point, but that point can be considered an ideal
point if the lines are parallel. This concept captures the idea that
intersecting lines can also be seen as converging toward a common
direction at infinity.
 Visual Perspective: Imagine looking down a straight
road. As the road gets farther and farther away, it
appears to converge at a single point on the horizon.
In projective geometry, this convergence point can
be thought of as an ideal point at infinity,
representing the direction in which the road is
heading.
 Meaning of an ideal point

Intersection point

(b, -a, 0)

A direction !
 Ideal points

Ideal points: Points on the X-Y plane or principal plane

parallel to projection plane.

For canonical coordinate

system, they are of the form:

An ideal point denotes a direction toward infinity!

 l(ka,kb,kc)

Line at infinity
l(a/c,b/c,1)

=0

Any ideal point c

a
 In projective geometry, the concept of the "line at
infinity" is a fundamental idea that extends the
concept of lines in Euclidean geometry.

 The line at infinity doesn't correspond to a physical

line but rather represents a way to handle points at
infinity and maintain certain projective properties.
 In projective geometry, lines are considered to be the
same if they intersect at a point at infinity.

 The line at infinity is introduced to ensure that every

pair of distinct lines intersects at a unique point,
including parallel lines.

 This concept helps maintain the property that any two

lines in projective space intersect at exactly one
point.
 A model for the projective plane
l(ka,kb,kc)

A plane passing
l(a/c,b/c,1)
through the origin.

b
A straight line passing
through the origin. c
a

From Hartley and Zisserman, “Multiple view geometry in computer vision”, Cambridge Univ. Press.
(2000)
 Projection of parallel lines
from any arbitrary plane
Canonical projection plane
(CPP)

Vanishing Point
(b, -a, 0)
y
z
x
o

Ideal Plane
 Vanishing points
Vanishing Points
corresponding to parallel
lines of a plane lie on a
line, called vanishing line.
Vanishing
Line
Vanishing Points
A real life example

Vanishing points
A journey toward infinity ….
  xTl=0, and lTx=0
 x=l x l’, and l=x x x’
Summary
 A point in a 2-D projective space represents a ray
passing through origin of an implicit 3D space.
 Requires additional dimension for representation.

 Homogeneous Coordinate Representation

 Straight lines in R2 are elements of a 2D projective
space.
 Points and lines hold duality theorem..
Camera Geometry
Image formation in optical
camera

O
p
 Pinhole camera
Z Principal axis
Y
World coordinate
Focal
P (X,Y,Z)
length f o
p(x,y) Image coordinate
C
Image plane
Camera Center
Principal plane X
(XY-plane)
Pinhole Camera:
Mapping from P P
3 2

Projection Matrix (P)

Non-Invertibility
 Unlike homography, which is a mapping from one 2D projective
space to another 2D projective space (P² → P²) and is invertible
(i.e., the mapping is one-to-one and has an inverse), the
projection matrix in the pinhole camera model is not invertible.
This is because:

 The matrix is not square (it’s 3x4), so it does not have a determinant and
therefore cannot have an inverse in the traditional sense.

 The mapping from 3D to 2D is not one-to-one. A single point in the 2D

image plane can correspond to an infinite number of points along a ray in
3D space. This phenomenon is due to the loss of depth information when
projecting from 3D to 2D.
Non-Invertibility
 When you project a point from 3D space onto a 2D image plane
using the pinhole camera model, the result is a single point on
the image plane.

 However, if you try to reverse this process and project back to

3D space, you get a projection ray rather than a unique point.
Any point along this ray in 3D space could correspond to the
same point on the 2D image plane.

 This lack of a unique inverse is why the mapping is not

invertible and is a key distinction from homography.
 Offset of principal point
Z p(px + x, py + y)
Y
o’
y’
x’ o P o(px,py)
p

O Principal
point
X
 Projection Matrix under the
offset

K (Camera Calibration Matrix)

 Shifting of world coordinate
Yc Zc

Y
P
Z p

C
R,t
Xc
O X
CCD Camera
 A charge-coupled device (CCD) is a
light-sensitive integrated circuit that
captures images by converting photons
to electrons.

 A CCD sensor breaks the image

elements into pixels.
CCD Camera model

No. of pixels
per unit length
 General Projective Camera

3
3
11 d.o.f

5
Extrinsic parameters: R, t
Intrinsic parameters: K
 Properties of projective
camera matrix P=[M | p4]
Rank of P: 3; # of extrinsic params: 6
Size: 3x4; # of intrinsic params: 5
d.o.f.=11;

Two independent equations

Minimum # of point correspondences between world

and image coordinates required to estimate P: 6
Estimation of the camera
matrix (P)

Redundant, as xi x (1) + yi x (2)= wi x (3)

 Estimation of the camera
matrix (P)

For n correspondences

Minimize ||Ap|| subject to ||p||=1

Use similar techniques, such as DLT.
 Principal plane, axis, and point
Principal Axis Zc
Yc

Principal point
Y
P
Z p

C
R,t Principal plane

O X Xc
Properties of projective
camera matrix P=[M | p4]
Properties of projective
camera matrix P=[M | p4]
 Properties of projective
camera matrix P=[M | p4]
 The projective camera matrix is a 3x4 matrix that describes the
mapping of a pinhole camera from 3D points in the world to 2D points
in an image. It has the following properties:

 It is a projective transformation, which means that it preserves lines

and preserves collinearity.

 It has 11 degrees of freedom.

 The first three columns of the matrix define the camera's intrinsic
parameters, which are the focal length, the principal point, and the
skew.
 The last column of the matrix defines the camera's extrinsic parameters,
which are the camera's position and orientation in the world.

 Camera center cannot be projected to a point in the image.

 The image of a point at infinity is a line that passes through the camera
center.

 The projective camera matrix is a fundamental concept in computer

vision. It is used in a variety of tasks, such as camera calibration, 3D
reconstruction, and object tracking.
 Applications of the projective
camera matrix
 Camera calibration: The projective camera matrix can be used to
calibrate a camera, which means determining the intrinsic and extrinsic
parameters of the camera. This is necessary for accurate 3D
reconstruction and other computer vision tasks.

 3D reconstruction: The projective camera matrix can be used to

reconstruct 3D objects from a set of 2D images. This is done by finding
the intersection of the rays that project the 2D points to the camera.

 Object tracking: The projective camera matrix can be used to track

objects in a sequence of images. This is done by finding the 2D points
that correspond to the same object in each image.
 Difference between pinhole cameras and
cameras with lenses
 Principle of Operation:
 Pinhole Camera: In a pinhole camera (also known as a
camera obscura), light from a scene passes through a tiny
aperture (the "pinhole") and projects an inverted image of
the scene onto a surface inside the camera. This works
based on the principle of light traveling in straight lines.

 Cameras with Lenses: Cameras with lenses use optical

lenses to focus light from the scene onto a photosensitive
medium (like film or a digital sensor). Lenses manipulate the
way light rays converge to create a sharp, properly oriented
image.
 Image Quality:
 Pinhole Camera: Pinhole cameras produce images with
soft focus and uniform sharpness across the image due to
the nature of light diffraction through the small aperture.
However, they generally lack high detail and sharpness.

 Cameras with Lenses: Lenses allow for precise control over

focus, aperture, and depth of field, resulting in images
with higher resolution, clarity, and varying degrees of
background blur.
 Aperture and Light Gathering:
 Pinhole Camera: The pinhole aperture is very small, which
limits the amount of light entering the camera. This can
result in longer exposure times.

 Cameras with Lenses: Lenses can have adjustable apertures

that control the amount of light entering the camera. Larger
apertures allow more light in, enabling faster shutter speeds
and better performance in low light conditions.

 Inversion of Image:
 Pinhole Camera: Pinhole cameras inherently produce
inverted images, where the top of the scene is at the
bottom of the image and vice versa.

 Cameras with Lenses: Lenses and lens systems can be

designed to correct for image inversion, resulting in images
that match the orientation of the actual scene.
 Practical Use:
 Pinhole Camera: Pinhole cameras are often used for artistic
or educational purposes due to their unique image
characteristics.

 Cameras with Lenses: Cameras with lenses are widely used

for professional photography, cinematography, scientific
imaging, and everyday photography due to their versatility
and ability to produce high-quality images.
 The concept of "cameras at infinity" is often used in computer graphics
and computer vision to simplify certain calculations and modeling tasks.

 It's a mathematical abstraction that helps make calculations more

tractable and intuitive, especially when dealing with perspective
projections and 3D transformations.

 A camera is typically used to simulate the process of capturing a 3D

scene and projecting it onto a 2D image plane. This involves defining a
camera position, orientation, field of view, and other parameters that
determine how the 3D world is projected onto the 2D plane.
 This abstraction eliminates the need to deal with certain
complex mathematical calculations that arise when dealing
with finite distances and perspective projections.

 When a camera is placed at infinity, the rays of light from

each point in the scene become effectively parallel,
simplifying the projection process.
 Key points about cameras at infinity
 Parallel Projection: With a camera at infinity, the projection becomes a
parallel projection rather than a perspective projection. This means that
lines that are parallel in the 3D scene remain parallel in the projected 2D
image.

 Preservation of Shapes and Angles: Because parallel lines remain

parallel, shapes and angles are preserved in the projection. This
simplifies various tasks like transformations and measurements.

 Translation-Invariant: The camera's position no longer matters since it's

effectively infinitely far away. This simplifies calculations involving
translation transformations.
 Depth Information: Cameras at infinity lose depth information. Objects
at different distances from the camera are projected with the same size,
making it impossible to distinguish between objects that are close and
those that are far away.

 Use Cases: Cameras at infinity are particularly useful for scenes where
depth is not critical, and you want to simplify calculations. They are
often used in architectural rendering, certain types of technical
drawings, and other scenarios where maintaining accurate perspective
isn't essential
 It's important to note that "cameras at infinity" is a
mathematical construct and not a physical reality.

 In real-world photography and computer graphics, cameras

have finite positions and finite lenses, leading to perspective
projections that accurately represent how objects in the real
world appear in 2D images.

 However, the concept of cameras at infinity is a valuable

tool for simplifying calculations and gaining insight into
certain aspects of geometry and transformations.
 An affine camera is a mathematical model used in
computer vision and computer graphics to represent the
projection of a 3D scene onto a 2D image plane.

 The affine camera model is a simplification of the

perspective camera model that preserves certain
properties, making it useful for various applications where
strict perspective projection is not required or where linear
transformations are sufficient.
 The affine camera model is based on the concept of affine
transformations, which include translation, rotation, scaling,
and shear.

 In contrast to the perspective camera model, which involves

non-linear depth-dependent transformations, the affine
camera model assumes that objects remain parallel and
have uniform scaling in the direction perpendicular to the
image plane.
 Key characteristics of an affine
camera model
 Parallel Lines Remain Parallel: In the projection, lines that
are parallel in the 3D scene remain parallel in the 2D image.
This property simplifies calculations involving objects that are
far away from the camera.

 Uniform Scaling in One Direction: The model assumes

uniform scaling along one direction perpendicular to the
image plane. This means that objects may appear distorted
in one direction, but their relative scales within the image are
preserved.
 Translation and Rotation: The affine camera model includes
translation and rotation transformations, allowing for
changes in the camera's position and orientation.

 Limited Depth Information: The model doesn't capture true

perspective effects, so objects at different depths are not
accurately portrayed. Depth information is typically limited,
and the model is most suitable for scenarios where this
limitation is acceptable.
 The affine camera model is particularly useful in scenarios
where more complex perspective transformations are not
needed, such as certain types of architectural rendering,
technical drawings, and image-based measurements.

 It simplifies computations by employing linear

transformations and preserving certain geometric
properties.
 It's important to note that while the affine camera model
provides simplifications that are advantageous in some
contexts, it's still an abstraction and doesn't fully capture
the complexity of real-world cameras and their perspective
projections.

 For applications requiring accurate rendering of perspective

and depth, the full perspective camera model or even more
advanced models may be necessary.
Non-affine: Otherwise
 Affine projection

o Affine projection matrix: 8 d.o.f.

o For estimating the matrix, it requires four point
correspondences.
 Affine Camera

Camera Center Direction of parallel rays (d)

 Estimation of an affine camera

For n points
 Summary
o Pinhole camera model provides the projection
matrix which maps a 3D point to an image point.
o Projection matrix:
o 3x4
o Dof: 11
o 5 intrinsic parameters and 6 extrinsic parameters.
o Minimum 6 point correspondences required for
estimation
o Affine projection matrix
o Last row [0 0 0 1]T
o Dof:8
o Minimum 4 point correspondences required to estimate.
 Summary (contd.)
o Geometry encoded in a projection matrix
o P=[ M | p4] or P=[p1 p2 p3 p4] or P=[r1T ; r2T ; r3T]
o Camera Center: -M-1p4
o For affine projection matrix: Right zero of M (A direction).
o Vanishing points
o X-axis: p1
o Y-axis: p2
o Z-axis: p3
o Image of world origin: p4
o Special planes passing through the camera center
o Principal plane: r3TX=0
o Direction of optical axis: <r31, r32, r33>
o Principal point: M [r31 r32 r33]T
o Plane formed with x-axis of image coordinate system: r1TX=0
o Plane formed with y-axis of image coordinate system: r2TX=0
 Summary (contd.)
o Geometric derivatives from Projection Matrix:
P= [M|p4]
o Projection ray formed at image point x.
o Direction ratio: M-1x
o A point on the ray:
o Camera center (-M-1p4)