CH-6 COMPUTER VISION

Write a detailed note on the optical flow algorithm with a

suitable example.

Optical Flow Algorithm

Definition: Optical flow refers to the pattern of apparent motion of objects, surfaces, or
edges in a visual scene, caused by the relative motion between the camera and the scene.
The optical flow algorithm computes the motion between two consecutive image frames
captured at different times.

Key Assumptions:

1. Brightness Constancy: The pixel intensity of an object remains constant between

consecutive frames.

2. Small Motion: The motion of objects between two frames is small enough to
approximate the change linearly.

3. Spatial Coherence: Neighboring pixels in the same object move in a coherent manner.

Mathematical Formulation: The brightness constancy assumption is expressed as:

I(x, y, t) = I(x + u, y + v, t + 1)
Where:

I(x, y, t) is the intensity of a pixel at position (x, y) in the first frame.

(u, v) are the motion components (optical flow vectors).
t is the time frame.

Using Taylor series expansion and neglecting higher-order terms, the equation simplifies to:

Ix u + Iy v + It = 0
​ ​ ​

Where:

Ix and Iy are the spatial intensity gradients.

​ ​

It is the temporal intensity gradient.

​

This equation is called the Optical Flow Constraint Equation. Since there is one equation but
two unknowns (u and v ), additional constraints are required to solve it.

Methods to Solve Optical Flow:

1/12
 1. Lucas-Kanade Method:

Assumes motion is constant within a small neighborhood.

Uses least-squares fitting to solve for motion vectors.

Suitable for small, smooth motion.

2. Horn-Schunck Method:

Imposes a smoothness constraint to reduce noise.

Solves for dense optical flow (motion at every pixel).

Example: Consider a video of a car moving from left to right.

In frame 1, the car's position is at x1 ​ = 10, and in frame 2, it moves to x2 = 20.

​

The optical flow algorithm calculates the displacement (u = 20 − 10 = 10) and

assumes v = 0 (no vertical movement).
The resulting motion vector is (u, v) = (10, 0).

Applications:

Motion detection (e.g., detecting moving objects in surveillance footage).

Object tracking (e.g., tracking vehicles or people in video).

Video stabilization (e.g., reducing camera shake in video recordings).

Robotics (e.g., estimating motion for navigation).

By using the optical flow algorithm, it becomes possible to analyze motion patterns
effectively in various real-world scenarios.

Explain the image brightness constancy equation in brief.

The Image Brightness Constancy Equation is a fundamental assumption in motion analysis

and optical flow. It states that the intensity of a pixel in an image remains constant over time
as the pixel moves in the scene.

Equation:

I(x, y, t) = I(x + u, y + v, t + 1)

Where:

I(x, y, t): Intensity of a pixel at location (x, y) at time t.

2/12
 (u, v): Displacement or motion vector of the pixel in the next frame.
t + 1: The next time frame.

Explanation:
The equation assumes that as an object moves, its appearance (intensity) does not
change. Only its position changes in the image over time.

It forms the basis for optical flow calculation by relating spatial and temporal changes in
intensity.

Simplification:
Using Taylor series expansion and ignoring higher-order terms, the equation is
approximated as:

Ix u + Iy v + It = 0
​ ​ ​

Where:

Ix , Iy : Spatial gradients of the intensity in the x- and y -directions.

​ ​

It : Temporal gradient (rate of change of intensity over time).

​

u, v : Motion components (optical flow vectors).

This equation is used to compute motion in a scene, but additional constraints are needed
since it has one equation with two unknowns (u and v ).

Discuss the concepts of motion parallax in brief.

Motion Parallax
Definition:
Motion parallax is a depth cue that arises when an observer moves relative to a scene.
Objects at different distances from the observer appear to move at different relative speeds
and directions, providing depth information.

Key Concepts:
1. Relative Motion:

Closer Objects: Appear to move faster relative to the observer.

Distant Objects: Appear to move more slowly.

3/12
 2. Direction of Motion:

Objects closer than the focal point appear to move in the opposite direction to the
observer's motion.

Objects farther than the focal point appear to move in the same direction as the
observer.

3. Depth Perception:
The rate and direction of relative motion help the brain estimate the depth and spatial
arrangement of objects in the scene.

Example:
When you look out of a moving car:

Trees near the road seem to move rapidly in the opposite direction.

Mountains far away seem to move slowly in the same direction as your car.

Applications:
Computer Vision: Used in 3D scene reconstruction and depth estimation.

Virtual Reality (VR): Enhances depth perception in immersive environments.

Robotics: Helps robots perceive the spatial arrangement of objects.

Motion parallax is a natural and powerful depth cue utilized by humans and machines to
understand 3D space.

Discuss optical flow algorithm in brief.

Optical Flow Algorithm

Definition:
Optical flow refers to the apparent motion of objects, edges, or regions in a visual scene
caused by relative motion between the camera and the scene. The optical flow algorithm
calculates this motion between two consecutive image frames.

Key Assumptions:
1. Brightness Constancy: The intensity of a moving pixel remains constant over time.

2. Small Motion: The movement between two frames is small and smooth.

3. Spatial Coherence: Neighboring pixels have similar motion.

4/12
Optical Flow Constraint Equation:
The algorithm uses the brightness constancy assumption:

Ix u + Iy v + It = 0
​ ​ ​

Where:

Ix , Iy : Intensity gradients in the x- and y -directions.

​ ​

It : Temporal intensity gradient (change over time).

​

u, v : Optical flow components (motion vectors).

This equation has one equation but two unknowns (u, v ), requiring additional constraints to
solve.

Common Methods:
1. Lucas-Kanade Method: Assumes motion is constant in small regions and solves using
least squares.

2. Horn-Schunck Method: Imposes a smoothness constraint to compute dense optical

flow.

Applications:
Motion Tracking: Tracks moving objects in videos.

3D Scene Reconstruction: Analyzes depth and motion in a scene.

Surveillance: Detects abnormal movements.

Robotics: Helps in navigation by estimating motion.

Optical flow algorithms are fundamental for understanding and analyzing motion in
computer vision systems.

Describe feature-based motion field estimation techniques

in details.

Feature-Based Motion Field Estimation Techniques

Feature-based motion field estimation involves identifying and tracking specific,
distinguishable features (e.g., corners, edges, or textures) across consecutive frames of a

5/12
video or image sequence to estimate motion. These techniques focus on sparse motion
estimation rather than dense motion at every pixel.

Key Steps in Feature-Based Motion Estimation:

1. Feature Detection:

Identify robust and easily trackable features in an image.

Common methods include:

Harris Corner Detector: Finds corners where intensity gradients change

significantly.

SIFT (Scale-Invariant Feature Transform): Detects scale- and rotation-invariant

features.

SURF (Speeded-Up Robust Features): A faster alternative to SIFT.

2. Feature Matching:

Match detected features between consecutive frames using descriptors.

Methods:

Euclidean Distance: Compares feature descriptors to find the best match.

Nearest-Neighbor Search: Finds the closest matching feature in the next frame.

3. Motion Vector Calculation:

Compute the displacement (motion vector) of matched features between frames.

Motion vector: (u, v), where:

u: Horizontal displacement.
v : Vertical displacement.
4. Outlier Removal (Optional):

Remove incorrect matches using techniques like RANSAC (Random Sample

Consensus) to ensure robust motion estimation.

6/12
Example:
Consider tracking the motion of a car in a video:

1. Feature Detection: Identify edges, corners, or other key points on the car.

2. Feature Matching: Match these points across consecutive frames.

3. Motion Vector Calculation: Compute the motion vectors for these points to determine
the car's direction and speed.

Advantages:
Efficiency: Focuses on specific features, making it computationally faster than dense
motion estimation.

Robustness: Works well for tracking prominent objects and motion in noisy images.

Limitations:
Sparsity: Only estimates motion at feature points, not for every pixel.

Occlusion Issues: Features may disappear when occluded, leading to incomplete motion
tracking.

Dependence on Feature Quality: Requires well-defined features; poor-quality features

reduce accuracy.

Applications:
Object Tracking: Follows moving objects in video sequences (e.g., vehicles, people).

Motion Analysis: Analyzes object trajectories for activities or behaviors.

Augmented Reality: Tracks objects to overlay digital content accurately.

Robot Navigation: Helps robots estimate motion and navigate through environments.

Feature-based motion estimation is a foundational technique in computer vision, balancing

efficiency and accuracy for many practical applications.

7/12
 What is optical flow? Discuss optical flow in motion analysis.

Optical Flow
Definition:
Optical flow is the apparent motion of objects, edges, or regions in a visual scene, caused by
the relative movement between the camera and the scene. It represents the distribution of
motion vectors, which indicate how pixels in an image frame move to corresponding pixels in
the next frame.

Optical Flow in Motion Analysis

Optical flow plays a crucial role in understanding and analyzing motion in computer vision. It
helps estimate the direction and velocity of motion within a scene, providing valuable
insights into object movements.

Key Concepts:

1. Motion Vectors:
Optical flow calculates motion as a vector field, where each vector describes the
displacement of a pixel between two frames.

2. Optical Flow Constraint Equation:

Based on the assumption of brightness constancy, optical flow is governed by:

Ix u + Iy v + It = 0
​ ​ ​

Where:

Ix , Iy : Spatial intensity gradients.

​ ​

It : Temporal intensity gradient.

​

u, v : Motion components (horizontal and vertical displacements).

3. Dense vs. Sparse Flow:

Dense Optical Flow: Estimates motion for every pixel (e.g., Horn-Schunck method).

Sparse Optical Flow: Tracks motion at specific feature points (e.g., Lucas-Kanade
method).

8/12
Applications in Motion Analysis:

1. Object Tracking:
Optical flow is used to track moving objects across frames in a video sequence, such as
pedestrians, vehicles, or animals.

2. Action Recognition:
Motion patterns derived from optical flow help classify human activities, such as walking,
running, or jumping.

3. Video Stabilization:
Detects and compensates for unwanted camera movements to produce stable video
output.

4. 3D Scene Reconstruction:
Uses motion analysis to infer depth and reconstruct 3D environments.

5. Autonomous Driving:
Helps detect obstacles, estimate object speeds, and predict motion in self-driving
vehicles.

Example:
In a video of a car driving down a road, optical flow can:

Detect the car's motion by identifying regions with significant displacement.

Compute motion vectors to estimate the car's speed and direction.

Advantages:
Captures detailed motion information.

Works for both rigid and non-rigid object motion.

Limitations:
Sensitive to lighting changes and noise.

Struggles with large displacements or occlusions.

9/12
In motion analysis, optical flow provides a robust framework for understanding and
interpreting motion dynamics, making it a cornerstone in fields like robotics, surveillance,
and augmented reality.

Discuss basics of the motion field of rigid objects with

necessary equations.

Motion Field of Rigid Objects

Definition:
The motion field refers to the apparent motion of image points in a visual scene, induced by
the relative movement between the observer (camera) and rigid objects in the scene. It is a
projection of the 3D motion of objects onto a 2D image plane.

When dealing with rigid objects, the motion field is determined by the motion of the
observer and the geometric properties of the scene.

Key Concepts:
1. Rigid Object Motion:

Rigid objects maintain a fixed shape and structure as they move.

Their motion can be described by a combination of translation and rotation.

2. 2D Motion Field Representation:

Each point in the 3D scene projects onto a corresponding point in the 2D image
plane.

The velocity of these points on the image plane constitutes the motion field.

Equations of Motion Field:

3D Motion of a Point:

10/12
Let a 3D point in the scene be represented as (X, Y , Z). If the camera has a translational
velocity T = (Tx , Ty , Tz ) and a rotational velocity Ω = (Ωx , Ωy , Ωz ), the 3D velocity V of
​ ​ ​ ​ ​ ​

the point is given by:

V =T+Ω×P

Where:

P = (X, Y , Z) is the 3D position of the point.

× denotes the cross product.

2D Motion Field in the Image Plane:

For a point projecting onto the image plane at (x, y), the motion field (u, v) is expressed as:

Tx xTz
u=− + + xyΩx − (1 + x2 )Ωy + yΩz
​ ​

​ ​ ​ ​ ​

Z Z
Ty yTz
v=− + + (1 + y 2 )Ωx − xyΩy − xΩz
​ ​

​ ​ ​ ​ ​

Z Z
Where:

u, v : Components of the 2D motion vector in the image plane.

Z : Depth of the point in 3D space.
x, y : Normalized coordinates on the image plane.

Observations:
1. Effect of Depth (Z ):

Points closer to the camera (Z small) exhibit larger motion vectors.

Distant points (Z large) appear to move more slowly.

2. Direction of Motion:

The direction and magnitude of motion depend on the translation T and rotation Ω
of the camera relative to the object.

11/12
Applications:
1. Depth Estimation: By analyzing the motion field, the depth (Z ) of objects in the scene
can be estimated.

2. Object Tracking: The motion field helps track rigid objects across image sequences.

3. 3D Scene Reconstruction: Combines motion field information to infer the 3D structure

of a scene.

4. Egomotion Estimation: Estimates the motion of the observer (camera) relative to the
environment.

Summary:
The motion field of rigid objects provides a mathematical description of how 3D motion
translates into observable 2D image motion. Using equations based on translation, rotation,
and depth, it enables critical tasks such as depth estimation, scene reconstruction, and
object tracking in computer vision.

12/12