COMPUTER VISION

(an introduction)
Dr Faisal Mufti ECE 7259 Advanced Computer Vision Spring 2012

ECE 7259 Advanced Computer Vision

About the course

Classes 7:30-9:00 (Tue) 6:00-7:30 (Thu) Room 2 ground floor (opp. library) Books
Computer Vision, A modern approach, David A. Forsyth, Jean Ponce Multiple View Geometry in Computer Vision, Cambridge University

Press, R. Hartley and A. Zisserman An invitation to 3D vision, Y. Ma, S. Soatto, J. Kosecka, and. S.S. Sastry, Springer-Verlag. Lecture notes
Web resources, Journals/Conferences
IEEE Trans Pattern Analysis and Machine Intelligence CVPR, ICIP

ECE 7259 Advanced Computer Vision

About the course

Course eGroup address: http://groups.yahoo.com/group/CASE_CV_SP_12 [email protected] Office hours: request meeting by email Contact: [email protected] or [email protected] You contact me by email I contact you by email or via eGroup: All announcement, home assignments, and guidelines will be distributed by email. Course TA Zakirullah [email protected]
You must send an email by Jan 31 from your active address with the subject CV course contact.

Those who do not send their contact address on time will not be added to the contact list!!!

ECE 7259 Advanced Computer Vision

About the course

Home assignments
Penalties

25% (3x assignments) 25% 25%

5% 5% 7.5% 7.5%

Midterm Project
Synopsis Mid review Final presentation

Project report

Final Exam-

25%

ECE 7259 Advanced Computer Vision

About the course

Home assignments
Penalties

25% (3x assignments) 25% 25%

Midterm Project
Synopsis Mid review

Attitude = 100%

Final presentation

Project report

5% 5% 7.5% 7.5%

Final Exam-

25%

ECE 7259 Advanced Computer Vision

Pre-requisites

- Everyone should have access to MATLAB

ECE 7259 Advanced Computer Vision

Scope of ECE 7259

Advance graduate course on image/video analysis Prerequisites:
ECE 6258 Digital Image Processing (desirable not mandatory) Not assuming you have much exposure on image processing at

undergraduate level Understanding of vectors and spaces and geometry is desirable. Everyone should have access to MATLAB

Emphasis on fundamental concepts

Provide theoretical foundations on multi-dimensional Image and Computer

Vision Coupled with assignments and projects for hands-on experience and reinforcement of the concepts

Follow-up courses
Machine Vision, Pattern recognition, Machine learning Multimedia communications and Visual Computing

ECE 7259 Advanced Computer Vision

What is Computer Vision?

Vision is the act of knowing what is where by looking. --Aristotle

ECE 7259 Advanced Computer Vision

What is Computer Vision?

Vision is the primary sense by which humans perceive the world. Understanding is much more than simply seeing, e.g. the 3D magic eye illusions that appear nonsensical at first. Computer vision aims to bestow this perceptive ability on computers, via intelligent interpretation of visual data.

ECE 7259 Advanced Computer Vision

10

3D Magic Eye

ECE 7259 Advanced Computer Vision

11

3D Magic Eye

ECE 7259 Advanced Computer Vision

12

Perception and vision

How much of a humans understanding of the world through vision is perceptual and how much is physical.

ECE 7259 Advanced Computer Vision

13

Perception and colour

How much of a humans understanding of the world through vision is perceptual and how much is physical.

ECE 7259 Advanced Computer Vision

14

What makes it hard?

What are the challenges:
Image resolution Noise Projection (no depth)

Complexity of environment
Base level human vision is too

good and it is backed by a perceptual engine that cannot be matched in an engineered system.

ECE 7259 Advanced Computer Vision

15

Blind-Spot Experiment
Draw an image similar to that below on a piece of paper (the dot and cross are about 6 inches apart)

Close your right eye and focus on the cross with your left eye Hold the image about 20 inches away from your face and move it slowly towards you The dot should disappear!

ECE 7259 Advanced Computer Vision

16

What is Computer Vision?

It is related to, but not equivalent to:
Photogrammetry Image Processing Artificial Intelligence

ECE 7259 Advanced Computer Vision

17

What is Computer Vision?

Computer vision combines elements of:
Image processing: manipulation of images

Geometry: modeling of shape

Engineering: signal processing, filtering, tracking, robotics Computer Science: Algorithmic efficiency,

Interpretation: AI, statistical decision tools.

To achieve this we require:

Visual information (signals) that can be input to a computer,

digital images are ideal for this. 2D image processing techniques for analysing these signals. 3D projective geometry for building a 3D model of the world from our data. Statistical tools to make decisions with confidence bounds. Algorithms to put it all together.

ECE 7259 Advanced Computer Vision

18

Why study Computer Vision?

Images and movies are everywhere Fast-growing collection of useful applications
building representations of the 3D world from pictures automated surveillance (whos doing what) movie post-processing face finding

Various deep and attractive scientific mysteries

how does object recognition work?

Greater understanding of human vision

ECE 7259 Advanced Computer Vision

19

Goal and objectives

Introduce the fundamental problems of computer vision. Introduce the main concepts and techniques used to solve those problems. Enable students to implement vision algorithms Enable students to make sense of the vision literature

ECE 7259 Advanced Computer Vision

20

Course Topics will try to cover

Filtering and Smoothing Resolution and Scale Space Eigen Analysis and Blob Fitting Feature Extraction Color and Light / Skin Color Segmentation /Active Shape Models Camera Models / Planar Mappings Mosaicing and Stabilization Stereo Vision Camera Motion Video Change Detection Video Tracking

ECE 7259 Advanced Computer Vision

21

Filtering and Smoothing

Linear operators Convolution Smoothing

ECE 7259 Advanced Computer Vision

22

Sharpening
An image can be sharpened

ECE 7259 Advanced Computer Vision

23

Masked Blur
Using masks you can specify which part of an image to operate on. Eg blurring the background (using a simple linear filter)

ECE 7259 Advanced Computer Vision

24

Gradient of an Image
This gives the 'embossed' effect in Photoshop

ECE 7259 Advanced Computer Vision

25

Resolution Pyramids / Scale Space

Image Resolution Sampling Theory Gaussian/Laplacian Pyramids Scale Space Theory

ECE 7259 Advanced Computer Vision

26

Eigen Analysis and Blob Fitting

Scatter Matrix Eigenvectors/Eigenvalues Ellipses/Quadratic Forms Center of Mass, Major/Minor Axes

ECE 7259 Advanced Computer Vision

27

Edge Detection
Detecting edge features in images.
This example uses Canny's method

ECE 7259 Advanced Computer Vision

28

Feature Extraction
Edges

Corner

Invariant Features

ECE 7259 Advanced Computer Vision

29

An Image in Frequency Domain

The Log-magnitude and phase of an image in the frequency domain.

ECE 7259 Advanced Computer Vision

30

Color and Light

Radiance / Reflection Illumination / Shading Chromaticity Color Constancy

ECE 7259 Advanced Computer Vision

31

Skin Color

Human Skin Color Distribution Finding Skin Identifying Adult Content

ECE 7259 Advanced Computer Vision

32

Segmentation

Region Segmentation Active Contours Intelligent Scissors

ECE 7259 Advanced Computer Vision

33

Colour Segmentation
An example of skin detection using a skin chrominance model

ECE 7259 Advanced Computer Vision

34

Active Shape Models

Statistical Shape Analysis Active Shape Models Active Appearance Models

ECE 7259 Advanced Computer Vision

35

Removing Periodic Noise

Periodic noise can be easily identified and attenuated in the frequency domain.

ECE 7259 Advanced Computer Vision

36

Reconstruction of 3D structure
Projective geometry is used to reconstruct the 3D structure of a scene from 2 or more views.
camera models, epipolar geometry

Fundamental matrix
3D reconstruction of a scene texture morphing

ECE 7259 Advanced Computer Vision

37

Pin hole camera

ECE 7259 Advanced Computer Vision

38

Pinhole Camera Model

We'll look at this in 3D,

ECE 7259 Advanced Computer Vision

39

Camera Projection Models

Projection Models Intrinsic (lens) Paramters Extrinsic (pose) Parameters Camera Calibration

ECE 7259 Advanced Computer Vision

40

Plane to Plane Mappings

Rigid, Similarity, Affine, & Projective Mappings Homography Estimation Image Warping

ECE 7259 Advanced Computer Vision

41

Mosaicing and Stabilization

Camera Rotation Homography Mosaicing Video Stabilization

ECE 7259 Advanced Computer Vision

42

Stereo Vision

Stereo Camera Setups Stereo Disparity / Parallax Epipolar Geometry Correspondence Matching Triangulation / Depth Recovery

ECE 7259 Advanced Computer Vision

43

Stereo Cameras
and consider stereo cameras.

ECE 7259 Advanced Computer Vision

44

3D reconstructions done from multiple views

Apply this to reconstructing an object from 2 views, using point correspondences.

ECE 7259 Advanced Computer Vision

45

and generate a 3D model.

ECE 7259 Advanced Computer Vision

46

Texture may be added to the model.

ECE 7259 Advanced Computer Vision

47

These four views allow this object to be fully reconstructed.

ECE 7259 Advanced Computer Vision

48

Building Rome in a Day: Colosseum

ECE 7259 Advanced Computer Vision

49

ECE 7259 Advanced Computer Vision

50

San Marco Square

ECE 7259 Advanced Computer Vision

51

ECE 7259 Advanced Computer Vision

52

3D reconstruction from Single Image

ECE 7259 Advanced Computer Vision

53

Image into 3D model

ECE 7259 Advanced Computer Vision

54

Camera Motion

Motion Field vs Optic Flow Flow Estimation Egomotion Estimation Structure from Motion

ECE 7259 Advanced Computer Vision

55

Video Change Detection

Video Sequences Background Modeling Change Detection

ECE 7259 Advanced Computer Vision

56

Video Tracking

Appearance-Based Tracking Sample Tracking Algorithms (e.g. Mean-Shift, Lucas-Kanade)

SOME VISION APPLICATIONS

ECE 7259 Advanced Computer Vision

58

Automated Video Surveillance

ECE 7259 Advanced Computer Vision

59

Face Detection

ECE 7259 Advanced Computer Vision

60

Face Recognition

ECE 7259 Advanced Computer Vision

61

3D Model Generation

ECE 7259 Advanced Computer Vision

62

Articulated Body Fitting

ECE 7259 Advanced Computer Vision

63

Sample Application: Assisted Living

ECE 7259 Advanced Computer Vision

64

Camera Servoing

ECE 7259 Advanced Computer Vision

65

Camera Servoing Application

ECE 7259 Advanced Computer Vision

66

Acquisition of Multi-View Video

Approach: control pan, tilt and zoom of multiple cameras to acquire centered, synchronized video views of a moving person.

ECE 7259 Advanced Computer Vision

67

Eyevision: Adding Master-Slave Control

CAMERA CALIBRATION!

ECE 7259 Advanced Computer Vision

68

Eyevision : SuperBowl XXXV

ECE 7259 Advanced Computer Vision

69

EyeVision Examples

ECE 7259 Advanced Computer Vision

70

Appearance-Based Tracking Framework

ECE 7259 Advanced Computer Vision

71

Vehicle Tracking

ECE 7259 Advanced Computer Vision

72

Robot Perception

Autonomous Navigation Obstacle Avoidance

ECE 7259 Advanced Computer Vision

73

Steering Angle Estimation

ECE 7259 Advanced Computer Vision

74

ECE 7259 Advanced Computer Vision

75

Detecting Other Moving Objects

ECE 7259 Advanced Computer Vision

76

Augmented Reality

ECE 7259 Advanced Computer Vision

77

Augmented Reality