2/22/24

Computer Vision
Course
School of Informations and
Communication Technology

Computer Vision
Chapter 2: Image formation, acquisition and
digitization
Computer Vision Group
School of Information and Communications Technology

1
 2/22/24

Content

• Image formation
• Human vision
• Image formation
• Acquisition and digitization: Digital camera
• Imaging sensor
• 2D signal and sampling
• Color:
• Primary color, additive/ subtractive color, color spaces
• Digital image representation and formats

CV Group – School of Information and Communication and Technology 3

Image formation

Image formation studies the forward process of producing images and

videos.
• Image formation encompasses the radiometric and geometric processes
by which 2D images of 3D objects are formed. To produce a real image,
the nature of the visual sensors (i.e. CCD and CMOS cameras), should be
studied.
• Imaging process is a mapping of an object to an image plane.
• With digital images, the image formation process also includes analog to
digital conversion, sampling
• Human color vision (Perception) : In the case of computer vision the light
incident on the sensor comprises the image. In the case of visual
perception, the human eye has a color dependent response to light which
is the spectral sensitivity of human vision.

CV Group – School of Information and Communication and Technology 4

2
 2/22/24

The Eye

• The human eye is a camera

• Iris - colored annulus with radial muscles
• Pupil - the hole (aperture) whose size is controlled by the iris
• What’s the sensor?

Slide by Steve Seitz

– photoreceptor cells (rods and cones) in the retina
CV Group – School of Information and Communication and Technology 5

Two types of light-sensitive receptors

Cones
cone-shaped
less sensitive
operate in high light
color vision
Rods
rod-shaped
highly sensitive
operate at night
gray-scale vision

James Hays

© Stephen E. Palmer, 2002

CV Group – School of Information and Communication and Technology 6

3
 2/22/24

Rod / Cone sensitivity

CV Group – School of Information and Communication and Technology 7

Physiology of Color Vision

Three kinds of cones:

440 530 560 nm.
RELATIVE ABSORBANCE (%)

100
S M L

50

400 450 500 550 600 650

WAVELENGTH (nm.)

© Stephen E. Palmer, 2002

CV Group – School of Information and Communication and Technology 8

© Stephen E. Palmer, 2002

4
 2/22/24

Tetrachromatism

• Most birds, and many other animals, have cones for

ultraviolet light.
• Some humans seem to have four cones (12% of females).

Bird cone
responses

James Hays

CV Group – School of Information and Communication and Technology 9

Image formation

Adapted from S. Seitz

CV Group – School of Information and Communication and Technology 10

10

5
 2/22/24

What is light?

• Light: The visible portion of the electromagnetic (EM) spectrum.

• Light occurs between wavelengths of approximately 400 and 700
nanometers.

CV Group – School of Information and Communication and Technology 11

11

Photometric image formation

• Illumination source: Sun, light …

• Photometric measurement:
• Perceptual brightness of visible electromagnetic energy of light.
• Optical system (lenses):
• An object (scene) may be illuminated by the light from an emitting
source.
• The light incident on the object is reflected in a manner dependent on
the surface properties of the object
• An illuminated object will scatter light toward a lens and the lens will
collect and focus the light to create the image
• Imaging sensor: CCD (charge-coupled device) or CMOS sensors cameras
provide the 2D sensed signal.
• Digital camera: 2D sensed signal is pass to analog-to-digital converter
(sampling), it create the digital image

CV Group – School of Information and Communication and Technology 12

12

6
 2/22/24

Photometric image formation

• Modeling the image formation process: 3D geometric features in the

world are projected into 2D features in an image.
• A simplified model of photometric image formation is illustrated.

o The scene is
illuminated by a
single source.
o The scene reflects
radiation towards
the camera.
o The camera senses
it via CCD/ CMOS

CV Group – School of Information and Communication and Technology 13

13

Acquisition and digitization: Digital camera

Digital camera: Image sensing and processing pipeline

Adapted from S. Seitz

CV Group – School of Information and Communication and Technology 14

14

7
 2/22/24

Digital camera

• Image acquisition:
• Optical system, aperture (capture), shutter
• Imaging sensor: CCD/ CMOS sensor camera consists of a
array of photodiodes. Each cell in the is light-sensitive diode
that converts photons to electrons.
• 2D sensed signal of image, video
• Digitization (ADC): Sampling and Quantization
• Sampling the 2D sensed signal create the samples or pixels
• Quantizing the sample values as the integer values of pixels
• Processing (DSP- Digital Signal Processing):
• Cameras perform a variety of digital signal processing
operations to enhance the image before compressing and storing
the pixel values in standard format file.

CV Group – School of Information and Communication and Technology 15

15

Sensor array : an example

CV Group – School of Information and Communication and Technology 16

16

8
 2/22/24

Real scene -> digital Image

Sensor array Electric Analog/Digital

conversion Number
Scene + charge
Light (discrete
(Continuous
signal) signal)

Digitization = Sampling + Quantization

CV Group – School of Information and Communication and Technology 17

17

Sampling and quantization

¡ Sample the 2D space on a regular grid

¡ Quantize each sample (round to nearest integer)

CV Group – School of Information and Communication and Technology 18

18

9
 2/22/24

Sampling and quantization

¡ Sample the 2D space on a regular grid

¡ Quantize each sample (round to nearest integer)

2D

CV Group – School of Information and Communication and Technology 19

19

Digital image

CV Group – School of Information and Communication and Technology 20

20

10
 2/22/24

Spatial resolution (sampling)

CV Group – School of Information and Communication and Technology 21

21

Gray-level resolution (Quantization)

CV Group – School of Information and Communication and Technology 22

22

11
 2/22/24

Color spaces
• Color spaces; different types of color modes
• Color represented by vector of components
v Red, Green, Blue (RGB)
v Hue, Saturation, Value (HSV)
v Luminance, chrominance (YUV, LUV)
v XYZ
• Color convert: RGB – YUV
Y = 0.299R + 0.587G + 0.114B
U = 0.493 (B - Y) ; V = 0.877 (R - Y)

é Y ù é 0.257 0.504 0.098 ù é R ù é 16 ù

êC ú = ê 0.439 - 0.368 - 0.071ú êG ú + ê128ú
ê Rú ê úê ú ê ú
êëC B úû êë- 0.148 - 0.291 0.439 úû êë B úû êë128úû

CV Group – School of Information and Communication and Technology 23

23

Color coordinate system

M µu s½c
Blue RGB
Magenta Black (0, 0, 0)
Red (255, 0, 0)
white Green (0, 255, 0)
Cyan Yellow (255, 255, 0)
Blue (0, 0, 255)
Red Magenta (255, 0, 255)
Cyan (0, 255, 255)
White (255, 255, 255)

Green Yellow YUV (Luminance)

Y = 0.299R + 0.587G + 0.114B
U=R-Y
Y Cb Cr (JPEG)
V=B-Y
Cb = U/2 + 0.5
Cr = V/1.6 + 0.5

CV Group – School of Information and Communication and Technology 24

24

12
 2/22/24

Color: Additive/Subtractive primary color

• Primary color: Red, Green, Blue (RGB) Colors:

• Additive colors: combination
of RGB
• Combination of RGB can be mixed to produce
Cyan, Magenta, Yellow (CMY) &White.
• Additive color reproduction system:
• Combination of RGB to reproduce a colored light.

• Subtractive colors CMY can be mixed to produce RBG & black

• Subtractive color reproduction system: A white light sequentially
passes through cyan, magenta, yellow filters to reproduce a
colored light.

CV Group – School of Information and Communication and Technology 25

25

Color spaces: RGB

Default color space

0,1,0

R=1
(G=0,B=0)

1,0,0 G=1
(R=0,B=0)

0,0,1

Any color = r*R + g*G + b*B B=1

(R=0,G=0)
• Strongly correlated channels
• Non-perceptual
Image from: http://en.wikipedia.org/wiki/File:RGB_color_solid_cube.png

CV Group – School of Information and Communication and Technology 26

26

13
 2/22/24

Nonlinear color spaces: HSV

• Perceptually meaningful dimensions:

• Hue, Saturation (chroma)
• Value (Intensity)

CV Group – School of Information and Communication and Technology 27

27

HSV (Hue – Saturation- Value)

• The Hue-Saturation-Value (HSV) color space is useful for

segmentation and recognition
• Non-linear conversion
• Visual representation of colors
• We identify for a pixel:
• The pixel intensity (value)
• The pixel color (hue + saturation)
• RGB does not have this separation

CV Group – School of Information and Communication and Technology 28

28

14
 2/22/24

HSV (Hue – Saturation- Value)

• Hue (H) is coded as an angle between 0 and 360

• Saturation (S) is coded as a radius between 0 and 1
• S = 0 : gray
• S = 1 : pure color
• Value (V) = MAX (Red, Green, Blue)

CV Group – School of Information and Communication and Technology 29

29

HSV (Hue – Saturation- Value)

• If we know the color of the object we are looking for, we can

model it using a hue interval
• Take care, because it is an angle (periodic value)
• Hue < 60° means nothing
• Is 350° smaller or bigger than 60°?
• Define an interval: 350° < Hue < 60° (for example)
• This interval is valid if Saturation > threshold (otherwise gray
level)
• This is independant of Value , which is more sensible to light
conditions

CV Group – School of Information and Communication and Technology 30

30

15
 2/22/24

Lab color space

• The Lab system (sometimes L*a*b*) is based on a study from

human vision
• independant from all technologies
• presenting colors as seen by the human eyes
• Colors are defined using 3 values
• L is the luminance, going from 0% (black) to 100% (white)
• a* represents an axis going from green (negative value, -
127) to red (positive value, +127)
• b* represents an axis going from blue (negative value, -
127) to yellow (positive value,+127)

CV Group – School of Information and Communication and Technology 31

31

Lab color space

CV Group – School of Information and Communication and Technology 32

32

16
 2/22/24

Color space vs. illumination conditions

• collected 10 images of the cube under varying illumination

conditions

• separately cropped every color to get 6 datasets for the 6 different

colors

Changes in color due to varying Illumination conditions

• Compute the density plot: Check the distribution of a particular

color say, blue or yellow in different color spaces. The density plot
or the 2D Histogram gives an idea about the variations in values for
a given color
Source: Vikas Gupta, Learn OpenCV
CV Group – School of Information and Communication and Technology 33

33

Color space vs. illumination conditions

• Similar illumination: very compact

Fig.: Density Plot showing the variation of values in color

channels for 2 similar bright images of blue color
Source: Vikas Gupta, Learn OpenCV
CV Group – School of Information and Communication and Technology 34

34

17
 2/22/24

Color space vs. illumination conditions

• Similar illumination: very compact

Fig.: Density Plot showing the variation of values in color channels for
2 similar bright images of yellow color
Source: Vikas Gupta, Learn OpenCV
CV Group – School of Information and Communication and Technology 35

35

Color space vs. illumination conditions

• Different illumination:

Fig.: Density Plot showing the variation of values in color channels

under varying illumination for the blue color Source: Vikas Gupta, Learn OpenCV
CV Group – School of Information and Communication and Technology 36

36

18
 2/22/24

Color space vs. illumination conditions

• Different illumination:

Fig.: Density Plot showing the variation of values in color channels

under varying illumination for the yellow color
Source: Vikas Gupta, Learn OpenCV
CV Group – School of Information and Communication and Technology 37

37

Color space vs illumination conditions

• Different illumination:
• RGB space: the variation in the value of channels is very high
• HSV: compact in H. Only H contains information about the
absolute color è a choix
• YCrCb, LAB: compact in CrCb and in AB
• Higher level of compactness is in LAB
• Convert to other color spaces (OpenCV):
• cvtColor(bgr, ycb, COLOR_BGR2YCrCb);
• cvtColor(bgr, hsv, COLOR_BGR2HSV);
• cvtColor(bgr, lab, COLOR_BGR2Lab);

CV Group – School of Information and Communication and Technology 38

38

19
 2/22/24

Image representation
Continuous Images as functions

• Monochromatic Image: A continuous brightness function of a

number of variables f, from R2 to R:
of(x, y) gives the intensity at position (x, y)
o Realistically, we expect the image only to be defined over
a rectangle, with a finite range
• A color image include 3 brightness functions of 3 color pasted
together (3 color component signals). We can write this as a
“vector-valued” function:
é r ( x, y ) ù
f ( x, y ) = ê g ( x, y ) ú
ê ú
êë b( x, y ) úû

CV Group – School of Information and Communication and Technology 39

39

Digital images representation

• Sample the 2D space on a regular grid is pixel

• Quantize each sample (round to nearest integer)
• Image data is represented as a matrix of integer values.

2D

Analog signal Digital signal: Data

Adapted from S. Seitz

CV Group – School of Information and Communication and Technology 40

40

20
 2/22/24

Definition: Digital images

• Digital image functions f represented as matrices X(i,j).

• Image data is represented by a rectangular array of integers
• An integer represents the brightness or darkness of the
monochromatic image at that point (pixel). Limited brightness
integer values (8 bit) = gray levels = values 0 to 255
• Definition: Digital image is a matrix X(i,j) of pixels, N:number of
rows, M: number of columns, Q: integer brightness values (levels)
of pixels
f (0,0) f (0,1) ... f (0, M - 1)
X(i, j) = f (1,0) f (1,1) ... f (1, M - 1)
... ... ... ...
f ( N - 1,0) f ( N - 1,1) ... f ( N - 1, M - 1)

CV Group – School of Information and Communication and Technology 41

41

Digital gray image

• Example: Matrix X(i,j) of Columns

pixels of a gray level
image
• Image data: 2D array
Rows

X(i,j) of integer brightness

value uint8 of pixels at
coordinates (i,j).
i-1, j i-1, j-1 i-1, j i-1, j+1

i, j-1 i, j i, j+1 i, j-1 i, j i, j+1

i+1, j i+1,j-1 i+1, j i+1,j+1

CV Group – School of Information and Communication and Technology 42

42

21
 2/22/24

RGB color images in Matlab

• Images represented as a matrix X(i,j)

• Suppose we have a NxM RGB image called “Im”
• Im(1,1,1) = top-left pixel value in R-channel
• Im(y, x, b) = y pixels down, x pixels to right in the bth channel
• Im(N, M, 3) = bottom-right pixel in B-channel
• imread(filename) returns a uint8 image (values 0 to 255)
• Convert to double format (values 0 to 1) with im2double
column
row 0.92 0.93 0.94 0.97 0.62 0.37 0.85 0.97 0.93 0.92 0.99
R
0.95 0.89 0.82 0.89 0.56 0.31 0.75 0.92 0.81 0.95 0.91
0.89 0.72 0.51
0.92
0.55
0.93
0.51 0.42 0.57 0.41 0.49 0.91 0.92
0.94 0.97 0.62 0.37 0.85 0.97 0.93 0.92 0.99
G
0.96 0.95 0.88
0.95 0.94
0.89 0.56
0.82 0.46
0.89 0.91
0.56 0.87
0.31 0.90
0.75 0.97
0.92 0.95
0.81 0.95 0.91
0.71 0.81 0.81
0.89 0.87
0.72 0.57
0.51 0.37
0.55 0.80
0.51 0.88
0.42 0.89
0.57 0.79
0.41 0.85
0.49 0.91 0.92
B
0.49 0.62 0.60 0.58 0.92
0.50 0.93
0.60 0.94
0.58 0.97
0.50 0.62
0.61 0.37
0.45 0.85
0.33 0.97 0.93 0.92 0.99
0.96 0.95 0.88
0.95 0.94 0.56 0.46 0.91 0.87 0.90 0.97 0.95
0.86 0.84 0.74 0.58 0.51 0.89
0.39 0.82
0.73 0.89
0.92 0.56
0.91 0.31
0.49 0.75
0.74 0.92 0.81 0.95 0.91
0.71 0.81 0.81 0.72
0.89 0.87 0.51
0.57 0.55
0.37 0.51
0.80 0.42
0.88 0.57
0.89 0.41
0.79 0.49
0.85 0.91 0.92
0.96 0.67 0.54
0.49 0.85
0.62 0.48
0.60 0.37
0.58 0.88
0.50 0.90
0.60 0.94
0.58 0.82
0.50 0.93
0.61 0.45 0.33
0.69 0.49 0.56 0.66 0.96
0.43 0.95
0.42 0.88
0.77 0.94
0.73 0.56
0.71 0.46
0.90 0.91
0.99 0.87 0.90 0.97 0.95
0.86 0.84 0.74 0.58 0.51 0.39 0.73 0.92 0.91 0.49 0.74
0.79 0.73 0.90 0.67 0.71
0.33 0.81
0.61 0.81
0.69 0.87
0.79 0.57
0.73 0.37
0.93 0.80
0.97 0.88 0.89 0.79 0.85
0.96 0.67 0.54 0.85 0.60
0.49 0.48 0.37 0.50
0.88 0.90 0.58
0.94 0.82 0.93
0.91 0.94 0.89 0.49 0.41 0.62
0.78 0.78 0.58
0.77 0.89 0.60
0.99 0.93 0.50 0.61 0.45 0.33
0.69 0.49 0.56 0.66 0.43 0.42 0.77 0.73
0.86 0.84 0.74 0.58 0.51 0.39 0.73 0.71 0.92
0.90 0.91
0.99 0.49 0.74
0.79 0.73 0.90 0.67 0.33 0.61 0.69 0.79 0.73 0.93 0.97
0.96 0.67 0.54 0.85 0.48 0.37 0.88 0.90 0.94 0.82 0.93
0.91 0.94 0.89 0.49 0.41 0.78 0.78 0.77 0.89 0.99 0.93
0.69 0.49 0.56 0.66 0.43 0.42 0.77 0.73 0.71 0.90 0.99
0.79 0.73 0.90 0.67 0.33 0.61 0.69 0.79 0.73 0.93 0.97
0.91
CV Group 0.94 of0.89
– School 0.49 and
Information 0.41 Communication
0.78 0.78 0.77 0.89 0.99 0.93
and Technology 43
Slide credit: Derek Hoiem

43

Digital image format

v Parameters for digital image formats:

• Digital image resolution: (height x width) in pixels
• Quantization (bits per pixel):
Gray level image: 8 bits/ pixel
RGB color image: 24 bits/ pixel
Binary image: 1 bit/ pixel
v Digital Image Storage: file stored in two parts: Header; Data
v Common image file formats:
• GIF (Graphic Interchange Format) -
• PNG (Portable Network Graphics)
• JPEG (Joint Photographic Experts Group)
• TIFF (Tagged Image File Format)
• PGM (Portable Gray Map)
• FITS (Flexible Image Transport System)

CV Group – School of Information and Communication and Technology 44

44

22
 2/22/24

Digital video format

• Parameters for digital video formats
• Digital image resolution (height x width) in pixels
• Quantization (bits per pixel)
• Frame rate (frames per second)
• Standard video file formats
• AVI, M-JPEG,
• H26X (ITU_T:H.261, H.263, H.263, H264)
• MPEG-1, MPEG-2, MPEG-4 Part 10 / H264 AVC,mp4…

CV Group – School of Information and Communication and Technology 45

45

THANK YOU !

CV Group – School of Information and Communication and Technology

46
46

23