Chapter 4

Color in Image and Video

Color in Image and Video — Basics of Color

Light and Spectra

In 1672, Isaac Newton discovered that white light could be split into many colors by a
prism. The colors produced by light passing through prism are arranged in precise array
or spectrum. The colors’ spectral signature is identified by its wavelength.

Invisible light Visible Light Invisible light

700nm 400nm

Microwave Red Yellow Violet X-ray

Radio wave Orange Green Blue
Infrared Indigo Ultra-violet

Visible light is an electromagnetic wave in the 400nm – 700 nm range (Blue~400nm,

Red~700nm, Green~500nm). Most light we see is not one wavelength, it’s a combination
of many wavelengths. For Example purple is a mixture of red and violet. 1nm=10-9m

Fig Light wavelengths

The profile above is called a spectrum.

1
The Color of Objects
Here we consider the color of an object illuminated by white light. Color is produced by
the absorption of selected wavelengths of light by an object. Objects can be thought of as
absorbing all colors except the colors of their appearance which are reflected back. A
blue object illuminated by white light absorbs most of the wavelengths except those
corresponding to blue light. These blue wavelengths are reflected by the object.

Fig White light composed of all wavelengths of visible light incident on a pure blue
object. Only blue light is reflected from the surface.

The Eye and Color Sensation

Our perception of color arises from the composition of light - the energy spectrum of
photons - which enter the eye. The retina on the inner surface of the back of the eye
contains photosensitive cells. These cells contain pigments which absorb visible light.
Two types of photosensitive cells
 Cones
 Rods
Rods: are not sensitive to color. They are sensitive only to intensity of light. They are
effective in dim light and sense differences in light intensity - the flux of incident
photons. Because rods are not sensitive to color, in dim light we perceive colored objects
as shades of grey, not shades of color.

Cones: allow us to distinguish between different colors. Three types of cones:

 Red cones: responds to red light
 Green cones : respond to green light
 Blue cones: responds to blue light

2
 Fig cross-sectional representation of the eye showing light entering through the pupil

The color signal to the brain comes from the response of the three cones to the spectra
being observed. That is, the signal consists of 3 numbers:
o Red
o Green
o Blue
 A color can be specified as the sum of three colors. So colors form a 3 dimensional
vector space.

For every color signal or photons reaching the eye, some ratio of response within the
three types of cones is triggered. It is this ratio that permits the perception of a particular
color.

 The following figure shows the spectral-response functions of the cones and the
luminous-efficiency function of the human eye.
 Eye responds differently to changes in different color and luminance.

Fig Cones and luminous-efficiency function of the human eye

Color Spaces
Color space specifies how color information is represented. It is also called color model.

3
Any color could be described in a three dimensional graph, called a color space.
Mathematically the axis can be tilted or moved in different directions to change the way
the space is described, without changing the actual colors. The values along an axis can
be linear or non-linear. This gives a variety of ways to describe colors that have an
impact on the way we process a color image.

There are different ways of representing color. Some of these are:

 RGB color space
 YUV color space
 YIQ color space
 CMY/CMYK color space
 CIE color space
 HSV color space
 HSL color space
 YCbCr color space

RGB Color Space

RGB stands for Red, Green, Blue. RGB color space expresses/defines color as a mixture
of three primary colors:
 Red
 Green
 Blue
All other colors are produced by varying the intensity of these three primaries and mixing
the colors. It is used self-luminous devices such as TV, monitor, camera, and scanner.

Fig RGB color model

Color images can be described with three components, commonly Red, Green, and Blue.
It combines (adds) the three components with varying intensity to make all other colors.
Absence of all colors (zero values for all the components) create black. The presence of
the three colors form white. These colors are called additive colors since they add
together the way light adds to make colors, and is a natural color space to use with video
displays.

4
Grey is any value where R=G=B, thus it requires all three (RGB) signals to produce a
"black and white" picture. In other words, a "black and white" picture must be computed
- it is not inherently available as one of the components specified.

Pure black (0,0,0)

Pure white(255,255,255)

CRT Displays
 CRT displays have three phosphors (RGB) which produce a combination of
wavelengths when excited with electrons.
 The gamut of colors is all colors that can be reproduced using the three
Primaries.
 The gamut of a color monitor is smaller than that of color models, E.g. CIE
(LAB) Model

Fig CRT display

CYM and CYMK

A color model used with printers and other peripherals. Three primary colors, cyan (C),
magenta (M), and yellow (Y), are used to reproduce all colors.

Fig CMY color space

The three colors together absorb all the light that strikes it, appearing black (as contrasted
to RGB where the three colors together made white). "Nothing" on the paper is white (as

5
contrasted to RGB where nothing was black). These are called the subtractive or "paint"
colors.

In practice, it is difficult to have the exact mix of the three colors to perfectly absorb all
light and thus produce a black color. Expensive inks are required to produce the exact
color, and the paper must absorb each color in exactly the same way. To avoid these
problems, a forth color is often added - black - creating the CYMK color "space", even
though the black is mathematically not required.

YCbCr
This color space is closely related to the YUV space, but with the coordinates shifted to
allow all positive valued coefficients. It is a scaled and shifted YUV.
 The luminance (brightness), Y, is retained separately from the chrominance (color).

Y-Luma component
Cb
Cr Chrominace

During development and testing of JPEG it became apparent that chrominance sub
sampling in this space allowed a much better compression than simply compressing RGB
or CYM. Sub sampling means that only one half or one quarter as much detail is retained
for the color as for the brightness.

 It is used in MPEG and JPEG compressions

Y = + 0.299 * R + 0.587 * G + 0.114 * B

Cb = 128 - 0.168736 * R - 0.331264 * G + 0.5 * B
Cr = 128 + 0.5 * R - 0.418688 * G - 0.081312 * B

CIE
In 1931, the CIE (Commite Internationale de E’clairage) developed a color model based
on human perception. They are based on the human eyes’ response to red green and blue
colors, and are designed to accurately represent human color perception. The CIE is a
device-independent color model and because of this it is used as a standard for other
colors to compare with. Device-independent means color can be reproduced faithfully on
any type of device, such as scanners, monitors, and printers (color quality does not vary
depending on the device).

There are different versions of CIE color model. The most commonly used are:
 CIE XYZ color model
 CIE L*a*b color model

6
Fig CIE color model

CIE XYZ
CIE XYZ color model defines three primaries called X, Y, and Z that can be combined to
match any color humans see. This relates to color perception of human eye. The Y
primary is defined to match the luminous efficiency of human eye. X and Z are obtained
based on experiment involving human observers.

Fig CIE XYZ chromacity diagram

7
 Edges represent pure colors
 Every color could be assigned a particular point on the coordinate plane
 The spectral purity of colors decreases as you move from the edges to the center of
the diagram
 Brightness is not taken into consideration in this model

CIE Lab Color Model

 A refined CIE model, named CIE L*a*b is introduced in 1976

 It is an improvement of CIE XYZ color model
L– represents Luminance
a – ranges from green to red
b – ranges from blue to yellow represents chrominance
 Used by Photoshop

Fig LAB model

The color space choice

In August 1991, the International Group 4 color fax Committee decided to assume
YCbCr would be the standard as they continued their studies. They noted that YCbCr
was mandatory for compatibility with business image systems such as desktop
publishing. For professional graphics, it was mandatory along with CIELAB for
calibration. At the high end of publishing, many color spaces had to be supported,
including YCbCr. In fact, YCbCr was the most widely used color space in all areas.

By the November 1992 Group 4 color fax meeting in Tokyo, CIELAB 1976 was selected
as the primary color space, with YCbCr as one of several secondary options. Some of the
people involved argue that the particular meeting was dominated by people with special
interests, and don't believe that decision will stand.

If CIELAB becomes the fax standard, it would logically be our choice. However, YCbCr
is much more widely used, and preferred by many technical experts.

8
Beside the RGB representation, YIQ and YUV are the two commonly used in video.

YIQ Color Model

YIQ is used in color TV broadcasting, it is downward compatible with Black and White
TV. The YIQ color space is commonly used in North American television systems. Note
that if the chrominance is ignored, the result is a "black and white" picture.

 Y (luminance) is the CIE Y primary

Y = 0.299R + 0.587G + 0.114B
 the other two vectors
I = 0.596R - 0.275G - 0.321B
Q = 0.212R - 0.528G + 0.311B
 I is red-orange axis, Q is roughly orthogonal to I.
 Eye is most sensitive to Y (luminance), next to I, next to Q.

YIQ is intended to take advantage of human color response characteristics. Eye is more
sensitive to Orange-Blue range (I) than in Purple-Green range (Q). Therefore less
bandwidth is required for Q than for I. NTSC limits I to 1.5 MHZ and Q to 0.6 MHZ. Y
is assigned higher bandwidth, 4MHZ.

YUV Color Model

 Established in 1982 to build digital video standard
 Works in PAL (50 fields/sec) or NTSC (60 fields/sec)
 The luminance (brightness), Y, is retained separately from the chrominance (color)

The Y component determines the brightness of the color (referred to as luminance or

luma), while the U and V components determine the color itself (it is called chroma). U is
the axis from blue to yellow and V is the axis from magenta to cyan. Y ranges from 0 to
1 (or 0 to 255 in digital formats), while U and V range from -0.5 to 0.5 (or -128 to 127 in
signed digital form, or 0 to 255 in unsigned form).

9
Fig YUV color model

One neat aspect of YUV is that you can throw out the U and V components and get a
grey-scale image. Black and white TV receives only Y (luminanace) component ignoring
the otheres. This makes it black-white TV compatible. Since the human eye is more
responsive to brightness than it is to color, many lossy image compression formats throw
away half or more of the samples in the chroma channels (color part) to reduce the
amount of data to deal with, without severely destroying the image quality.

This image shows a slightly tilted representation of the YUV color cube, looking at the
dark (Y = 0) side. Notice how in the middle it is completely black, which is where U and
V are zero, and Y is as well. As U and V move towards their limits, you start to see their

10
effect on the colors.

This image shows the same cube, from the bright side (Y = 1). Here we have bright white
in the middle of the face, with very bright colors on the corners where U and V are also at
their limits.

Y = R * 0.299 + G * 0.587 + B * 0.114

U = R * -0.169 + G * -0.332 + B * 0.500
V = R * 0 .500 + G * -0.419 + B * -0.0813

 The YUV color space is commonly used in European television.

 if the chrominance is ignored, the result is a "black and white" picture.

The CMY Color Model

 Cyan, Magenta, and Yellow (CMY) are complementary colors of RGB. They can be
used as Subtractive Primaries.
 CMY model is mostly used in printing devices where the color pigments on
the paper absorb certain colors (e.g., no red light reflected from cyan ink) and in
painting.

Fig RGB and CMY cubes

11
Cyan, magenta, and yellow are used as subtractive primaries

Conversion between RGB and CMY

E.g., convert White from (1, 1, 1) in RGB to (0, 0, 0) in CMY
C=1-R
M=1-G
Y=1-B

CMYK color model

 Sometimes, an alternative CMYK model (K stands for Black) is used in color printing
(e.g., to produce darker black than simply mixing CMY), where
K = min(C, M, Y),
C = C - K,
M = M - K,
Y = Y - K.

Colors on self-luminous devices, such as televisions and computer monitors, are

produced by adding the three RGB primary colors in different proportions. However,
color reproduction media, such as printed matter and paintings, produce colors by
absorbing some wavelengths and reflecting others.

The three RGB primary colors, when mixed, produce white, but the three CMY primary
colors produce black when they are mixed together. Since actual inks will not produce
pure colors, black (K) is included as a separate color, and the model is called CMYK.
With the CMYK model, the range of reproducible colors is narrower than with RGB, so
when RGB data is converted to CMYK data, the colors seem dirtier.

HSL Color Space

HSL stands for Hue Saturation Lightness.
H-represents hue (color)
S-represents saturation. It goes from fully saturated color to equivalent gray.

12
Fig HSL color space

The HSL color space stands for Hue, Saturation, Lightness (also luminance or
luminosity). HSL is drawn as a double cone or double hexcone. The two apexes of the
HSL double hexcone correspond to black and white. The angular parameter corresponds
to hue, distance from the axis corresponds to saturation, and distance along the black-
white axis corresponds to lightness.

HSV Color Space (also called HSB)

Stands for Hue Saturation Value.
H-represents color type (red, blue, yellow). It ranges from 0-360 degrees.
Saturation-the vibrancy of color. It ranges from 0-100%.
Value-brightness of color. It ranges from 0-100%.

Fig HSV color model

13
Summary of Color
 Color images are encoded as (R,G,B) integer triplet values. These triplets encode how
much the corresponding phosphor should be excited in devices such as a monitor.
 Three common systems of encoding in video are RGB, YIQ, and YcrCb(YUV).
 Besides the hardware-oriented color models (i.e., RGB, CMY, YIQ, YUV), HSB
(Hue, Saturation, and Brightness, e.g., used in Photoshop) and HLS (Hue, Lightness,
and Saturation) are also commonly used.
 YIQ uses properties of the human eye to prioritize information. Y is the black and
white (luminance) image; I and Q are the color (chrominance) images. YUV uses
similar idea.
 YUV is a standard for digital video that specifies image size, and decimates the
chrominance images (for 4:2:2 video)
 A black and white image is a 2-D array of integers.

14