Colour Image Processing

Allan Hanbury

PRIP, Vienna University of Technology

Favoritenstraße 9/1832
A-1040 Vienna, Austria
[email protected]
http://www.prip.tuwien.ac.at/~hanbury
 Contents

• Introduction
• Processing Vectorial Images
• Alternative Colour Spaces
• 3D-polar Coordinate Colour Spaces
• Processing and Analysing Colour Images
 Contents

• Introduction
• Processing Vectorial Images
• Alternative Colour Spaces
• 3D-polar Coordinate Colour Spaces
• Processing and Analysing Colour Images
 Physical Background

• Visible light: a narrow band of

electromagnetic radiation →
380nm (blue)  780nm (red)
• Wavelength: Each physically
distinct colour corresponds to at
least one wavelength in this band.
• Pure Colours: Pure or
monochromatic colours do not
exist in nature.
• Spectrum: Intensity as
a function of wavelength.

From http://fuse.pha.jhu.edu/~wpb/spectroscopy/basics.html

• The colour of an object: is the product of the

spectrum of the incident light with the light
absorption and/or reflection properties of the object.
 Human colour perception

• The human eye does not perceive individual light

wavelengths.
• It contains three types of colour receptor (cones)
which integrate over parts of the spectrum:

From http://math.ucr.edu/home/baez/physics/General/BlueSky/blue_sky.html
• It is therefore possible to characterise a psycho-visual
colour by specifying the amounts of three primary
colours: red, green and blue, mixed together.
• This leads to the standard RGB space used in
television, computer monitors, etc.
• We specify the levels of R, G and B in the range [0, 1],
but they can easily be extended to other ranges (8-bit
integers for example).
(1,1,1)

RGB

(0,0,0)
 Problems with Processing Colour Images

• When processing colour images, the following

problems (amongst others) have to be dealt with:
– The images are vectorial → 3 numbers are associated with
each pixel.
– The colours recorded by a camera are heavily dependent on
the lighting conditions.
 Lighting conditions

• The lighting conditions of the scene have a large

effect on the colours recorded.

Image taken lit by a flash. Image taken lit by a

tungsten lamp.
• The following four images of the same scene were
acquired under different lighting conditions:
 Dealing with Lighting Changes

• Knowing just the RGB values is not enough to know

everything about the image.
– The R, G and B primaries used by different devices are
usually different.
• For scientific work, the camera and lighting should be
calibrated.
• For multimedia applications, this is more difficult to
organise:
– Algorithms exist for estimating the illumination colour.
 Contents

• Introduction
• Processing Vectorial Images
• Alternative Colour Spaces
• 3D-polar Coordinate Colour Spaces
• Processing and Analysing Colour Images
 Processing Vectorial Images
• A vectorial image has a vector at each pixel. For
colour images, these vectors each have 3 components.
• Vectorial images with larger numbers of components
also exist, e.g. in satellite imagery.
• There are two ways one can process vectorial images:
– Marginal processing.
– Vectorial processing.
Greyscale

Colour

f (x, y) {0,1,…, N} f (x, y) [ {0,…,N}, {0,…,N}, {0,…,N} ]

 Marginal Processing
• Each channel is processed separately:

Red Red
Scalar process.

Green Green
Scalar process.

Blue Blue
Scalar process.
 Vectorial Processing

• The colour triplets are processed as single units:

Red Red

Green Vectorial Green

process.
Blue Blue
The Problem of False Colours

False
Colours !!
Marginal
Median

Vectorial
Median
 The Problem of False Colours

Marginal  R  100 
 
median  V  100 
 
 B  0 
 R  100   R0   R  100 
     
 V 0   V  100   V  100 
     
 B  0   B  0   B  100 
 R0 
 
Vectorial  V  100 
median  
 B0 
 Contents

• Introduction
• Processing Vectorial Images
• Alternative Colour Spaces
• 3D-polar Coordinate Colour Spaces
• Processing and Analysing Colour Images
 Alternative Colour Spaces

• Various other colour representations can be

calculated from the RGB representation.
• This can be done for:
– Decorrelating the colour channels:
• principal components.
– Bringing colour information to the fore:
• Hue, saturation and brightness.
– Perceptual uniformity:
• CIELuv, CIELab, …
 Processing Strategy

Red Red

Processing
Green Green
T T-1
Blue Blue
 Colour spaces
• RGB (CIE), RnGnBn (TV - National Television Standard Comittee)
• XYZ (CIE)
• UVW (UCS de la CIE), U*V*W* (UCS modified by the CIE)
• YUV, YIQ, YCbCr
• YDbDr
• DSH, HSV, HLS, IHS
• Munsel colour space (cylindrical representation)
• CIELuv
• CIELab
• SMPTE-C RGB
• YES (Xerox)
• Kodak Photo CD, YCC, YPbPr, ...
 Contents

• Introduction
• Processing Vectorial Images
• Alternative Colour Spaces
• 3D-polar Coordinate Colour Spaces
• Processing and Analysing Colour Images
 3D-polar Coordinate Colour Spaces
• These spaces use a cylindrical (3D-polar) coordinate system to
encode the following three psycho-visual coordinates:
– Hue (dominant colour seen)
• Wavelength of the pure colour observed in the signal.
• Distinguishes red, yellow, green, etc.
• More the 400 hues can be seen by the human eye.
– Saturation (degree of dilution)
• Inverse of the quantity of “white” present in the signal. A pure colour has
100% saturation, the white and grey have 0% saturation.
• Distinguishes red from pink, marine blue from royal blue, etc.
• About 20 saturation levels are visible per hue.
– Brightness
• Amount of light emitted.
• Distinguishes the greylevels.
• The human eye perceives about 100 levels.
 3D-polar Coordinate Colour Spaces
• The transformation of the RGB colour space to a hue, saturation
and brightness colour space is essentially a conversion from a
set of rectangular coordinates to a set of (3D-polar) cylindrical
coordinates.

• Yet there are many such spaces triangle HSI

described in books. HSV
• How does one choose which one to HLS
IHS
use?
 Idea Behind the Transformation
• In the RGB space, the vectors [R, G, B] specify the amount of each
red, green and blue primary in the colour.
– For convenience, we take R, G, B  [0, 1]
– The valid coordinates form the RGB cube [0, 1]  [0, 1]  [0, 1].
• The basic idea behind the transformation to the hue, saturation and
brightness coordinate system is to place a new axis between [0, 0, 0]
and [1, 1, 1], and to specify the colours in 3D-polar coordinates based
on this axis.
– This new axis passes through all the grey points (R = G = B), so we call it the
achromatic axis.

Achromatic axis
ic
at

Hue
is om

[1,1,1]
ax ch r

H
A

[0,0,0] Saturatio
n Brightness
 Basic Problem Achromatic
axis
• Many of the spaces were originally developed for
easy numerical specification of colours in computer

Conic HSV
graphics applications.
• Due to its brightness function, the “natural” shape
of the HSV colour gamut is a cone.
• However, with this shape, there are many
coordinates which are not valid. H=180° H=0°
• In order to avoid costly verification of the validity
of specified coordinates, these gamuts were often
artificially expanded into cylinders.
• These cylindrical spaces have often been
unwittingly carried over into image processing
applications.

Cylindrical HSV
 Standard RGB → HSV Transform
• max = sup(R, G, B) min = inf(R, G, B)
• L = max

•  max  min
 if max  0
S   max
0 otherwise
•
 maxG Bmin if R  max
 BR
H t   max min  2 if G  max
 R
 maxt  mint
4 if B  max
G
• If Ht

•
 Conic HLS
• The same problem occurs for the
HLS transformation.
• Here the double cone has been H=180° H=0°

artificially expanded at both ends.

Cylindrical HLS
 Standard RGB → HLS Transform
• max = sup(R, G, B) min = inf(R, G, B)
•
L max  min
2
R, G and B are between 0 and 1.

•  max  min
if L 1
max  min 2
Sc  

max  min
2  max  min if L 1
2
•
 GB
max  min if R  max

H  BR
t
• If Ht tmax  2 if
t
 min
G  max
 R G
 4 if B  max
 max  min

•
 Removal of the Brightness
Dependence of the Saturation (1)
• HSV model
 maxR, G, B   min R, G , B 
 if maxR, G, B   0
S HSV  maxR, G, B 
0 otherwise

• But the HSV brightness LHSV = max(R, G, B)

• So the saturation with the brightness-dependence
removed is
NC
S HSV  maxR, G, B   minR, G, B 
 Removal of the Brightness
Dependence of the Saturation (2)
• HLS model
0 if max R, G, B   min R, G, B 
max R, G, B   min R, G, B 
S HLS  if LHLS  12
max R, G, B   min R, G, B 
max R, G, B   min R, G, B 
otherwise
2  max R, G, B   min R, G, B 

• where the brightness

maxR, G, B   minR, G, B 
LHLS 
2
• The brightness dependence is removed by using

S NC
HLS 
 S HLS 1  2 12  LHLS 
• to give

S NC
HLS  maxR, G, B   min R, G, B 

Same as the non-cylindrical saturation obtained for HSV!

• Consider the colour image below. Not all the pixels which appear
white have RGB coordinates of exactly [1, 1, 1] (similar for black
pixels).
• These small variations are amplified in the saturation images due to
the expansion of the cones into cylinders.

“Le chanteur”, Joan Mirò HSV cylindrical HLS cylindrical

(bottom half inverted) saturation saturation
 • The advantage of this measure of saturation is visible on
the example image.

Com
par
e to

Le chanteur, Joan Mirò HSV cylindrical

Proposed Saturation
(bottom half inverted) saturation
 Advantages of the IHLS space

• This is the Improved HLS space.

• It has the following advantages:
– The saturation is low for black and white pixels.
– The brightness is independent of the saturation (can be
shown mathematically). This means that you are free to
choose any brightness, luminance or lightness function.
– The saturation values can be easily compared. This is
important for mathematical morphology operators.
 Transformation into the IHLS space

• The full transformation is implemented efficiently as:

Or your own
Y  0.2126R  0.7152G  0.0722B favourite
brightness
expression.
S  maxR, G, B   minR, G, B 

 R  12 G  12 B 
H   arccos 
 R 2  G 2  B 2  RG  RB  BG 2 
1

Trigonometric
version is more
accurate. 360  H  if B  G
H 
H  otherwise
 Contents

• Introduction
• Processing Vectorial Images
• Alternative Colour Spaces
• 3D-polar Coordinate Colour Spaces
• Processing and Analysing Colour Images
Processing and Analysing Colour Images

• Some applications which take advantage of the good

properties of the IHLS space are presented.
• The following are discussed:
– Hue statistics.
– Mathematical morphology on colour images.
 B
R

G
Change of representation

S
H

Image Processing or Analysis

Basic Idea

L 

S 
H 

Change of representation
B
R

G
 Processing the Hue Component

Colour image – “The virgin”, Hue component

P. Serra (16th century)

The reds and violets are separated by a Histogram of the hue

large discontinuity, even though they
look similar.
 Hue Statistics
• For the brightness and the saturation, one can use standard
statistical methods for calculating the mean, standard deviation,
etc.
• For data ai (i = 1, 2, …, n) distributed on the unit circle, the mean
direction is that of the resultant vector obtained by adding unit
vectors with directions ai.
• A measure of the variation in the directions of the data is given by
the length of this vector divided by n (the mean length), which has
the following characteristics:
– range [0:1]
– Values close to 1  the data is less spread out.

n=3
 Mean
• Given n values of the hue Hi
• The mean direction H is calculated as follows:

n n (1)
A   cos H i , B  sin H i , R  A2  B 2 2

i 1 i 1

arctan BA  if B  0, A(2) 0

H  arctan BA    if A0
arctan B   2 if B  0, A  0
•  R is: A
The mean length

(3)

R
R
n
 Hue Mean which Takes the
Saturation Into Account
• The previous formulation is standard in the texts on circular
statistics, but it ignores the fact that not all hues have the same
importance.
• We take this into account by weighting the length of each hue
vector by the associated saturation value.

n=3
• Let Si be the saturation associated with hue Hi.
• We replace equation (1) by:
n n
AS   Si cos H i , BS  Si sin H i , RS2  AS2  BS2
• To calculate
i 1 the saturation-weightedi mean
1 direction HS, we replace A and B by
AS and BS in equation (2).
• Equation (3) becomes:

RS
R 
S
• Note that RS remains a measure

of the
n angular dispersion, and does not give
S
information on the mean of the saturation.
i 1 i
 Example
• H = 327°
Colour Image

• R = 0.24

Weighted
• HS = 20°
Hue threshold
• R = 0.16
S 307° - 347°
Hue histogram
Hue

Saturation Hue threshold

0° - 40°
Saturation

histogram
 Mathematical Morphology in a
3D-Polar Coordinate Colour Space
• One can easily order the brightness and saturation
1 1
values.
• On the other hand, the hue is defined on the circle, for
which there is no obvious order (blue larger than red?,
green smaller than red? ).
• Applying morphological operators to the brightness and 0 0

saturation is therefore easy. L S

– We just have to be sure that we order the vectors and not the
components (i.e. marginal order), so as to avoid introducing
false colours.
– For this purpose we use the lexicographical order.
• Applying mathematical morphology to the hue is
H 0°

trickier, and won’t be discussed in this talk.

 Lexicographical Order
• This is the order in which words are arranged
in a dictionary.

 
Aardvark
Abacus
Abandon

                                             .
.
.
Borough
Bough
.
.
 Lexicographical Order for Vectors
• For example, given two vectors x = (x1, x2, x3) and y = (y1, y2, y3):
 x1  y1
or


x  y if  x1  y1 and x2  y 2
or


 x1  y1 and x2  y 2 and x3  y3

• The lexicographical order is a total vector order. This means:

– There are no pairs of vectors for which the order is unknown.
– The maximum and minimum of a set of vectors is always part of the set 
no false colours!
• The disadvantage is that one of the vector components has to play
a dominant role.
 Angular Distance
a  a
• To enable us to work on the circle, we
begin by defining an angular distance. a a
• Given a circle C with centre o o
• We choose an arbitrary origin a0 on the circle.
The points ai are then described by their curvilinear
coordinate between 0 and 2 starting at a0.
• Given two points a and a, the size of the acute angle aoa
is
 a  a if a  a  
Da, a  a  a  
2  a  a if a  a  
 Lexicographical Order with
Brightness at the Top Level (1)
• We define
 Li  L j
 or

ci  c j if  Li  L j and Si  S j
 or

 Li  L j and Si  S j and H i  H 0  H j  H 0

• and
 Li  L j
 or

ci  c j if  Li  L j and Si  S j
 or

 Li  L j and Si  S j and H i  H 0  H j  H 0
 Lexicographical Order with
Brightness at the Top Level (2)

• H0 is an arbitrary parameter. It does not have a major effect on

the results, as, being at the third level, it is almost never taken
into account.
• The erosion is defined in the standard way as follows:
f (x) = {f (y) : f (y) = inf [f (z)] , z Bx }

and the dilation:

 f (x) = {f (y) : f (y) = sup [f (z)] , z  Bx }
 Example
Brightness at the

Opening
Erosion
top level

Closing
Dilation

Original Image
(293  418)
SE: Square of
size 5  5
 Lexicographical Order with
Saturation at the Top Level
• We define
S i  S j
 or

ci  c j if S i  S j and Li  L j
 or

S i  S j and Li  L j and H i  H 0  H j  H 0
• and
Si  S j
 or

ci  c j if S i  S j and Li  L j
 or

S i  S j and Li  L j and H i  H 0  H j  H 0
Example
Saturation at the

Opening
Erosion
top level

Closing
Dilation

Original image
(293  418)

SE: Square of
size 5  5
 Colour Top-Hat

• We can define a top-hat operator for colour images

similar to the one used for greyscale images.
• We simply calculate the Euclidean distance E
between each corresponding pixel of the original
image I and the result of one of the colour opening 
or closing  operators:
– Opening top-hat: E I , I 

– Closing top-hat: E I , I 
Original Image Saturation
Example

We wish to extract the greyish These lines are visible in the

lines in between the mosaic tiles saturation image

A colour morphological closing Top-hat operator. The pixel-by-

using a lexicographical order with pixel Euclidean distance between
saturation at the first level the two colour images to the left.
 Summary
• A 3D-polar coordinate representation of the RGB colour space
can be very useful in image processing and analysis.
• One should be careful with how the saturation is defined.
• Applications demonstrated are:
– Colour statistics.
– Colour morphology.
• Further applications include:
– Colour histograms.
– 2-dimensional brightness-saturation histograms (have been used for
segmentation).
– …