Digital Image Processing

Chapter 1: Introduction
What Is Digital Image Processing?

A sample
digital
image.
662*640*2
56.
y

x
 f(x,y): A two-dimensional function,
where x and y are spatial
coordinates, and the amplitude of f
at any pair of coordinates (x,y) is
called the intensity or gray level of
the image at that point.
x size: 662, y size: 640, gray levels:
256
Digital image: x, y, and the
amplitude values of f are all finite,
discrete quantities
 A sample
color
digital
image,
800*600
*24 bits
 Pixel: The elements of a digital
image.

Pixels
What is an image?

Some form of representation for

visual information.
A picture worth a thousand words.
A digital image is composed of
digitized quantities defined on a
rectangular grid of pixels. For
example, 1024x1024 ~ 1megpixel.
Mathematically, a digital image is a
matrix of number, or a function on
a rectangular domain.
 What is an image?

Some form
Enhance, of representation
extract for
wanted information
visual
from, information.
analyze and interpret an
image.
A picture worth a thousand words.
Low
A level
digital image is composed of
digitized quantities
compression, denoising, defined on a
deblurring,
rectangular
segmentation, grid of pixels. For
Highexample,
level 1024x1024 ~ 1megpixel.
Mathematically, a digital image is a
recognition/classification,
matrix of number,
interpretation, or a function on
a rectangular domain.
 What is an image?
Some form of representation for visual
information.
A picture worth a thousand words.
A digital image is composed of digitized
quantities defined on a rectangular grid of
pixels. For example, 1024x1024 ~ 1megpixel.
Mathematically, a digital image is a matrix of
number, or a function on a rectangular
domain.
An image contains rich information with a lot
of redundancy.
What math has to do with it?

Mathematical modeling
characterize/quantify features, such
as noise, edge, textures, shape, .
Mathematical theory

Efficient numerical algorithms

For digital image processing, it is

all about numbers and math!
Denoising
Inpainting
Deblurring + inpainting
Segmentation
Shape recognition
Face recognition
The Origins of Digital Image
Processing

One of the first applications of

digital images was in the newspaper
industry, when pictures were first
sent by submarine cable between
London and New York.
 Medical imaging
Remote Earth resource observations
Astronomy
High-energy plasmas and electron
microscopy
Examples of Fields tat Use Digital
Image Processing

Electromagnetic energy spectrum

Gamma-
Ray
Imaging
X-ray
Imaging
Imaging in
the
Ultraviolet
Band
Imaging in
the Visible
and Infrared
Bands
 Remote sensing
 Weather
Observa
tion,
visible
and
infrared
bands
 Infrared
imaging
 Automated
visual
inspection
 Examples: Industrial Inspection
Human operators are
Images taken from Gonzalez & Woods, Digital Image Processing (2002)

expensive, slow and

unreliable
Make machines do the
job instead
Industrial vision
systems
are used in all kinds of
industries
Can we trust them?
Examples: PCB Inspection

Printed Circuit Board (PCB)

inspection
Machine inspection is used to
determine that all components are
present and that all solder joints are
acceptable
Both conventional imaging and x-ray
imaging are used
Imaging in the Microwave Band
 Imaging in the Radio Band

MRI
Examples in which Other Imaging
Modalities Are Used

Sound
 Ultrasound
 Electron Microscope
 Images generated by computers
Fundamental Steps in Digital Image
Processing

Methods whose input and output

are images
Methods whose outputs are
attributes extracted from those
images
 What is a Digital Image?
Images taken from Gonzalez & Woods, Digital Image Processing (2002)

A digital image is a representation

of a two-dimensional image as a finite
set of digital values, called picture
elements or pixels
 What is a Digital Image? (cont)
Images taken from Gonzalez & Woods, Digital Image Processing (2002)

Pixel values typically represent gray

levels, colours, heights, opacities etc
Remember digitization implies that
a digital image is an approximation of
a real scene
1 pixel
What is a Digital Image? (cont)

Common image formats include:

1 sample per point (B&W or Grayscale)
3 samples per point (Red, Green, and
Blue)
4 samples per point (Red, Green, Blue,
and Alpha, a.k.a. Opacity)
What is Digital Image Processing?

Digital
image processing focuses on
two major tasks
Improvement of pictorial information
for human interpretation
Processing of image data for storage,
transmission and representation for
autonomous machine perception
Some argument about where image
processing ends and fields such as
image analysis and computer vision
start
 What is DIP? (cont)

The continuum from image

processing to computer vision can be
broken up into low-, mid- and high-
level processes
Low Level Process Mid Level Process High Level Process
Input: Image Input: Image Input: Attributes
Output: Image Output: Attributes Output: Understanding
Examples: Noise Examples: Object Examples: Scene
removal, image recognition, understanding,
sharpening segmentation autonomous navigation

In this course we will

stop here
 History of Digital Image Processing
Images taken from Gonzalez & Woods, Digital Image Processing (2002)

Early 1920s: One of the first

applications of digital imaging was in
the news-
paper industry Early digital image
The Bartlane cable picture
transmission service
Images were transferred by submarine
cable between London and New York
Pictures were coded for cable transfer
and reconstructed at the receiving end
on a telegraph printer
 History of DIP (cont)
Images taken from Gonzalez & Woods, Digital Image Processing (2002)

Mid to late 1920s: Improvements

to the Bartlane system resulted in
higher quality images
New reproduction
processes based
on photographic
techniques
Increased number
of tones in Improved
digital image Early 15 tone digital
reproduced images image
 History of DIP (cont)
Images taken from Gonzalez & Woods, Digital Image Processing (2002)

1960s: Improvements in computing

technology and the onset of the space
race led to a surge of work in digital
image processing
1964: Computers used to
improve the quality of
images of the moon taken
by the Ranger 7 probe
Such techniques were used
A picture of the moon taken
in other space missions by the Ranger 7 probe
including the Apollo landings
minutes before landing
 History of DIP (cont)
Images taken from Gonzalez & Woods, Digital Image Processing (2002)

1970s: Digital image processing

begins to be used in medical
applications
1979: Sir Godfrey N.
Hounsfield & Prof. Allan M.
Cormack share the Nobel
Prize in medicine for the
invention of tomography,
the technology behind Typical head slice CAT
image
Computerised Axial
Tomography (CAT) scans
History of DIP (cont)

1980s - Today: The use of digital

image processing techniques has
exploded and they are now used for
all kinds of tasks in all kinds of areas
Image enhancement/restoration
Artistic effects
Medical visualisation
Industrial inspection
Law enforcement
Human computer interfaces
Examples: The Hubble Telescope

Launched in 1990 the Hubble

telescope can take images of
very distant objects
However, an incorrect mirror
made many of Hubbles
images useless
Image processing
techniques were
used to fix this
 Examples: Artistic Effects
Artistic effects are
used to make
images more
visually appealing,
to add special
effects and to
make composite
images
 Examples: Medicine
Images taken from Gonzalez & Woods, Digital Image Processing (2002)

Take slice from MRI scan of canine

heart, and find boundaries between
types of tissue
Image with gray levels representing
tissue density
Use a suitable filter to highlight edges

Original MRI Image of a Dog Heart Edge Detection Image

 Examples: GIS
Images taken from Gonzalez & Woods, Digital Image Processing (2002)

Geographic Information Systems

Digital image processing techniques are
used extensively to manipulate satellite
imagery
Terrain classification
Meteorology
 Examples: GIS (cont)
Night-Time Lights of
Images taken from Gonzalez & Woods, Digital Image Processing (2002)

the World data set

Global inventory of
human settlement
Not hard to imagine
the kind of analysis
that might be done
using this data
 Examples: Industrial Inspection
Human operators are
Images taken from Gonzalez & Woods, Digital Image Processing (2002)

expensive, slow and

unreliable
Make machines do the
job instead
Industrial vision
systems
are used in all kinds of
industries
Can we trust them?
 Examples: Law Enforcement
Image processing
Images taken from Gonzalez & Woods, Digital Image Processing (2002)

techniques are used

extensively by law
enforcers
Number plate
recognition for speed
cameras/automated
toll systems
Fingerprint recognition
Enhancement of CCTV
images
 Examples: HCI
Try to make human
computer interfaces more
natural
Face recognition
Gesture recognition
Does anyone remember
the
user interface from
Minority Report?
These tasks can be
extremely difficult
 Key Stages in Digital Image
Processing
Image Morphological
Restoration Processing

Image
Segmentation
Enhancement

Image Object
Acquisition Recognition

Representation
Problem Domain
& Description
Colour Image Image
Processing Compression
 Key Stages in Digital Image
Processing:
Image Aquisition
Images taken from Gonzalez & Woods, Digital Image Processing (2002)

Image Morphological
Restoration Processing

Image
Segmentation
Enhancement

Image Object
Acquisition Recognition

Representation
Problem Domain
& Description
Colour Image Image
Processing Compression
 Key Stages in Digital Image
Processing:
Image Enhancement
Images taken from Gonzalez & Woods, Digital Image Processing (2002)

Image Morphological
Restoration Processing

Image
Segmentation
Enhancement

Image Object
Acquisition Recognition

Representation
Problem Domain
& Description
Colour Image Image
Processing Compression
 Key Stages in Digital Image
Processing:
Image Restoration
Images taken from Gonzalez & Woods, Digital Image Processing (2002)

Image Morphological
Restoration Processing

Image
Segmentation
Enhancement

Image Object
Acquisition Recognition

Representation
Problem Domain
& Description
Colour Image Image
Processing Compression
 Key Stages in Digital Image
Processing:
Morphological Processing
Images taken from Gonzalez & Woods, Digital Image Processing (2002)

Image Morphological
Restoration Processing

Image
Segmentation
Enhancement

Image Object
Acquisition Recognition

Representation
Problem Domain
& Description
Colour Image Image
Processing Compression
 Key Stages in Digital Image
Processing:
Segmentation
Images taken from Gonzalez & Woods, Digital Image Processing (2002)

Image Morphological
Restoration Processing

Image
Segmentation
Enhancement

Image Object
Acquisition Recognition

Representation
Problem Domain
& Description
Colour Image Image
Processing Compression
 Key Stages in Digital Image
Processing:
Object Recognition
Images taken from Gonzalez & Woods, Digital Image Processing (2002)

Image Morphological
Restoration Processing

Image
Segmentation
Enhancement

Image Object
Acquisition Recognition

Representation
Problem Domain
& Description
Colour Image Image
Processing Compression
 Key Stages in Digital Image
Processing:
Representation & Description
Images taken from Gonzalez & Woods, Digital Image Processing (2002)

Image Morphological
Restoration Processing

Image
Segmentation
Enhancement

Image Object
Acquisition Recognition

Representation
Problem Domain
& Description
Colour Image Image
Processing Compression
 Key Stages in Digital Image
Processing:
Image Compression
Image Morphological
Restoration Processing

Image
Segmentation
Enhancement

Image Object
Acquisition Recognition

Representation
Problem Domain
& Description
Colour Image Image
Processing Compression
 Key Stages in Digital Image
Processing:
Colour Image Processing
Image Morphological
Restoration Processing

Image
Segmentation
Enhancement

Image Object
Acquisition Recognition

Representation
Problem Domain
& Description
Colour Image Image
Processing Compression
Summary

We have looked at:

What is a digital image?
What is digital image processing?
History of digital image processing
State of the art examples of digital
image processing
Key stages in digital image processing
Next time we will start to see how it
all works
 Image operations
 Spatial filtering
 Filtering in
the
frequency
domain
 Image
restoration
 Color image processing
Components of an Image Processing
System
 Digital Image Processing

Lecture 2 Elements of Visual

Perception and Image
Formation
Roadmap
Introduction Structure of human eye
Image format (vector vs.
bitmap) Brightness adaptation
IP vs. CV vs. CG and Discrimination
HLIP vs. LLIP Image formation in
Image acquisition human eye and Image
Image enhancement
Image restoration formation model
Image compression Basics of exposure
Color image processing
Image segmentation Resolution
Image description
Sampling and
Pattern recognition
quantization
Research issues

76
Questions
Brightness adaptation
Dynamic range
Weber ratio
Cones vs. rods
Hexagonal sampling
Fovea or blind spot
Flexible lens and ciliary body
Near sighted vs. far sighted
Image resolution
Sampling vs. quantization
77
Structure of the human eye

The cornea and sclera outer cover

The choroid
Ciliary body
Iris diaphragm
Lens
The retina
Cones vision (photopic/bright-light
vision): centered at fovea, highly
sensitive to color
Rods (scotopic/dim-light vision): general
view
Blind spot 78
Human eye

79
Cones vs. Rods

80
Hexagonal pixel

Cone distribution on
the fovea (200,000
cones/mm2)

Models human visual system more precisely

The distance between a given pixel and its
immediate neighbors is the same
Hexagonal sampling requires 13% fewer
samples than rectangular sampling
ANN can be trained with less errors

81
More on the cone
mosaic

The cone mosaic of fish retina

http://www.nibb.ac.jp/annual_report/2003/03ann502.html

Lythgoe, Ecology of Vision (1979)

Human retina mosaic The mosaic array of

-Irregularity reduces visual
acuity for high-frequency signals most vertebrates is
-Introduce random noise regular
82
A mosaicked multispectral camera

83
Brightness adaptation

Dynamic range of
human visual
system
10-6 ~ 104
Cannot accomplish
this range
simultaneously
The current
sensitivity level of
the visual system
is called the
brightness 84
Brightness discrimination

Weber ratio (the experiment) DIc/I

I: the background illumination
DIc : the increment of illumination
Small Weber ratio indicates good discrimination
Larger Weber ratio indicates poor discrimination

85
Psychovisual effects

The perceived
brightness is not
a simple
function of
intensity
Mach band
pattern
Simultaneous
contrast
And more (see
link) 86
Image formation in the eye

Flexible lens
Controlled by the tension in the
fibers of the ciliary body
To focus on distant objects?
To focus on objects near eye?
Near-sighted and far-sighted

87
Image formation in the eye

Light receptor Brain

radiant electrical
energy impulses

88
 A simple image formation model
f(x,y): the intensity is called the gray level
for monochrome image
f(x, y) = i(x, y).r(x, y)
0 < i(x, y) < inf, the illumination (lm/m2)
0< r(x, y) < 1, the reflectance
Some illumination figures (lm/m2)
90,000: full sun - 0.01: black
velvet
10,000: cloudy day - 0.93: snow
0.1: full moon
1,000: commercial office

89
Camera exposure
ISO number
Sensitivity of the film or the sensor
Can go as high as 1,600 and 3,200
Shutter speed
How fast the shutter is opened and
closed
f/stop
The size of aperture
1.0 ~ 32

90
Sampling and Quantization

91
 Uniform sampling
Digitized in spatial domain (IM x N)
M and N are usually integer powers of
two
Nyquist theorem and Aliasing
Sampled
(0,0) (0,1) (0,2) (0,3) (0,0) (0,0) (0,2) (0,2)
by 2
(1,0) (1,1) (1,2) (1,3) (0,0) (0,0) (0,2) (0,2)
(2,0) (2,1) (2,2) (2,3) (2,0) (2,0) (2,2) (2,2)
(3,0) (3,1) (3,2) (3,3) (2,0) (2,0) (2,2) (2,2)

Non-uniform sampling
communication
92
More on aliasing

Aliasing (the Moire effect)

93
original Sampled by 2 Sampled by 4

94
Sampled by 8 Sampled by 16
Uniform quantization

Digitized in amplitude (or pixel

value)
PGM 256 levels 4 levels
255 3

192
2
128
1
64

0 0
95
original 128 levels (7 bits) 16 levels (4 bits)

96
4 levels (2 bits) 2 levels (1 bit)
Image resolution
Spatial resolution
Line pairs per unit distance
Dots/pixels per unit distance
dots per inch - dpi
Intensity resolution
Smallest discernible change in intensity
level

The more samples in a fixed range,

the higher the resolution
The more bits, the higher the
resolution 97
The research

Artificial retina
(refer to the link)
Artificial vision
(refer to the link)
3-D interpretation
of line drawing
Compress sensing

98
3D interpretation of line drawing

Emulation approach
A given 3-D interpretation is considered less likely
to be correct if some angles between the wires are
much larger than others

99
Research publications
Conferences (IEEE)
International Conference on Image Processing
(ICIP)
International Conference on Computer Vision
(ICCV)
International Conference on Computer Vision and
Pattern Recognition (CVPR)
Journals (IEEE)
Transactions on Image Processing (TIP)
Transactions on Medical Imaging (TMI)
Transactions on Pattern Analysis and Machine
Intelligence (PAMI)
IEEE Explore
100
Summary
Structure of human eye
Photo-receptors on retina (cones vs.
rods)
Brightness adaptation
Brightness discrimination (Weber
ratio)
Be aware of psychovisual effects
Image formation models
Digital imaging
Sampling vs. quantization
Image resolution
101
A Simple Image Model

Image: a 2-D light-intensity

function f(x,y)

The value of f at (x,y) the

intensity (brightness) of the image
at that point

0 < f(x,y) <

Digital Image Acquisition
A Simple Image Model

Nature of f(x,y):

The amount of source light incident on

the scene being viewed

The amount of light reflected by the

objects in the scene
 A Simple Image Model
Illumination & reflectance components:

Illumination: i(x,y)
Reflectance: r(x,y)

f(x,y) = i(x,y) r(x,y)

0 < i(x,y) <

and 0 < r(x,y) < 1
(from total absorption to total
reflectance)
A Simple Image Model

Sample values of r(x,y):

0.01: black velvet
0.93: snow

Sample values of i(x,y):

9000 foot-candles: sunny day
1000 foot-candles: cloudy day
0.01 foot-candles: full moon
A Simple Image Model

Intensity of a monochrome image f

at (xo,yo): gray level l of the image
at that point
l=f(xo, yo)

Lmin l Lmax
Where Lmin: positive
Lmax: finite
A Simple Image Model
In practice:
Lmin = imin rmin and
Lmax = imax rmax

E.g. for indoor image processing:

Lmin 10 Lmax 1000

[Lmin, Lmax] : gray scale

Often shifted to [0,L-1] l=0: black
l=L-1: white
Sampling & Quantization

The spatial and amplitude

digitization of f(x,y) is called:

image sampling when it refers to

spatial coordinates (x,y) and

gray-level quantization when it refers

to the amplitude.
Digital Image
Sampling and Quantization
A Digital Image
Sampling & Quantization

f (0,0) f (0,1) ... f (0, M 1)

f (1,0) ... ... f (1, M 1)
f ( x, y )
... ... ... ...

f ( N 1,0) f ( N 1,1) ... f ( N 1, M 1)

Digital Image Image Elements

(Pixels)
 Sampling & Quantization

Important terms for future discussion:

Z: set of real integers

R: set of real numbers

 Sampling & Quantization

Sampling: partitioning xy plane into a

grid

the coordinate of the center of each grid is

a pair of elements from the Cartesian
product Z x Z (Z2)

Z2 is the set of all ordered pairs of

elements (a,b) with a and b being
integers from Z.
Sampling & Quantization

f(x,y) is a digital image if:

(x,y) are integers from Z2 and

f is a function that assigns a gray-level
value (from R) to each distinct pair of
coordinates (x,y) [quantization]

Gray levels are usually integers

then Z replaces R
Sampling & Quantization

The digitization process requires

decisions about:

values for N,M (where N x M: the

image array)

and

the number of discrete gray levels

allowed for each pixel.
Sampling & Quantization

Usually, in DIP these quantities are

integer powers of two:
N=2n M=2m and G=2k

number of gray
levels

Another assumption is that the

discrete levels are equally spaced
between 0 and L-1 in the gray
scale.
Examples
Examples
Examples
Examples
Sampling & Quantization

If b is the number of bits required

to store a digitized image then:

b = N x M x k (if M=N, then b=N2k)

Storage
Sampling & Quantization

How many samples and gray levels

are required for a good
approximation?

Resolution (the degree of discernible

detail) of an image depends on sample
number and gray level number.
i.e. the more these parameters are
increased, the closer the digitized array
approximates the original image.
Sampling & Quantization

How many samples and gray levels

are required for a good
approximation? (cont.)

But: storage & processing requirements

increase rapidly as a function of N, M,
and k
Sampling & Quantization

Different versions (images) of the

same object can be generated
through:

Varying N, M numbers
Varying k (number of bits)
Varying both
Sampling & Quantization

Isopreference curves (in the Nm

plane)

Each point: image having values of N

and k equal to the coordinates of this
point

Points lying on an isopreference curve

correspond to images of equal
subjective quality.
Examples
Isopreference Curves
 Sampling & Quantization

Conclusions:
Quality of images increases as N & k
increase
Sometimes, for fixed N, the quality
improved by decreasing k (increased
contrast)
For images with large amounts of detail,
few gray levels are needed
Nonuniform
Sampling & Quantization

An adaptive sampling scheme can improve

the appearance of an image, where the
sampling would consider the characteristics
of the image.

i.e. fine sampling in the neighborhood of sharp

gray-level transitions (e.g. boundaries)
Coarse sampling in relatively smooth regions

Considerations: boundary detection, detail

content
Nonuniform
Sampling & Quantization

Similarly: nonuniform quantization

process

In this case:
few gray levels in the neighborhood of
boundaries
more in regions of smooth gray-level
variations (reducing thus false
contours)