EE465: Introduction to Digital

Image Processing 1
Limitation of Imaging
Technology
Two plagues in image acquisition
Noise interference
Blur (motion, out-of-focus, hazy weather)
Difficult to obtain high-quality images
as imaging goes
Beyond visible spectrum
Micro-scale (microscopic imaging)
Macro-scale (astronomical imaging)
What is Noise?
Wiki definition: noise means any
unwanted signal
One persons signal is another ones
noise
Noise is not always random and
randomness is an artificial term
Noise is not always bad (see stochastic
resonance example in the next slide)
EE465: Introduction to Digital
Image Processing 2
Stochastic Resonance
EE465: Introduction to Digital
Image Processing 3
no noise heavy noise
light noise
EE465: Introduction to Digital
Image Processing 4
Image Denoising
Where does noise come from?
Sensor (e.g., thermal or electrical
interference)
Environmental conditions (rain, snow etc.)
Why do we want to denoise?
Visually unpleasant
Bad for compression
Bad for analysis
EE465: Introduction to Digital
Image Processing 5
electrical interference
Noisy Image Examples
thermal imaging
ultrasound imaging
physical interference
EE465: Introduction to Digital
Image Processing 6
(Ad-hoc) Noise Modeling
Simplified assumptions
Noise is independent of signal
Noise types
Independent of spatial location
Impulse noise
Additive white Gaussian noise
Spatially dependent
Periodic noise


EE465: Introduction to Digital
Image Processing 7
Noise Removal Techniques
Linear filtering
Nonlinear filtering
Recall
Linear system
EE465: Introduction to Digital
Image Processing 8
Image Denoising
Introduction
Impulse noise removal
Median filtering
Additive white Gaussian noise removal
2D convolution and DFT
Periodic noise removal
Band-rejection and Notch filter
EE465: Introduction to Digital
Image Processing 9
Impulse Noise (salt-pepper Noise)
Definition
Each pixel in an image has the probability of p/2 (0<p<1) being
contaminated by either a white dot (salt) or a black dot (pepper)
W j H i
j i X
j i Y
s s s s

=
1 , 1
) , (
0
255
) , (
with probability of p/2
with probability of p/2
with probability of 1-p
noisy pixels
clean pixels
X: noise-free image, Y: noisy image
Note: in some applications, noisy pixels are not simply black or white,
which makes the impulse noise removal problem more difficult
EE465: Introduction to Digital
Image Processing 10
Numerical Example
P=0.1
128 128 128 128 128 128 128 128 128 128
128 128 128 128 128 128 128 128 128 128
128 128 128 128 128 128 128 128 128 128
128 128 128 128 128 128 128 128 128 128
128 128 128 128 128 128 128 128 128 128
128 128 128 128 128 128 128 128 128 128
128 128 128 128 128 128 128 128 128 128
128 128 128 128 128 128 128 128 128 128
128 128 128 128 128 128 128 128 128 128
128 128 128 128 128 128 128 128 128 128
X
128 128 255 0 128 128 128 128 128 128
128 128 128 128 0 128 128 128 128 0
128 128 128 128 128 128 128 128 128 128
128 128 0 128 128 128 128 128 128 128
128 128 128 128 128 128 128 128 128 128
128 128 128 128 128 128 128 128 128 128
128 128 128 128 128 128 128 128 128 128
0 128 128 128 128 255 128 128 128 128
128 128 128 128 128 128 128 128 128 255
128 128 128 128 128 128 128 255 128 128
Y
Noise level p=0.1 means that approximately 10% of pixels are contaminated by
salt or pepper noise (highlighted by red color)

EE465: Introduction to Digital
Image Processing 11
MATLAB Command
>Y = IMNOISE(X,'salt & pepper',p)
Notes:
- The intensity of input images is assumed to be normalized to [0,1].
If X is double, you need to do normalization first, i.e., X=X/255;
If X is uint8, MATLAB would do the normalization automatically
- The default value of p is 0.05 (i.e., 5 percent of pixels are contaminated)
- imnoise function can produce other types of noise as well (you need to
change the noise type salt & pepper)
EE465: Introduction to Digital
Image Processing 12
Impulse Noise Removal Problem
Noisy image Y
filtering
algorithm
Can we make the denoised image X as close
to the noise-free image X as possible?
^
X
^
denoised
image
EE465: Introduction to Digital
Image Processing 13
Median Operator
Given a sequence of numbers {y
1
,,y
N
}
Mean: average of N numbers
Min: minimum of N numbers
Max: maximum of N numbers
Median: half-way of N numbers

Example
] 56 , 55 , 54 , 255 , 52 , 0 , 50 [ = y

54 ) ( = y median

] 255 , 56 , 55 , 54 , 52 , 50 , 0 [ = y

sorted
EE465: Introduction to Digital
Image Processing 14
y(n)
W=2T+1
)] ( ),..., ( ),..., ( [ ) (

T n y n y T n y median n x + =
1D Median Filtering

Note: median operator is nonlinear
MATLAB command: x=median(y(n-T:n+T));
EE465: Introduction to Digital
Image Processing 15
Numerical Example
] 56 , 55 , 54 , 255 , 52 , 0 , 50 [ = y

T=1:
] , 56 , 55 , 54 , 255 , 52 , 0 , 50 , [ 56 50 = y

] 55 , 55 , 54 , 54 , 52 , 50 , 50 [

= x

Boundary
Padding
EE465: Introduction to Digital
Image Processing 16
x(m,n)
W: (2T+1)-by-(2T+1) window
2D Median Filtering
)] , ( ),..., , ( ),..., , (
),..., , ( ),..., , ( [ ) , (

T n T m y T n T m y n m y
T n T m y T n T m y median n m x
+ + +
+ =
MATLAB command: x=medfilt2(y,[2*T+1,2*T+1]);
EE465: Introduction to Digital
Image Processing 17
Numerical Example
225 225 225 226 226 226 226 226
225 225 255 226 226 226 225 226
226 226 225 226 0 226 226 255
255 226 225 0 226 226 226 226
225 255 0 225 226 226 226 255
255 225 224 226 226 0 225 226
226 225 225 226 255 226 226 228
226 226 225 226 226 226 226 226
0 225 225 226 226 226 226 226
225 225 226 226 226 226 226 226
225 226 226 226 226 226 226 226
226 226 225 225 226 226 226 226
225 225 225 225 226 226 226 226
225 225 225 226 226 226 226 226
225 225 225 226 226 226 226 226
226 226 226 226 226 226 226 226
Y
X
^
Sorted: [0, 0, 0, 225, 225, 225, 226, 226, 226]
EE465: Introduction to Digital
Image Processing 18
Image Example
P=0.1
Noisy image Y X
^
denoised
image
3-by-3 window
EE465: Introduction to Digital
Image Processing 19
Image Example (Cont)
3-by-3 window 5-by-5 window
clean
noisy
(p=0.2)
EE465: Introduction to Digital
Image Processing 20
Reflections
What is good about median operation?
Since we know impulse noise appears as
black (minimum) or white (maximum) dots,
taking median effectively suppresses the
noise
What is bad about median operation?
It affects clean pixels as well
Noticeable edge blurring after median
filtering
EE465: Introduction to Digital
Image Processing 21
Idea of Improving Median Filtering
Can we get rid of impulse noise without
affecting clean pixels?
Yes, if we know where the clean pixels are
or equivalently where the noisy pixels are
How to detect noisy pixels?
They are black or white dots

EE465: Introduction to Digital
Image Processing 22
Median Filtering with Noise Detection
Noisy image Y
x=medfilt2(y,[2*T+1,2*T+1]);
Median filtering
Noise detection
C=(y==0)|(y==255);
xx=c.*x+(1-c).*y;
Obtain filtering results
EE465: Introduction to Digital
Image Processing 23
Image Example
clean
noisy
(p=0.2)
w/o
noise
detection
with
noise
detection
EE465: Introduction to Digital
Image Processing 24
Image Denoising
Introduction
Impulse noise removal
Median filtering
Additive white Gaussian noise removal
2D convolution and DFT
Periodic noise removal
Band-rejection and Notch filter
EE465: Introduction to Digital
Image Processing 25
Additive White Gaussian Noise
Definition
Each pixel in an image is disturbed by a Gaussian random variable
With zero mean and variance o
2

W j H i N j i N
j i N j i X j i Y
s s s s
+ =
1 , 1 ), , 0 ( ~ ) , (
), , ( ) , ( ) , (
2
o
X: noise-free image, Y: noisy image
Note: unlike impulse noise situation, every pixel in the image contaminated
by AWGN is noisy
EE465: Introduction to Digital
Image Processing 26
Numerical Example
o
2
=1
X Y

128 128 128 128 128 128 128 128
128 128 128 128 128 128 128 128
128 128 128 128 128 128 128 128
128 128 128 128 128 128 128 128
128 128 128 128 128 128 128 128
128 128 128 128 128 128 128 128
128 128 128 128 128 128 128 128
128 128 128 128 128 128 128 128
128 128 129 127 129 126 126 128
126 128 128 129 129 128 128 127
128 128 128 129 129 127 127 128
128 129 127 126 129 129 129 128
127 127 128 127 129 127 129 128
129 130 127 129 127 129 130 128
129 128 129 128 128 128 129 129
128 128 130 129 128 127 127 126
EE465: Introduction to Digital
Image Processing 27
MATLAB Command
>Y = IMNOISE(X,gaussian',m,v)
>Y = X+m+randn(size(X))*v;
or
Note:
rand() generates random numbers uniformly distributed over [0,1]
randn() generates random numbers observing Gaussian distribution
N(0,1)
EE465: Introduction to Digital
Image Processing 28
Image Denoising
Noisy image Y
filtering
algorithm
Question: Why not use median filtering?
Hint: the noise type has changed.
X
^
denoised
image
EE465: Introduction to Digital
Image Processing 29
f(n)
h(n)
g(n)
) ( ) ( ) ( ) ( ) ( ) ( ) ( n h n f n f n h k n f k h n g
k
= = =

=
- Linearity
- Time-invariant property
) ( ) ( ) ( ) (
2 2 1 1 2 2 1 1
n g a n g a n f a n f a + +
) ( ) (
0 0
n n g n n f
Linear convolution
1D Linear Filtering
See review section
EE465: Introduction to Digital
Image Processing 30

=
jwn
e n f w F ) ( ) (
}

=
t
t
t
dw e w F n f
jwn
) (
2
1
) (
forward
inverse
Note that the input signal is a discrete sequence
while its FT is a continuous function
) ( ) ( n h n f
time-domain convolution
) ( ) ( w H w F
frequency-domain multiplication
Fourier Series
EE465: Introduction to Digital
Image Processing 31
Filter Examples
0 0.5 1 1.5 2 2.5 3 3.5
0
0.2
0.4
0.6
0.8
1
1.2
1.4
1.6
1.8
2
LP
HP
w
|H(w)|
Low-pass (LP)
h(n)=[1,1]
|h(w)|=2cos(w/2)
High-pass (LP)
h(n)=[1,-1]
|h(w)|=2sin(w/2)
EE465: Introduction to Digital
Image Processing 32
forward transform inverse transform
Properties
- periodic
- conjugate symmetric
) ( ) ( k y N k y = +
) ( ) (
*
k y k N y =
1D Discrete Fourier Transform
Proof:

=
=
1
0
) ( ) (
N
n
kn
N
W n x k y

=
1
0
) ( ) (
N
n
kn
N
W k y n x


=

=
+
= = = +
1
0
1
0
) (
) ( ) (
N
n
kn
N n
N
n
n N k
N n
k y W x W x N k y


=

= = =
1
0
*
1
0
) (
) ( ) (
N
n
kn
N n
N
n
n k N
N n
k y W x W x k N y
Proof:
}
2
exp{
N
j
W
N
t
=
EE465: Introduction to Digital
Image Processing 33
Matrix Representation of 1D DFT
F A =
(
(
(
(

NN N
N
N N
a a
a a
... ...
... ... ... ...
... ... ... ...
... ...
1
1 11
1 ,
,
1
2
= =
=

N
N
N
j
N
kl
N kl
W e W
W
N
a
t
Re
Im
N
W
kl
N
N
l
l k
W x
N
y

=
=
1
1

=
=
N
l
l kl k
x a y
1
DFT:
EE465: Introduction to Digital
Image Processing 34
Fast Fourier Transform (FFT)*
Invented by Tukey and Cooley in 1965
Basic idea: divide-and-conquer





Reduce the complexity of N-point DFT from O(N
2
)
to O(Nlog
2
N)



=

=
=

=
+ =
+ = =
1 2
2 / 1 2
2
2 / 2
1 2
) 1 2 (
1 2
2
2
2
1
0
m n
km
N m
k
m n
km
N m
m n
m k
N m
m n
m k
N m
N
n
kn
N n k
W x W W x
W x W x W x y
N
N/2-point DFT
N/2-point DFT
N-point DFT
EE465: Introduction to Digital
Image Processing 35
Filtering in the Frequency Domain
) ( ) ( ) ( n h n f n g =
convolution in the time domain is equivalent to
multiplication in the frequency domain
) ( ) ( ) ( k H k F k G =
f(n) h(n)
g(n) F(k) H(k) G(k)
DFT
EE465: Introduction to Digital
Image Processing 36
2D Linear Filtering
f(m,n)
h(m,n)
g(m,n)
) , ( ) , ( ) , ( ) , ( ) , (
,
n m f n m h l n k m f l k h n m g
l k
= =

=
2D convolution
MATLAB function: C = CONV2(A, B)
EE465: Introduction to Digital
Image Processing 37
2D Filtering=Two Sequential
1D Filtering
Just as we have observed with 2D
transform, 2D (separable) filtering can
be viewed as two sequential 1D filtering
operations: one along row direction and
the other along column direction
The order of filtering does not matter

) ( ) ( ) ( ) ( ) , (
1 1 1 1
m h n h n h m h n m h = =
h
1
: 1D filter
EE465: Introduction to Digital
Image Processing 38
Numerical Example
h
1
(m)=[1,1], h
1
(n)=[1,-1]
1D filter
(

1 1
1 1
) ( ) (
1 1
n h m h
(

1 1
1 1
) ( ) (
1 1
m h n h
MATLAB command:
>h1=[1,1];h2=[1,-1];
>conv2(h1,h2)
>conv2(h2,h1)
EE465: Introduction to Digital
Image Processing 39
Fourier Series (2D case)

=
+
=
m n
n w m w j
e n m f w w F
) (
2 1
2 1
) , ( ) , (
Note that the input signal is discrete
while its FT is a continuous function
) , ( ) , ( n m h n m f
spatial-domain convolution
) , ( ) , (
2 1 2 1
w w H w w F
frequency-domain multiplication
EE465: Introduction to Digital
Image Processing 40
Filter Examples
Low-pass (LP)
h
1
(n)=[1,1]
|h
1
(w)|=2cos(w/2)
1D
h(n)=[1,1;1,1]
|h(w
1
,w
2
)|=4cos(w
1
/2)cos(w
2
/2)
2D
w
1

w
2

|h(w
1
,w
2
)|
EE465: Introduction to Digital
Image Processing 41
Image DFT Example
Original ray image X
choice 1: Y=fft2(X)
EE465: Introduction to Digital
Image Processing 42
Image DFT Example (Cont)
choice 1: Y=fft2(X) choice 2: Y=fftshift(fft2(X))
Low-frequency at the center Low-frequency at four corners
FFTSHIFT Shift zero-frequency component to center of spectrum.
EE465: Introduction to Digital
Image Processing 43
Gaussian Filter
)
2
exp( ) , (
2
2
2
2
1
2 1
o
w w
w w H
+
=
)
2
exp( ) , (
2
2 2
o
n m
n m h
+
=
FT
>h=fspecial(gaussian, HSIZE,SIGMA); MATLAB code:
EE465: Introduction to Digital
Image Processing 44
(o=1)
PSNR=24.4dB
Image Example
PSNR=20.2dB
noisy
(o=25)
denoised
denoised
(o=1.5)
PSNR=22.8dB
Matlab functions: imfilter, filter2
45
Gaussian Filter=Heat Diffusion
2
2
2
2
) , , ( ) , , (
) , , (
) , , (
y
t y x I
x
t y x I
t y x I
t
t y x I
c
c
+
c
c
= A =
c
c
Linear Heat Flow Equation:
) ( ) 0 , , ( ) , , ( t G y x I t y x I =
scale
A Gaussian filter
with zero mean
and variance of t
Isotropic diffusion:
46
Basic Idea of Nonlinear
Diffusion*
x
y
I(x,y)
image I
image I viewed as a 3D surface (x,y,I(x,y))
Diffusion should be anisotropic
instead of isotropic
Experimental Results
EE465: Introduction to Digital
Image Processing 47
(Gaussian filtering)
PSNR=24.4dB PSNR=20.2dB
noisy
(o=25)
linear diffusion
(TV filtering)
PSNR=27.5dB
nonlinear diffusion
Hammer-Nail Analogy
EE465: Introduction to Digital
Image Processing
48
Gaussian filter
median filter
salt-pepper/
impulse noise
Gaussian noise
periodic noise
???
EE465: Introduction to Digital
Image Processing 49
Image Denoising
Introduction
Impulse noise removal
Median filtering
Additive white Gaussian noise removal
2D convolution and DFT
Periodic noise removal
Band-rejection and Notch filter
EE465: Introduction to Digital
Image Processing 50
Periodic Noise
Source: electrical or electromechanical
interference during image acquistion
Characteristics
Spatially dependent
Periodic easy to observe in frequency domain
Processing method
Suppressing noise component in frequency
domain
EE465: Introduction to Digital
Image Processing 51
Image Example
spatial
Frequency (note the four pairs of bright dots)
EE465: Introduction to Digital
Image Processing 52

Band Rejection Filter

+ s + s
=
otherwise
W
D w w
W
D
w w H
1
2 2
0
) , (
2
2
2
1
2 1
w
1

w
2

EE465: Introduction to Digital
Image Processing 53
Image Example
Before filtering
After filtering
Advanced Denoising
Techniques*
EE465: Introduction to Digital
Image Processing 54
I
N
V
I
s
V
I
E
V
I
W
V
j i j i N
I I I
, , 1
= V

j i j i S
I I I
, , 1
= V
+
j i j i E
I I I
, 1 ,
= V
+
j i j i W
I I I
, 1 ,
= V

] [
,
1
,
I c I c I c I c I I
W W E E S S N N
t
j i
t
j i
V + V + V + V + =
+

W E S N d I g c
d d
, , , ||), (|| = V =
Basic idea: from linear diffusion (equivalent to Gaussian filtering)
to nonlinear diffusion (with implicit edge-stopping criterion)