U-5

Wavelets and Multi-resolution image

processing
L-19 Uncertainty principles of Fourier Transform, Time-frequency
localization
L-20 continuous wavelet transforms
L-21 wavelet bases and multi-resolution analysis,
L-22 wavelets and Sub-band filter banks
L-23 wavelet packets
L-19
Uncertainty principles of Fourier Transform
 Fourier Analysis
Unbounded Function
f(x) = A(k) cos(kx) dk
•x = space, k = wavenumber = 2/
•cos(kx) is a wave with wavelength 2/k
•A(k) is the Fourier transform of f(x)
•f(x) = spatial pattern; A(k) = spectrum
•x and k are conjugates
 Fourier Analysis
f(t) = A() cos(t) dk
•t = time,  = angular frequency= 2/T
•cos(t) is a wave with period 2/
•A() is the Fourier transform of f(t)
•f(t) = time signal; A() = spectrum
•t and  are conjugates
 Meaning of FT

FT
time domain frequency domain

FT
space domain wavenumber domain
 Waves and Uncertainty

Frequency and wavelength known exactly,

position and time not determined

Frequency and wavelength less specific,

position and time more specific

Frequency and wavelength not determined,

position known exactly
 Fourier Transforms
• The narrower the distribution in space f(x),
the more wavenumbers must be specified
• The narrower the distribution in time f(t), the
more frequencies must be specified
 Uncertainty, defined
• Root Mean Square (RMS)
(u)2 = <(u – <u>)2>
 <x> = expectation of x
• It turns out
x k ≥ ½
• and
t  ≥ ½
 Position and Momentum
x k ≥ ½
• Recall p = h/ = h/(2/k) = kh/(2) = k h
• So k = p/h, k = p/h
x p ≥ h/2
 Time and Energy
t  ≥ ½
• Recall E = hf = h(/2) = h
• So  = E/h,  = E/h
t E ≥ h/2
 Uncertainty and Confinement
• Confine mass m to space of size a
• Non-relativistic, K = p2/(2m)
• x p = h/2
• What is the minimum K?
 Time
Timefrequency
frequencylocalization
localization

Most real world signals exhibit the energy compactness properties

where the majority information concentrates in specific regions in the
spectrum.

M-bank filters are used to partition a signal into different frequency

channels, with which energy compact regions in the frequency space
are located.

Although we can locate information at different frequency bands in

this way, an important point is missing.

WHEN? Briefly speaking, the time at which a particular piece of

information exists in a region of the frequency spectrum.

Obviously both localization in time and frequency spaces are

important. Lets have some revision first.
 Filter
Filter, ,convolution
convolutionand
andTransform
Transform

x(t)

h(t-)

y()
 Filter
Filter, ,convolution
convolutionand
andTransform
Transform

x(t)

h(t-)

y()
 Filter
Filter, ,convolution
convolutionand
andTransform
Transform

x(t)

h(t-)

y()
 Filter
Filter, ,convolution
convolutionand
andTransform
Transform

x(t) is the input signal

h(t) is the impulse response of the filter.

y() is the filter output

if h(t) is shifted continuously, y() is also continuous

For a particular instance h(t - ),

y() is the projection of x(t) on h(t - )

y() = < x(t) , h(t - ) > (dot product)

h(t) = ejt ---> Fourier Transform

 Time
TimeFrequency
FrequencyLocalization
Localization

f(t)

Short time Fourier Transform

h(t-b)
 Time
TimeFrequency
FrequencyLocalization
Localization

h(t-b)

t
ejt

w(t-b)
t
 Time
TimeFrequency
FrequencyLocalization
Localization

What is the width of the window ?

What is w(t) ? Should it be a simple on/off function?

 Time
TimeFrequency
FrequencyLocalization
Localization

What is the width of the window ?

What is w(t) ? Should it be a simple on/off function?

If the width is too wide, the time resolution is poor,

but frequency resolution is good

i.e., same spectrum will be assumed over a long time duration

 Time
TimeFrequency
FrequencyLocalization
Localization

What is the width of the window ?

What is w(t) ? Should it be a simple on/off function?

If the width is too wide, the time resolution is poor,

but frequency resolution is good

i.e., same spectrum will be assumed over a long time duration

If the width is too short, the frequency resolution is poor,

but time resolution is good

i.e., not possible to discriminate small difference in frequency

 (t )

t
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
 (t )

t
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

 (t  4)

t
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
 (t )

t
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

 (t  4)

t
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

t  4 
 
 
 2 

t
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
 t b
 
 a 
1.  shifted by b units.

2.  stretched by ‘a’ times if a>1

3.  compress by ‘a’ times if a<1

 
  t  b  1 t b 

j t
F      e
  a   2    a 

 a  a  e
ˆ  b ˆ a  
 a b 0

L et        
ˆ  *

 a     a    
ˆ *
ˆ  is the F o urier Tra nsfo rm o f  t 

A stretch in time window corresponds to a compression in the

frequency window and vice versa
 Time and frequency windows

Frequency window

Short time window:

Long frequency window

Long time window:

Short frequency window

Time window
x(t)

Poor Good
h t   

t  
h 
 2 

t  
h 
 4  Good Poor

Time localization Frequency localization

 Time and frequency windows

To detect low frequency content, a large time window is

required for good frequency localization

To detect high frequency content, a small time window is

required for good time localization
 L-20
Continuous Wavelet
Transforms
Oxford Dictionary: A wavelet is a small wave.

Wikipedia: A wavelet is a mathematical function

used to divide a given function or continuous-time
signal into different scale components.

A Wavelet Transform is the representation of a

function by wavelets.
What is Wavelet Analysis ?
 And…what is a wavelet…?

 A wavelet is a waveform of effectively

limited duration that has an average value
of zero.
 Wavelet's properties
• Short time localized waves with zero integral
value.

• Possibility of time shifting.

• Flexibility.

The Continuous Wavelet Transform (CWT)
 A mathematical representation of the
Fourier transform:

F ( w )  f (t ) e  iwt
dt

 Meaning: the sum over all time of the
signal f(t) multiplied by a complex
exponential, and the result is the Fourier
coefficients F() .
Wavelet Transform (Cont’d)
 Those coefficients, when multiplied by a
sinusoid of appropriate frequency , yield
the constituent sinusoidal component of
the original signal:
 Wavelet Transform
 And the result of the CWT are Wavelet
coefficients .
 Multiplying each coefficient by the
appropriately scaled and shifted wavelet
yields the constituent wavelet of the original
signal:
 Wavelet Transform

• Uses a variable length window, e.g.:

– Narrower windows are more appropriate at high
frequencies
– Wider windows are more appropriate at low
frequencies
 What is a wavelet?
• A function that “waves” above and below the
x-axis with the following properties:
• Varying frequency
• Limited duration
• Zero average value
• This is in contrast to sinusoids, used by FT,
which have infinite duration and
Sinusoid constant
Wavelet
frequency.
 Types of Wavelets

• There are many different wavelets, for

example:
Haar Morlet Daubechies
Basis Functions Using Wavelets
• Like sin( ) and cos( ) functions in the Fourier
Transform, wavelets can define a set of basis
functions ψk(t):
f (t )  ak k (t )
k

• Span of ψk(t): vector space S containing all

functions f(t) that can be represented by ψk(t).
 Basis Construction – “Mother”
Wavelet
The basis can be constructed by applying translations and
scalings (stretch/compress) on the “mother” wavelet ψ(t):

Example:
scale

ψ(t)

translate
Basis Construction - Mother Wavelet

(dyadic/octave grid)

 jk (t )

scale =1/2j
(1/frequency)
j

k
Continuous Wavelet Transform (CWT)

translation scale parameter scale =1/2j

parameter (measure (measure of frequency) (1/frequency)
of time)

Forward 1 t 
C ( , s )   f t  

 dt
CWT: s t  s 

mother wavelet
normalization (i.e., window
constant function)
 Illustrating CWT
1. Take a wavelet and compare it to a section at the
start of the original signal.
2. Calculate a number, C, that represents how
closely correlated the wavelet is with this section
of the signal. The higher C is, the more the
similarity.

1 t  
C ( , s )   f t    dt
s t  s 
 Illustrating CWT (cont’d)

3. Shift the wavelet to the right and repeat step 2 until

you've covered the whole signal.

1 t 
f t 

C ( , s)    dt
s t  s 
 Illustrating CWT (cont’d)

4. Scale the wavelet and go to step 1.

1 t  
C ( , s)  f t    dt
st  s 

5. Repeat steps 1 through 4 for all scales.

 Visualize CTW Transform
• Wavelet analysis produces a time-scale view of the input signal or
image.

1 t  
C ( , s)  f t    dt
st  s 
 Continuous Wavelet Transform
(cont’d)

1 t 
Forward CWT: C ( , s )   f t  

 dt
s t  s 

1 t
Inverse CWT: f (t ) 
s

 s
C ( , s ) (
s
)d ds

Note the double integral!

Fourier Transform vs Wavelet
Transform
weighted by F(u)
Fourier Transform vs Wavelet
Transform
weighted by C(τ,s)

1 t
f (t ) 
s

 s
C ( , s ) (
s
) d ds
 Properties of Wavelets
• Simultaneous localization in time and scale
- The location of the wavelet allows to explicitly
represent the location of events in time.
- The shape of the wavelet allows to represent
different detail or resolution.
 Properties of Wavelets (cont’d)

• Sparsity: for functions typically found in

practice, many of the coefficients in a wavelet
representation are either zero or very small.
1 t
f (t ) 
s 
 s
C ( , s ) (
s
)d ds
 Properties of Wavelets (cont’d)

1 t
f (t ) 
s 
 s
C ( , s ) (
s
)d ds

• Adaptability: Can represent functions with

discontinuities or corners more efficiently.

• Linear-time complexity: many wavelet

transformations can be accomplished in O(N)
time.
 Audio compression.
 Speech recognition.

 Image and video

compression
 Denoising Signals

 Motion Detection and

tracking
 L-21
WAVELET BASES
AND
MULTI-RESOLUTION ANALYSIS
 Wavelets vs. Fourier Transform
• Fourier transform (FT) represent a signal in terms of
sinusoids
• FT provides a signal which is localized only in the frequency
domain
• No any information of the signal in the time domain
• Basis functions of the wavelet transform (WT) are small
waves located in different times
• Obtained using scaling and translation of a scaling function
and wavelet function
• Therefore, the WT is localized in both time and frequency.
• In addition, the WT provides a multiresolution system
useful in several applications .e.g. image communications
and image data base
57
 Wavelets vs. Fourier Transform
• If a signal has a discontinuity, FT produces many coefficients with large
magnitude (significant coefficients)
• But WT generates a few significant coefficients around the discontinuity
• Nonlinear approximation is a method to benchmark the approximation power of
a transform.
• In nonlinear approximation we keep only a few significant coefficients of a
signal and set the rest to zero
• Then we reconstruct the signal using the significant coefficients
• WT produces a few significant coefficients for the signals with discontinuities
• Thus, better results for WT nonlinear approximation when compared with the
FT
• Most natural signals are smooth with a few discontinuities (are piece-wise
smooth)
• Speech and natural images are such signals
• Hence, WT has better capability for representing these signal when compared
with the FT
58
 Consider a waveform scaled by ‘s’ and shifted by ‘’

t 
 s t   s  
 0.5

 s 
The larger the value of s, the more the waveform is compressed.

The waveform is shifted to the left for positive .

For discrete signal, let s 2 j

t   t  ks 
2     j

  2 t  k  jk
 s   s 
The family has two groups, one generated from a mother and the other
from a father wavelets.
An arbitrary can be transformed into the space formed by members of a
wavelet family

 
t 
W f  , s   s

 0.5
 f t    dt  t dt 0
  s  

An inverse transform is also available.

 
1   0.5  t     dsd 
f t  
C
 
  W f  , s   s   s   s 2

ˆ  
 2

C :  d  


Haar Basis
 Haar expansion is a two-point avarage
and difference operation
 The basis functions are given as
 1 2 , n 2k

2 k [n] 1 2 , n 2k , 2k  1 
2 k 1[n]  1 2 , n 2k  1
 0, otherwise  0, otherwise

 It follows that
2 k [n] 0 [n  2k ], 2 k 1[n] 1[n  2k ]

61
Haar Basis
 The transform is
1
X [2k ]   2 k , x   x[2k ]  x[2k  1],
2
1
X [2k  1]  2 k 1 , x   x[2k ]  x[2k  1]
2
 The reconstruction is obtained from
x[n]   X [k ]k [n]
k Z

62
The Harr father wavelet function is a unit step of length 1. It generates a
family of scaling functions as shown below:

k=0 k=1 k=2 k=3

j=
2

j= k=0 k=1
1

j= k=0 The father

0
0 1
The Harr mother wavelet function is a bipolar [-1,1] unit step of length 0.5.
It generates a family of wavelet functions as shown below:

k=0 k=1 k=2 k=3

j=
2

j= k=0 k=1
1

j= k=0 The mother

0
0 1
 Scaling functions are like Low Pass Filter. Approximating a waveform f(t)
with the set of scaling functions for certain value of ‘j’ gives the coarse
form of f(t) at the resolution denoted by ‘j’.

f0(t)

f(t)

0 1 2

0 1 2 f1(t)

‘j’ must be large enough to capture all

the details in f(t).
0 1 2

 

f j t    c j k  jk t    c j k 2 j / 2  2 j t  k 
k   k  
Pure use of scaling function to approximate a waveform is viable but not
efficient. Very often the scale has to be very fine to capture all the
details, resulting in many coefficients.

f j t   S j
The wavelet functions are like a set of high pass filter which captures the
difference between two resolutions (i.e. two values of j). Let Sj represent
the space corresponding to scale ‘j’,

 
f j t    c j k  jk t   d k  t 
j jk
k   k  
 
  c j k 2 j/2
 2 t  k 
j
 j
d k 2 j/2
 
2 j
t k 
k   k  
In another words, a high resolution subspace at level ‘j’ can be formed
by combining a lower resolution subspaces at ‘j-1’, i.e.,

S j  S j  1  W j  1  S j  2  W j  2  W j  1
 S 0  W 0  W1  ...W j  1

We have

 M 1 
f M t    c0 k  0 k t    d k  t 
j jk
k   j 0 k  
 M 1 
  c0 k 2 j/2
 2 t  k  
j
 j
d k 2 j/2
 
2 j
t k 
k   j 0 k  
An important feature of scaling functions: Each can be derived from
translation of double-frequency copies of itself, as


 t    2 h0 k  2 t  k 
k  

Similarly,


 2 t    2 h0 k  4 t  k 
k  

It means that a scaling function can be built from higher frequency

(resolution) replicas of itself.

This is known as Multiresolution analysis.

Similar important feature of wavelet functions: Each can be derived
from translation of double-frequency copies of scaling function, as


 t    2 h1 k  2 t  k 
k  

It means that a scaling function can be derived from the father wavelet!

The father wavelet determines the characteristics of all members of the

wavelet family.

h0 k  : scaling function coefficien ts

h1 k  : wavelet function coefficien ts
  t 
1

t  t    2 h0 k  2 t  k 
1 k  
 2 t 
1  2 h 0 0  2 t  2 h 0 1 2 t  k 

t
0.5 1
 2 t  1  By inspection,
1
1
h0 0  h0 1 
t
2
0.5 1
  t 
1


1
t  t    2 h1 k  2 t  k 
k  

-1
 2 t   2 h1 0  2 t  2 h1 1  2 t  k 
1

t By inspection,
0.5 1
 2 t  1 
1 1
h1 0   , h1 1 
1 2 2

t
0.5 1
 
c j k    c m h m  2 k 
j 1 0
m  


d j k    c m h m  2 k 
j 1 1
m  

alternatively, an inverse relation is also available, as

 
c j 1 k    c m h k  2 m    d m h k  2 m 
j 0 j 1
m   m  
 LP 2 c j k 
LP  h0  k 
c j 1 k 
HP  h1  k 
HP 2 d j k 

c j k  2 LP
LP h0 k 
c j 1 k  HP h1 k 

d j k  2 HP
 c j  2 k 
LP 2
c j  1 k 
LP 2
c j k 
LP 2 HP 2
c j 1 k 
HP 2
d j  2 k 

HP 2 d j  1 k 

d j k 

The signal samples, scaled down by 2j/2, is always used as the first
set of coefficients cj[k].
The above decomposition only contains relations between the terms
‘c’ and ‘d’, so where is the signal f(t)?

First, cj+1 is one resolution level higher than cj.

Next for a digital signal, as the level advances, it will ultimately

reaches a maximum resolution limited by the sampling lattice, lets call
this cmax.

Obviously, cmax is simply the signal itself.

An interesting point. The father and mother wavelets established the

formulation, but absent in the decomposition and reconstruction of
the signal. It acts like a catalysis!
 cols rows
N N
N HP 2  Diagonal HH
rows cols 2 2
N
HP 2 2
cols rows
N N
LP 2  Vertical HL
2 2
N N
cols rows
N N
HP 2  Horizontal LH
rows cols N  N 2 2
2
LP 2
cols rows
N N
LP 2  Low pass LL
2 2
 N N
 rows
2 2 cols
diagonal 2 HP
cols rows
N N
 rows cols + 2 HP
2 2
vertical 2 LP

N N N N
 rows
N
2 2 cols N
2
horizontal 2 HP
cols rows
N N
 rows + 2 LP
2 2 cols
Low pass 2 LP
 N N
 rows cols
2 2
diagonal 2 HP
cols rows
N N
 rows cols + 2 HP
2 2
vertical 2 LP

N N N N
 N
2 2 rows cols N
2
horizontal 2 HP
cols rows
N N
 rows cols + 2 LP
2 2
Low pass 2 LP
 cols rows
N N

N HP 2 Diagonal HH
rows cols 2 2
N
2
HP 2
cols rows
N N

LP 2 Vertical HL
2 2
N N

cols
rows
N N

N HP 2 Horizontal LH
2 2
rows cols N 
2
LP 2
cols
rows
N N

Low pass LL
LP 2
2 2
 L-22

Wavelets and Sub-

band filter banks
 Filter banks
• The lower resolution coefficients can be
calculated from the higher resolution
coefficients by a tree-structured algorithm
(i.e., filter bank).
Subband
encoding
(analysis)

h0(-n) is a lowpass filter and h1(-n) is a highpass filter

 Filter banks (cont’d)
Next level:

h0(-n) is a lowpass filter and h1(-n) is a highpass filter