Linear filtering

Motivation: Noise reduction

Given a camera and a still scene, how can you
reduce noise?

Take lots of images and average them!

What’s the next best thing?
Source: S. Seitz
Moving average
• Let’s replace each pixel with a weighted
average of its neighborhood
• The weights are called the filter kernel
• What are the weights for the average of a
3x3 neighborhood?

1 1 1
1 1 1
1 1 1

“box filter”
Source: D. Lowe
Defining convolution
• Let f be the image and g be the kernel. The
output of convolving f with g is denoted f * g.

( f  g )[m, n]  f [m  k , n  l ] g[k , l ]
k ,l

• Convention: kernel is “flipped”

• MATLAB: conv2 vs. filter2 (also imfilter)
Source: F. Durand
Key properties
• Linearity: filter(f1 + f2 ) = filter(f1) + filter(f2)
• Shift invariance: same behavior regardless of
pixel location: filter(shift(f)) = shift(filter(f))
• Theoretical result: any linear shift-invariant
operator can be represented as a convolution
 Properties in more detail
• Commutative: a * b = b * a
• Conceptually no difference between filter and signal
• Associative: a * (b * c) = (a * b) * c
• Often apply several filters one after another: (((a * b1) * b2) * b3)
• This is equivalent to applying one filter: a * (b1 * b2 * b3)

• Distributes over addition: a * (b + c) = (a * b) + (a * c)

• Scalars factor out: ka * b = a * kb = k (a * b)
• Identity: unit impulse e = […, 0, 0, 1, 0, 0, …],
a*e=a
 Annoying details
What is the size of the output?
• MATLAB: filter2(g, f, shape)
• shape = ‘full’: output size is sum of sizes of f and g
• shape = ‘same’: output size is same as f
• shape = ‘valid’: output size is difference of sizes of f and g

full same valid

g g g g
g g

f f f

g g g
g g g
Annoying details
What about near the edge?
• the filter window falls off the edge of the image
• need to extrapolate
• methods:
– clip filter (black)
– wrap around
– copy edge
– reflect across edge

Source: S. Marschner
Annoying details
What about near the edge?
• the filter window falls off the edge of the image
• need to extrapolate
• methods (MATLAB):
– clip filter (black): imfilter(f, g, 0)
– wrap around: imfilter(f, g, ‘circular’)
– copy edge: imfilter(f, g, ‘replicate’)
– reflect across edge: imfilter(f, g, ‘symmetric’)

Source: S. Marschner
Practice with linear filters

0 0 0
0 1 0 ?
0 0 0

Original

Source: D. Lowe
Practice with linear filters

0 0 0
0 1 0
0 0 0

Original Filtered
(no change)

Source: D. Lowe
Practice with linear filters

0 0 0
0 0 1 ?
0 0 0

Original

Source: D. Lowe
Practice with linear filters

0 0 0
0 0 1
0 0 0

Original Shifted left

By 1 pixel

Source: D. Lowe
Practice with linear filters

?
1 1 1
1 1 1
1 1 1

Original

Source: D. Lowe
Practice with linear filters

1 1 1
1 1 1
1 1 1

Original Blur (with a

box filter)

Source: D. Lowe
Practice with linear filters

0 0 0 1 1 1
0 2 0
0 0 0
- 1 1 1
1 1 1
?
(Note that filter sums to 1)
Original

Source: D. Lowe
Practice with linear filters

0 0 0 1 1 1
0 2 0
0 0 0
- 1 1 1
1 1 1

Original Sharpening filter

- Accentuates differences
with local average

Source: D. Lowe
Sharpening

Source: D. Lowe
Smoothing with box filter revisited
• Smoothing with an average actually doesn’t compare
at all well with a defocused lens
• Most obvious difference is that a single point of light
viewed in a defocused lens looks like a fuzzy blob; but
the averaging process would give a little square

Source: D. Forsyth
Smoothing with box filter revisited
• Smoothing with an average actually doesn’t compare
at all well with a defocused lens
• Most obvious difference is that a single point of light
viewed in a defocused lens looks like a fuzzy blob; but
the averaging process would give a little square
• Better idea: to eliminate edge effects, weight
contribution of neighborhood pixels according to their
closeness to the center, like so:

“fuzzy blob”
 Gaussian Kernel

0.003 0.013 0.022 0.013 0.003

0.013 0.059 0.097 0.059 0.013
0.022 0.097 0.159 0.097 0.022
0.013 0.059 0.097 0.059 0.013
0.003 0.013 0.022 0.013 0.003

5 x 5,  = 1

• Constant factor at front makes volume sum to 1 (can be

ignored, as we should re-normalize weights to sum to 1 in
any case)
Source: C. Rasmussen
Choosing kernel width
• Gaussian filters have infinite support, but
discrete filters use finite kernels

Source: K. Grauman
Choosing kernel width
• Rule of thumb: set filter half-width to about
3σ
Example: Smoothing with a Gaussian
Mean vs. Gaussian filtering
Gaussian filters
• Remove “high-frequency” components from
the image (low-pass filter)
• Convolution with self is another Gaussian
• So can smooth with small-width kernel, repeat, and get
same result as larger-width kernel would have
• Convolving two times with Gaussian kernel of width σ is
same as convolving once with kernel of width σ√2
• Separable kernel
• Factors into product of two 1D Gaussians

Source: K. Grauman
Separability of the Gaussian filter

Source: D. Lowe
Separability example

2D convolution
(center location only)

The filter factors

into a product of 1D
filters:

Perform convolution
along rows: * =

Followed by convolution =
along the remaining column: *

Source: K. Grauman
Separability
• Why is separability useful in practice?
Review: Linear filtering
• Properties of convolution
• Properties of Gaussian kernels
Noise
• Salt and pepper
noise: contains
random occurrences
of black and white
pixels
• Impulse noise:
contains random
occurrences of white
pixels
• Gaussian noise:
variations in
intensity drawn from
a Gaussian normal
distribution

Source: S. Seitz
Gaussian noise
• Mathematical model: sum of many
independent factors
• Good for small standard deviations
• Assumption: independent, zero-mean noise

Source: M. Hebert
 Reducing Gaussian noise

Smoothing with larger standard deviations suppresses noise,

but also blurs the image
Reducing salt-and-pepper noise
3x3 5x5 7x7

What’s wrong with the results?

Alternative idea: Median filtering
• A median filter operates over a window by
selecting the median intensity in the window

• Is median filtering linear?

Source: K. Grauman
Median filter
• What advantage does median filtering have
over Gaussian filtering?
• Robustness to outliers

Source: K. Grauman
Median filter
Salt-and-pepper noise Median filtered

MATLAB: medfilt2(image, [h w])

Source: M. Hebert
 Median vs. Gaussian filtering
3x3 5x5 7x7

Gaussian

Median
Sharpening revisited
What does blurring take away?

– =

original smoothed (5x5) detail

Let’s add it back:

+α =

original detail sharpened

Unsharp mask filter
f   ( f  f  g ) (1   ) f   f  g  f  ((1   )e  g )

image blurred unit impulse

image (identity)

unit impulse
Gaussian Laplacian of Gaussian
Application: Hybrid Images

A. Oliva, A. Torralba, P.G. Schyns,

“Hybrid Images,” SIGGRAPH 2006
Application: Hybrid Images

A. Oliva, A. Torralba, P.G. Schyns,

“Hybrid Images,” SIGGRAPH 2006