CSE483 Computer Vision

Image Enhancement in the Spatial Domain

Prof. Mahmoud Khalil

Spring 2022
1
Image Enhancement in the Spatial Domain
Image Enhancement in the Spatial Domain
 We pass the filter on every pixel in the
original image. Also, use the original image
in every iteration. Note how the nature of
this algorithm (always iterating over the
Convolution Example – Step 1 original image) makes it parallelizable!
Check tutorial for more talk about "convolution"
and "correlation" in case Prof. asks about either.. 1 1 1 2 2 2 3
Mathematically speaking, what we do is actually called "correlation" but
we call it convolution in the examples for whatever reason... So if you're -1 2 1 2 1 3 3
asked to do both at some point, you should know how to. The difference
is basically that you rotate the filter 180 degrees in the other method.
-1 -1 1 2 2 1 2
h 1 3 2 2
1 1 1
-1 2
4 2 2 3 5
-1 2
-2 1 3 3
2 2 1 2
f*h
1 3 2 2
f 4
Convolution Example – Step 2
1 1 1 2 2 2 3
-1 2 1 2 1 3 3
-1 -1 1 2 2 1 2
h
1 3 2 2
1 1 1
2
-2 2
4 2 3 5 4
2
-2 1
-1 3 3
2 2 1 2
f*h
1 3 2 2
f 5
Convolution Example – Step 3
1 1 1 2 2 2 3
-1 2 1 2 1 3 3
-1 -1 1 2 2 1 2
h
1 3 2 2
1 1 1
2 2
-2 2
4 3
5 4 4
2 1
-1 3
-3 3
2 2 1 2
1 3 2 2
f
f*h 6
Convolution Example – Step 4
1 1 1 2 2 2 3
-1 2 1 2 1 3 3
-1 -1 1 2 2 1 2
h
1 3 2 2
1 1 1
2 2 2
-2 3
6 1 5 4 4 -2
2 1 3
-3 3
-3 1
2 2 1 2
1 3 2 2
f f*h 7
Convolution Example – Step 5
1 1 1 2 2 2 3
-1 2 1 2 1 3 3
-1 -1 1 2 2 1 2
1 3 2 2
h

1 2 2 2 3 5 4 4 -2
-1 4
2 1 3 3 9
-1 -2
2 2 1 2
1 3 2 2
f f*h 8
Convolution Example – Step6
1 1 1 2 2 2 3
-1 2 1 2 1 3 3
-1 -1 1 2 2 1 2

h 1 3 2 2

2 2 2 3
5 4 4 -2
-2
2 2
1 3 3
9 6
2
-2 2
-2 1 2
1 3 2 2

f
f*h 9
 Image Enhancement in the Spatial Domain
These two are called many names; the
Averaging/Smoothing/Blurring Filters
Extra:
Aside from the local-favoring
benefit of the filter to the right,
it is also better because it uses
multiples of two, and this is a
lot easier for the CPU to do,
cause all it needs to do is shift
left or right when doing
multiplication or division,
where the weights will need
multiplications by 2 and their
Gaussian divisor will need divisions by 2.

- Why should we divide by the sum of the weights (i.e., make the result sum to 1)? Because if I have an all 1s
window, the resulting pixel should also be 1. Otherwise, the filter would be manipulating the picture wrongly.
For example, average grade of three students who got 60 can't end up being something other than 60... lol.

- The filter to the right focuses more on the pixel that we're convolving at, in larger size filters this is more
10

effective, and the general reason is to favor the effect of local/nearby pixels since farther pixels are most
likely unrelated and would thus leave a stronger blur that feels much more different than the image.
 Here's a test on a sample synthetic data. Synthetic data has the advantage of us including
as many corner cases as possible in one scenario, but has the inherent disadvantage of not
really being a correct simulation of real-life data that we should actually be testing against.

Image Enhancement in the Spatial Domain

What are the hyper-parameters of this filter?

- Before I answer, what are hyper-parameters in the

first place? They are the parameters that influence
the output upon changing. Usually we aim to see
the effect of increasing them to the maximum or
decreasing them to the minimum to get a general
understanding of their effect.
- For example, the volume of radio. Lowest means
mute, highest means too loud.
- Ok so what about the hyper-parameters in here?
1. Filter size (3x3, 5x5, etc...)
2. Weights distribution (pure avg, weight avg, etc...)
3. Number of applications
- For example, to go from image 1 to image 6 here,
you could apply multiple times, or increase the size.
- What about the and max. hyper-parameter? If you
apply it infinite no. of times, you should expect to
get the average of all existing pixels reflecting the
weight distribution (so if it is a pure avg. filter, you 11

get the average of them all... but what about if it's

the weighted filter? Needs research)
 When question has numbers, your answer
must include numbers. When given symbols,
you must use them. When given just text,
Problem 3.21 you should only respond in similar text.

The three images shown were blurred using square averaging

masks of sizes n = 23,25, and 45, respectively. The vertical bars
on the left lower part of (a) and (c) are blurred, but a clear
separation exists between them. However, the bars have
merged in image (b), in spite of the fact that the mask that
produced this image is significantly smaller than the mask that
produced image (c). Explain this.
When its size was 25x25, in (b), the horizontal row of vertical bars got totally
smoothed into some continuous solid shape. From that, we can include that
passing the filter throughout this area always gave the same result. The only
way for that to happen if the sum of pixels in every iteration was the same.
The only way for that to happen is if these bars were of a width X and
separated by a gap Y such that X+Y = 25, say if they are 20 and 5, then at the
first bar I'll have 20 blacks and 5 whites. 1 pixel to the right I'd have 19+1
blacks and 5 whites. So on so forth.

12
 Image Enhancement in the Spatial Domain

13
Discuss what happens if you multiply (c) by (a). The result is that the bigger stars or galaxies will
stay in the picture while the rest will disappear. So, Avg. --> Threshold --> Multiply == Eliminate.
Important: The gray-scale range is 0...255
Random: If I tell you I have a q*q white block, this means
that it is totally surrounded by black blocks, because if it's
not, then I'm lying about it being q*q... (if there's a layer
Problem 3.22 around it of white, then it'd be (q+1)*(q+1), not q*q)

Consider an application such as the one shown in Fig. 3.34,

in which it is desired to eliminate objects smaller than those
enclosed in a square of size q X q pixels.
Suppose that we want to reduce the average gray level of
those objects to one-tenth of their original average gray level.
In this way, those objects will be closer to the gray level of the
If you see "Give", background and they can then be eliminated by thresholding.
it means no need Give the (odd) size of the smallest averaging mask that will
for proof...
accomplish the desired reduction in average gray level in only
one pass of the mask over the image.
- To reduce the pixels to 1/10th of their value, you'll need your filter size to be
equal to some value such that the window has 10*q*q pixels overall; such that
q*q of them would be the foreground color and the rest background, thus
giving a ratio of 1/10 upon summing then dividing by the total (that is 10*q*q).
14

- Another approach of solving it is: Let the filter size be S (i.e., S*S pixels). The
result of one pixel would be 255*q*q / S*S. Let that = 255*1/10.
Then S = q*sqrt(10). Take its ceil to the nearest odd value, ggwpez
 Example: T

00011111000
00000100000
00000000000
Problem 3.20 00000100000

(a)In a character recognition application, text pages are reduced to binary

form using a thresholding transformation function of the form shown in
Fig. 3.2(b). This is followed by a procedure that thins the characters until
they become strings of binary 1's on a background of 0's. Due to noise, the
binarization and thinning processes result in broken strings of characters
with gaps ranging from 1 to 3 pixels. One way to "repair" the gaps is to run
an averaging mask over the binary image to blur it, and thus create bridges
of nonzero pixels between gaps. Give the (odd) size of the smallest
averaging mask capable of performing this task. ASuppose the worst case; the center pixel of a 3-pixel-gap.
3x3 filter won't work, it will be empty and won't close the
gap (in one iteration). A 5x5 will work in one iteration though.

(b) After bridging the gaps, it is desired to threshold the image in order to
convert it back to binary form. For your answer in (a), what is the
minimum value of the threshold required to accomplish this, without
causing the segments to break up again? Again going with that worst case, if I had a gap of 3,
then applying the filter on the center pixel means there
15
are only 2 foreground pixels and the remaining 5x5
pixels are all empty (three of them gaps, the rest
Important: The gray-scale range is 0...255 backgound), and so just less than 2(255)/25 should be
If not mentioned, make sure to assume it! the threshold, anything below it is not an averaged-gap.
 So far the filters we saw are:
1. Averaging/Smoothing/Blur
2. Statistical (Median/Max/Min)

Order‐Statistics Filters
 Median Filter

 Maximum Filter

 Minimum Filter

16
 Order‐Statistics Filters

- Why does it work? Because in the normal image, the probability of having a max-wight or a max-black as part of the image itself (the salt and 17
pepper) is not that high, so when you do median filter it puts these very large/small values at either end of the range, thus eliminating them.

- When would it fail? If the number of salt/pepper noise pixels covers ceil( n^2 ) or more (where n = filter size), then one of them will surely be
the median. Note however that they have to be of the same type (i.e., ceil(n^2) of salt or ceil(n^2) of pepper)) for this condition to hold.
 Remember, in averaging filter we add stuff up - similar to integration. So far the filters we saw are:
Hence in sharpening, we do differentiation - take differences! 1. Averaging/Smoothing/Blur
2. Statistical (Median/Max/Min)
3. Sharpening
Sharpening Spatial Filters The opposite of averaging/smoothing/blurring

This is the response of this slice.

The gray level profile

should be from 0 to 255,
but here to simplify, he
made the range up to 7.

Derivative1: Take right - this, 18

or left - this, just stick to one.

Notice how 2nd derivative gives a lot
Derivative2: ( r - t ) - ( t - l ) more edges than there actually is!
 Sharpening Spatial Filters – Laplacian Filter
there you go; just like
you deduced last slide
 First derivative right - this
(r - t) - (t - l)
 Second derivative =r-t-t+l
= r + l - 2t

continuous to discrete = loss in accuracy

1st Order Derivative = Gaussian, local maxima/minima

2nd Order Derivative = Laplacian, zero-crossings
19

There's another form of derivatives which is (right minus left),

which would give us: f(x+1, y) - f(x-1, y) + f(x, y+1) - f(x, y-1)
In averaging, the output had a range of 0 to 255. In here, the output won't be so due
Again, on averaging: If you're applying the filter on a plain
to the non-existence of the divisor. For example, filter mask (a) has -255*4 to 255*4
area (totally smooth) the result should be unchanged. This is
(-255*4 if it's all zeroes except the center, 255*4 if only the ones are 255s). However,
the condition for smoothing. Sharpening has a condition too,
we can only plot the range 0 to 255. So, we can divide by 8 and add 127 to convert it.
that is the total of the weights must be zero. Because if given

Sharpening Spatial Filters a picture with no transitions at all (all smooth), the result
should be an empty image - no "edges" of transitions.

20
 Since the operations are linear, we can do
sharpening in one step instead of these two:
Step 1: Use identity filter
Sharpening Spatial Filters Step 2: Use sharpen filter and add on top of
identity filter's result.
Merge the steps: Add the two filters!

Identity filter is a filter

of all zeroes except in
the center, a 1.

21
Image Enhancement in the Spatial Domain
And this would
be the result:

24
(a) is original. (b) is Laplace filter of (a), made into 0 to 255 range by dividing and taking absolute value.
(c) is Laplace filter of (a), made into 0 to 255 range by dividing and adding. It now shows edges directions!

Image Enhancement in the Spatial Domain

22
Problem 3.23

In a given application an averaging mask is

applied to input images to reduce noise, and then
a Laplacian mask is applied to enhance small
details. Would the result be the same if the order
of these operations were reversed?
Any filter where you can just add up the product of every pixel with its corresponding
weight in the filter, then divide at the end or not, is considered linear. Hence, the
result would be the same here since both filters are indeed linear. Consider the
Laplacian filter that had -1s everywhere except center. Add it to the averaging filter,
you get the identity filter. This means operation reversed successfully, and direction
doesn't matter cause linear. However, the Laplacian filter that had zeroes in its
corners won't get you back to identity, though still the direction doesn't matter.

Extra: The statistical filters are non-linear, they don't work with the same idea of 23

giving weights in a filter and adding them up.

Extra Extra: Fourier Transform can be done linearly.

Image Enhancement in the Spatial Domain

Also called "Unsharp"

Previously we used A = 1 for normal

sharpening. Increasing it probably
emphasizes the details of the
original picture or something?

25
This is a random example filter;
it shifts images to the right 1px.
 Image Enhancement in the Spatial Domain
m is the hyper-parameter here

This transformation increases the range of the middle

part in the original image, say if it were from 55 to This transformation does the same thing but very 26
200, that range is expanded to 5 to 250 for example, abruptly, such that everything before m turns to black
and anything below 55 is mapped to below 5 in the and everything after m turns to white in the output.
output, likewise 200+ --> 250+, which means the
middle region now has more contrast
 Image Enhancement in the Spatial Domain

Identity: In = Out
Negative: Swaps black w/ white
Inverse log: As you can see, it
takes a large region of the input
and maps it into a small region
of the output's dark part, hence
the output is darkened input
nth power: Same, but weaker
nth root: Opposite of nth power 27

Log: Opposite of Inverse log

 "Gamma Correction" Eqn
Has everything from before
and more, except for the
Negative one.

Lighten
Output = Const. * Input Gamma

So, two hyper-parameters, the

constant and the gamma, but
here we assume the constant is
constantly (hehe) = 1, so only
Darken one hyper-parameter lives on.

28
Gamma Correction

29
Gamma Correction

30
 So far we've used global filter techniques. This is bad. Local processing provides much more
robust and practical results, global is very rarely expected to give good practical results.

Contrast Stretching
This filter takes the input region
from 3L/8 to 5L/8 and expands it
onto L/8 to 7L/8, i.e., increasing the
contrast in that specific region of
the input image.

As for the regions after and before,

it darkens the first and lightens the
second, thus, overall it enhances
the contrast, darkens the darker
areas, enlightens the lit areas, and
increases the contrast of the area
in between them.

The red line (I drew) is the identity,

and shows indeed that the dark
areas got darker, light areas got
31

lighter, while the middle part's

contrast increased.
 Image Enhancement in the Spatial Domain

The filter creator supposed

cars would always be in the In the other filter, he just
region A to B, so in filter (a) highlights A to B with the
he simply makes everything constant bright gray but 32
else black and makes the preserves the other levels
area from A to B a constant as they are in the input.
bright gray value.
Histogram Equalization

An algorithm to equalize the

histogram would be to get the
original picture's histogram,
33
get
a transformation function that
maps high intensities to lower
ranges and vice versa, multiply
them, and tadaaaa
Image Enhancement in the Spatial Domain

34