EENG 860 Special Topics: Digital Image Processing

Lecture 3: Histogram Processing and

Fundamentals of Spatial Filtering

Dr. Ahmadreza Baghaie

Department of Electrical and Computer Engineering
New York Institute of Technology

Spring 2020

Readings: Chapter 3 (sections 3.3 and 3.4)

1 / 41
How to read: at least twice, once laid back, once with full focus.
Table of Content

●
Histogram Processing
●
Histogram Equalization
●
Histogram Matching (Specification)
●
Local Histogram Processing
●
Fundamentals of Spatial Filtering
●
Correlation vs. Convolution
●
Separable Kernel Filters

2 / 41
Histogram Processing

● Let rk, for k=0,1,…,L-1, represent the intensities of an L-level image f(x,y).
●
The unnormalized histogram of f is defined as
h(r k )=n k , for k=0,1,2 ,... , L−1
where nk is the number of pixels of f with intensity rk.

●
The subdivisions of the intensity scale are called histogram bins.
●
The normalized histogram of f is defined as

h(r k ) nk
p (r k )= =
MN MN
where as usual, M and N are the number of image rows and columns.

3 / 41
Histogram Processing

● The sum of p(rk) for all values of k is always 1:

L−1

∑ p (r k )=1
k=0

● The components of p(rk) are the probabilities of intensity levels in the

image.
●
Histograms are very good indicators of the image contrast:
– Low-contrast: histogram with narrow intensity range;

– High-contrast: histogram with broad intensity range

4 / 41
Histogram Processing

●
How to enhance the contrast in an image? Since a more even distribution
in the histogram can be linked to higher contrast, it is natural to assume if a
histogram is expanded to cover the full range of intensity values, the
contrast is improved.
●
In an ideal image, every intensity level corresponds to the same number of
pixels. Histogram equalization aims to do this.

5 / 41
Histogram Equalization: Continuous Case

●
Assume a continuous range of intensity values, r in the range [0, L-1].
●
Let’s have a transformation (intensity mapping) function:
s=T (r) 0≤r≤L−1
●
Function conditions:
a) T(r) is a monotonic increasing function in the interval [0,L-1];
b) 0 ≤ T(r) ≤ L-1 for 0 ≤ r ≤ L-1.
• Function T is one-to-one and reversible if it is strictly monotonic (a’).

6 / 41
Histogram Equalization: Continuous Case

●
Condition (a) guarantees that output intensity values will never be less
than the corresponding input values, preventing artifacts and intensity
reversals.
●
Condition (b) guarantees that the range of the output intensities is the
same as the input image.
●
Condition (a’) guarantees that the mapping function from r to s is one-to-
one and reversible.

7 / 41
Histogram Equalization: Continuous Case

● Let pr(r) and ps(s) be the Probability Distribution Functions (PDFs) on

intensity values in the two images.
● From probability theory, if pr(r) and T(r) are known, and if T(r) is
continuous and differentiable over the range of the values, then the PDF of
the transformed variable s is:
dr
p s (s)= p r (r)| |
ds
●
This is too general. Let’s assume a more specific function in DIP:
r

s=T (r)=( L−1)∫ p r (w) dw

0
where w is the dummy integration variable.
●
The integral on the right is the Cumulative Distribution Function (CDF).
8 / 41
Histogram Equalization: Continuous Case

●
Let’s assume a more specific function in DIP:
r

s=T (r)=( L−1)∫ p r (w) dw

0

where w is the dummy integration variable.

●
Since PDFs are always positive, and the integral is the area under the
function, the transformation function is monotonic and increasing,
satisfying condition (a); Not necessarily condition (a’)
●
Also, for the lower limit r=0, we have s=0 and for the upper limit r=L-1, the
integral adds up to 1, which makes s=L-1. This satisfies condition (b)
which states the range of the output intensities should be the same as
input intensities.
●
Now let’s drive the PDF of s.

9 / 41
Histogram Equalization: Continuous Case

[ ]
r
ds dT (r) d
dr
=
dr
=( L−1)
dr
∫ pr (w)dw =( L−1) pr (r)
0

● Therefore, ps(s) can be derived as:

dr 1 1
p s (s)= p r (r)| |= pr (r)| |= for 0≤s≤L−1
ds ( L−1) p r (r) L−1
which is a uniform distribution!

Equalization

10 / 41
 Histogram Equalization: Continuous Case

 2r
●
An example:
 0  r  L 1
pr (r )   ( L  1) 2
0 else

●
Then we have:
r r r
2w 2 r2
s=T (r)=(L−1)∫ Pr (w)dw=(L−1)∫ 2
dw = ∫ w dw=
0 0 ( L−1) L−1 0 L−1
●
Therefore:
ds 2r dr L  1
 T (r ) 
'

dr L 1 ds 2r
dr 2r L 1 1
●
And finally: p s ( s )  pr ( r )  
ds ( L  1) 2r
2
L 1
11 / 41
Histogram Equalization: Discrete Case

●
For discrete values, we have probabilities and summations, instead of
probability densities and integrals.
●
We defined the normalized histogram for an image of size M-by-N and for
nk as the number of pixels that have intensity rk as:
nk
p r (r k )=
MN
●
The discrete transformation can be defined as:
k k
L−1
s k =T (r k )=(L−1) ∑ p r (r j )= ∑ n j , for k=0,1,2 ,... , L−1
j=0 MN j=0
where L is the number of intensity levels.
●
The procedure is called a histogram equalization or histogram
linearization transform.

12 / 41
Histogram Equalization: Discrete Case

●
Example: a 64-by-64, 3-bit (L=8) image,
with the provided histogram.
● Let’s define sk as:
k
sk =( L−1) ∑ pr (r j ), for k =0,1 , ... , L−1
j=0

●
We will have:

s0 =7 p r (r 0 )=1.33
s1=7 ( p r (r 0 )+ p r (r 1 ))=3.08
s2 =7( p r (r 0 )+ p r (r 1 )+ p r (r 2 ))=4.55
s3 =...

13 / 41
Histogram Equalization: Discrete Case

●
Fractional s values need to be s0  1.33  1 s4  6.23  6
rounded to the closest integer value in s1  3.08  3 s5  6.65  7
the range [0,7].
s2  4.55  5 s6  6.86  7
s3  5.67  6 s7  7.00  7

14 / 41
Histogram Equalization: Discrete Case

●
Because a histogram is an approximate PDF, and no new intensity values
are created in the process, perfectly flat histograms are rare in practical
applications.

●
Unlike the continuous case, it cannot be proved than the discrete
histogram equalization results in a uniform histogram.

●
However, histogram equalization is fully automatic, without the need for
any parameter specification.

●
The inverse transformation from s back to r is:

r k =T −1 (s k )
●
This transformation satisfies two initial conditions, if all intensity levels are
present in the input image. Or in another word, none of the histogram bins
of the image are empty.
15 / 41
Histogram Equalization: Discrete Case

●
Advantage: no parameters needed.

●
Conceptually optimal

16 / 41
Histogram Matching (Specification)

●
Histogram equalization produces a transformation function to generate an
output image with a uniform histogram.

●
This is not always ideal. What if we want the specify the shape ourselves?

●
The method to achieve this is called histogram matching or histogram
specification.

●
Let’s assume the continuous case again.

● Assume an image with PDF pr(r) that we want to transform to an image

with PDF pz(z).

● Here pz(z) is not uniform.

17 / 41
Histogram Matching: Continuous Case

●
Let s be a random variable defined as before:
r

s=T (r)=( L−1)∫ p r (w) dw

0
where w is the dummy variable of integration.

●
Let’s do the same based on variable z:
z

G( z)=(L−1)∫ p z (v)dv=s
0
where v is the dummy variable of integration.

●
Combining the two equations, we have:
G( z)=s=T (r)⇒ z=G−1 (s)=G−1 [T (r)]

18 / 41
Histogram Matching: Continuous Case

●
The procedure can be summarized as:

1) Obtain pr(r) from the input image;

2) Use the specified PDF, pz(z), to obtain function G(z);

3) Compute the inverse transformation z=G-1(s), which is a mapping

from s to z.

4) Obtain the output image by first equalizing the input (r to s). Then,
for each pixel s in the equalized image perform the inverse
mapping z=G-1(s) to obtain the corresponding pixel in the output
image.
●
In general, finding the analytical expression for G-1 is not a trivial task.
In the discrete case this is not a problem.
19 / 41
Histogram Matching: Discrete Case

●
In the discrete case, first histogram equalization transformation from r to s
k
is performed:
sk =T (r k )=( L−1) ∑ pr (r j ) for k =0,1,2 ,... , L−1
j=0
● Similarly given a specific value of sk, for a value of q we find G(zq) such
q
that:
G(z q )=( L−1) ∑ p z ( z i )=sk
i=0
where pz(zi) is the i’th value of the specified histogram.
● The desired value of zq is computed from the inverse transform:
z q =G −1 (sk )
●
In practice, no need to compute the inverse of G. Since the intensity levels
are integers, it is simple to compute all the possible values of G using the
second equation for q=0,1,2,….,L-1. These values are rounded to their
nearest integer value and stored in a lookup table. Given a particular value
of sk, we look for the closest match in the table.
20 / 41
Histogram Matching: Discrete Case

●
To summarize:
1) Compute histogram pr(r) of the input image, and then histogram-
equalize it to the uniform histogram. Round the resulting values, sk, to
the integer range [0,L-1];
2) Compute all values of function G(zq) for q=0,1,2,…,L-1, round the
values to integers in range [L-1], and store them in a lookup table.
3) For every value of sk, k=0,1,2,…,L-1, use the stored value of G from
Step 2 to find the corresponding value of zq so that G(zq) is closest to
sk. When more than one value of zq gives the same match, choose the
smallest value by convention.
4) Form the histogram-specified image by mapping every equalized pixel
with value sk to the corresponding pixel with value zq in the histogram-
specified image, using the mappings found in Step 3.

21 / 41
Histogram Matching (Specification)

●
Example: same 64-by-64, 3-bit
(L=8) image as the previous
example.

22 / 41
Histogram Matching (Specification)

●
The first step is to compute the rounded histogram-equalized values, as
we did in the previous example: s0  1.33  1 s4  6.23  6
s1  3.08  3 s5  6.65  7
s2  4.55  5 s6  6.86  7
s3  5.67  6 s7  7.00  7
● Then, we compute the values of G(zq) using the values of pz(zq):

●
These values are then rounded to their nearest integer values:

23 / 41
Histogram Matching (Specification)

●
The first three values of G are equal, so it is not
strictly monotonic, therefore condition (a’) is
violated.
●
Given this, we follow step three of the histogram
matching approach which states we need to find
the smallest value of zq so that the value G(zq) is
the closest to sk. This is done for every value of
sk to create the required mapping from s to z.
● For example, for s0=1, we have G(z3)=1 which is
a perfect match, which means that every pixel in
the histogram-equalized image with value of 1, is
mapped to a pixel with value 3 in the histogram-
specified image.

24 / 41
Histogram Matching (Specification)

●
The final step is to use the computed mappings to
map every pixel in the histogram-equalized image
into the corresponding pixel in the newly created
histogram-specified image.
●
The final result does not match the specified
histogram exactly, however the general trend was
achieved.
●
Obtaining the histogram-equalized image is not
necessary. We can list the mappings from r to s,
and from s to z, and then use these mappings to
map the original pixels directly into the pixels of
the histogram-specified image.

25 / 41
Histogram Matching vs. Histogram Equalization

• Left:
1) Input image
2) Initial Histogram

• Middle
1) Histogram equalization result
2) Histogram equalization
transformation
3) Histogram of histogram
equalized image

• Right
1) Histogram specification result
2) Specified histogram
3) Transformation G(zq), labeled
(1), and G-1(sk) labeled (2)
4) Histogram of the histogram-
specified image

26 / 41
Global Histogram Processing

●
Both histogram equalization and histogram matching are global histogram
processing methods, since pixels are modified based on the intensity
distribution of the entire image.
●
If the aim is to enhance details over small areas of an image, they fail,
since the number of pixels in small areas have less significant impact in
the computation of global transformations.
●
To remedy this, we can devise transformations based on the intensity
distribution of pixel neighborhoods.
●
For this, we define a neighborhood, move its center from pixel to pixel in a
raster scanning scheme, and at each location, histogram equalization or
matching is performed and a new mapped intensity is computed for the
center pixel of the neighborhood.
●
Since only one row or column of the neighborhood changes in a one-pixel
translation, the histogram from the previous step can be updated so there
is no need for full calculation of the histogram in each neighborhood.
27 / 41
Global vs. Local Histogram Processing

●
Example of global vs. local (3-by-3 neighborhood) histogram equalization
in a noisy image, with very faint shapes in the dark regions.

28 / 41
Fundamentals of Spatial Filtering

●
Two components in any spatial
filter definition:
– A neighborhood

– An operation defined in the

neighborhood

●
Spatial Filtering:
– Linear spatial filters (e.g.
mean)
– Nonlinear spatial filters (e.g.
median)

29 / 41
Linear Spatial Filtering

●
Sum-of-products operation between an image f and a filter kernel w.
●
The kernel is an array which defines the neighborhood of operation, and its
coefficients determine the nature of the filter. w (−1,−1) w (−1 , 0) w (−1 ,1)
●
A kernel is also called mask, template, and window. w (0 ,−1) w (0, 0) w (0 , 1)
●
Assume a 3-by-3 kernel like the figure.
w(1 ,−1) w(1, 0) w(1 , 1)

●
At any point (x,y), the response, g(x,y), of the filter is the sum-of-products
of the kernel coefficients and the image pixels encompassed by the kernel:
g ( x , y)=w (−1 ,−1) f ( x−1 , y−1)+w (−1,0) f ( x−1 , y )+...
+w (0,0) f ( x , y)+...+w (1,1) f ( x+1 , y+1)
●
As the coordinates (x,y) change, the center of the kernel moves from pixel
to pixel, generating the filtered image g.

30 / 41
Linear Spatial Filtering

●
The center coefficient of the kernel, w(0,0), aligns with the pixel at location
(x,y).
●
A generalized kernel of size (mxn), with m=2a+1 and n=2b+1, where a
and b are non-negative integers, can be applied to the image to create the
filtered image g(x,y) such as:
a b
g ( x , y)= ∑ ∑ w(s ,t ) f ( x+ s , y+t )
s=−a t =−b

where x and y are changed so the center of the kernel goes through every
pixel in image f once.

31 / 41
Correlation vs. Convolution: 1D Case

●
Correlation: (also shown with )
a
(w⊕f )(x)= ∑ w (s) f (x+s)
s=−a

●
Convolution: (also shown with )
a
(w⊗f )(x)= ∑ w (s) f (x−s)
s=−a

●
Function f is a discrete unit impulse.

{
δ ( x− x 0 )= 1
0
if x=x 0
otherwise
●
Pre-rotating the kernel results in an
exact copy of the kernel.
●
Linear spatial filtering and spatial
convolution are synonymous.
32 / 41
Correlation vs. Convolution: 2D Case

●
The same can be said about 2D case.
●
Correlation:
a b
(w⊕f )(x , y )= ∑ ∑ w( s ,t) f ( x+s , y +t)
s=−a t=−b

●
Convolution: a b
(w⊗f )(x , y )= ∑ ∑ w( s ,t) f ( x−s , y−t )
s=−a t=−b

●
Function f is a discrete unit impulse.

{
δ ( x− x 0 , y− y 0 )= 1 if x=x 0 and y= y 0
0 otherwise
●
Pre-rotating the kernel results in an
exact copy of the kernel.
●
Linear spatial filtering and spatial
convolution are synonymous. 33 / 41
Correlation vs. Convolution: Properties

Properties Convolution Correlation

Commutative f ⊗g=g⊗f -
Associative f ⊗(g⊗h)=(f ⊗g)⊗h -
Distributive f ⊗(g+h)=(f ⊗g)+(f ⊗h) f ⊕(g+h)=(f ⊕g)+(f ⊕h)

●
Because of the commutative property of convolution, it is not important
whether the kernel or the image is pre-rotated.
●
Also, performing multistage filtering is possible, if the kernels are
convolved first, and the resulting kernel is applied to the image.

w=w 1⊗w 2⊗...⊗w N

34 / 41
Separable Filter Kernels

●
A 2D function G(x,y) is said to be separable if it can be written as the
product of two 1D functions, G1(x) and G2(y):
G( x , y)=G1 ( x )∗G2 ( y)
●
For example, the following kernel w is separable, since:

[]
1
[
1 1 1 1] []
w= 1 1 1 ⇒ c= 1 and r= 1 ⇒ w=c r T
1
●
A separable kernel of size mxn can be expressed as the outer product of
two vectors v and w: T
w=v w
where v and w are vectors of size mx1 and nx1 respectively.
●
For a square kernel of size mxm we have:
w=v v T
●
These are equivalent to the 2D convolution of a column vector and a row
vector. 35 / 41
Separable Filter Kernels

●
Why is it important? Recall the commutative and associative properties of
the colvolution:
f ⊗g=g⊗f and f ⊗( g⊗h)=(f ⊗g)⊗h
●
Now with a separable kernel w:
w⊗f =(w 1⊗w 2 )⊗f =(w 2 ⊗w 1 )⊗f =w 2 ⊗(w1 ⊗f )=(w 1 ⊗f )⊗w 2
●
Assume an image of size MxN and a kernel of size mxn.
●
The computational advantage of a separable in comparison to a non-
separable kernel in terms of the number multiplications and summations
can be derived as (how? DIY!):
MNmn mn
C= =
MN (m+n) m+n
●
For a kernel of 11x11, this is 5.5. What is it for a kernel of 101x101?
36 / 41
Separable Filter Kernels

●
Matrix formed by product of a column vector and a row vector always has
rank of 1.
●
In another word, rows and columns of the matrix are linearly dependent,
meaning the rows differ only by a constant multiplier. The same is true for
the columns.
●
When a matrix has a rank of 1 (rank function in MATLAB), to get the
separable kernels:
1) Find any non-zero element in the matrix, denote its value as E.
2) Form vectors c and r equal to the column and row in the kernel
containing the element in Step 1, respectively;
3) Given these, v=c and wT=r/E.

37 / 41
Filtering in Spatial and Frequency Domains

●
Fourier Transform (FT) is used to go from the spatial to frequency domain.
●
FT decomposes the signal (image) into its consisting frequency
components.
●
Two fundamental properties:
1) Convolution in the spatial domain, is equivalent to multiplication in the
frequency domain, and vice versa.
2) An impulse function of amplitude A in the spatial domain, is a constant
function of value A in the frequency domain, and vice versa.

38 / 41
How to Build Spatial Filter Kernels?

●
There are three main approaches for building spatial kernel filters:
1) Formulating based on mathematical properties; for example using
integration (averaging) for blurring an image, or computing local
derivatives to sharpen an image.
2) Sampling a 2D spatial function whose shape has a desired property;
for example creating a weighted-average filter by sampling a 2D
Gaussian function, or sampling the inverse FT of a 2D filter that is
specified in the frequency domain.
3) Designing a spatial filter with a specified frequency response; for
example designing a 1D spatial filter and then forming a separable or
circularly symmetric 2D version.

39 / 41
What is next?

●
Smoothing (lowpass) spatial filters
●
Sharpening (highpass) spatial filters
●
Highpass, bandreject, and bandpass filters from lowpass filters
●
Combining spatial enhancement methods

40 / 41
Questions?
[email protected]

41 / 41