CSC 425

Digital Image
Processing

11/21/2022 Dr. Maryam Al-Berry 1

[email protected]
 Region-based
Segmentation

11/21/2022 2
Region-based Segmentation
Basic Formulation
• Partition image region 𝑹 into subregions such that:

• 𝑷(𝑹𝒊 ): logical predicate property defined over 𝑹𝒊 .

Region-based Segmentation – (cont.)
1- Region Growing
• Group based a predefined criteria of growth.
• Start with a set of seed points.
• Append to each seed neighboring pixels that
satisfy the criteria (have properties similar to
the seed).
Region-based Segmentation – (cont.)
1- Region Growing
• Choice of seed
• Choice of criteria
• Connectivity properties
• Stopping rule
• Additional criteria
Region-based Segmentation – (cont.)
1- Region Growing
• Choice of seed
– It depends on the nature of the problem.
– If targets need to be detected using infrared images for
example, choose the brightest pixel(s).
– Without apriori knowledge, compute the histogram
and choose the gray-level values corresponding to the
strongest peaks.
Region-based Segmentation – (cont.)
1- Region Growing
• Choice of criteria
– The homogeneity predicate can be based on any
characteristic of the regions in the image such as
• average intensity
• variance
• color
• texture
• motion
• shape
• size
Region-based Segmentation – (cont.)
1- Region Growing
• Choice of seed
• Choice of criteria
• Connectivity properties
– Four connectivity vs eight connectivity.
• Stopping rule
– When criteria is not met.
Region-based Segmentation – (cont.)
1- Region Growing
• Example
Region-based Segmentation – (cont.)
1- Region Growing
• Example
Region-based Segmentation – (cont.)
1- Region Growing
• Advantages:
1. Region growing methods can correctly separate the regions that have
the same properties we define.
2. Region growing methods can provide the original images which have
clear edges with good segmentation results.
3. The concept is simple. We only need a small number of seed points to
represent the property we want, then grow the region.
4. We can determine the seed points and the criteria we want to make.
5. We can choose the multiple criteria at the same time.
6. It performs well with respect to noise.
Region-based Segmentation – (cont.)
1- Region Growing
• Disadvantage:
1. The computation is consuming.
2. Noise or variation of intensity may result in holes or over
segmentation.
3. This method may not distinguish the shading of the real
object.
Region-based Segmentation – (cont.)
2- Region Merging
Region-based Segmentation – (cont.)
3- Region Splitting
Region-based Segmentation – (cont.)
4- Region Splitting and Merging
• Subdivide image initially into a set of
arbitrary disjoint regions.
• Then merge and/or split the regions to
satisfy the segmentation conditions.
Region-based Segmentation – (cont.)
4- Region Splitting and Merging
• Quadtree:
- Split into 4 disjoint quadrants any region 𝑅𝑖 for
which 𝑃(𝑅𝑖 ) = FALSE.
- Merge any adjacent region 𝑅𝑗 and 𝑅𝑘 for which
𝑃(𝑅𝑖 ∪ 𝑅𝑘 ) = TRUE
- Stop when no further merging or splitting is
possible.
Region-based Segmentation – (cont.)
4- Region Splitting and Merging
Example: 𝑃(𝑅𝑖 ) = TRUE if at least 80% of the
pixels in 𝑅𝑖 have the property |𝑧𝑗 − 𝑚𝑖 | ≤ 2𝜎𝑖

Original Image Result of split/merge Result of thresholding

Region-based Segmentation – (cont.)
4- Region Splitting and Merging
• Advantages:
– The image could be split progressively according to our
demanded resolution because the number of splitting
level is determined by us.
– We could split the image using the criteria we decide,
such as mean or variance of segment pixel value.
– In addition, the merging criteria could be different to the
splitting criteria.
Region-based Segmentation – (cont.)
4- Region Splitting and Merging
• Disadvantages:
– It may produce the blocky segments.
– The blocky segment problem could be reduced by
splitting in higher level, but the trade off is that
the computation time will arise.
The Use of Motion in Segmentation
The Use of Motion in Segmentation

Motion
Detection

Wavelet-
Background Frame Optical Flow
based
Subtraction Differencing Techniques
Techniques
Segmentation Methods (some)
Image Segmentation

Edge Region Fuzzy theory ANN

CNN
Threshold Genetics PDE Deep

• Graph cuts
• Graph-based • Interactive
• Color • Sports
• Active Contours • Medical images
• Mean shift • Surveillance
• Matting • Videos
• Machine learning • Infrared
• etc. • etc.
Examples

Normalized cuts segmentation

Richard Szeliski, Computer Vision Algorithms and Applications, Springer 2010. (Chapter 5)
Examples

Graph cuts for region segmentation

(a) the energy function is encoded as a maximum flow problem; (b) the minimum cut determines the
region boundary

Richard Szeliski, Computer Vision Algorithms and Applications, Springer 2010. (Chapter 5)
Examples

Segmentation with a directed graph cut

(a) directed graph; (b) image with seed points; (c) the undirected graph incorrectly continues
the boundary along the bright object; (d) the directed graph correctly segments the light gray
region from its darker surround.

Richard Szeliski, Computer Vision Algorithms and Applications, Springer 2010. (Chapter 5)
Examples – (cont.)
 Morphological
Image Processing
Fundamental Steps of DIP
Wavelet and
Multi- Compression
resolution
Output is generally images

Color Image Morphological

Output is generally attributes

Processing Processing

Knowledge base
Restoration Segmentation

Representation
Enhancement and
Description

Object
Acquisition Recognition
Problem Domain
Contents

1. What is Morphology?
2. Fundamentals
3. Basic Morphological Operations
4. Compound Operations
5. Basic Morphological Algorithms
What is Morphology?
Mathematical morphology
• Morphological image processing describes a
range of image processing techniques that
deal with the shape of features in an image.
• Hence, used in image analysis based on
shape.
• Nonlinear operations that are based on
Minkowski’s set theory.
What is Morphology? – (cont.)
• Used to extract image components that are
useful in the representation and description
of regions shapes.
• Can be used to remove imperfections in the
segmented image and provide information
on the form and structure of the image.
• Pre- or post-processing.
What is Morphology? – (cont.)
Uses of Image Morphology
• image enhancement • shape analysis
• image segmentation • image compression
• image restoration • component analysis
• edge detection • curve filling
• texture analysis
• general thinning
• particle analysis
• feature detection
• feature generation
• skeletonization • noise reduction
What is Morphology? – (cont.)
Examples

Image after segmentation Image after segmentation and

morphological processing
Fundamentals
Mathematical Morphology
• In binary images, sets represents groups of
pixels at specific locations.
• Each pixel is represented in the 2D integer
space Z2 by vector (x,y) of white or black.
• Morphological operations change the
“shape” of the objects of interest according
to set theory concepts, e.g. union,
intersection, and complement.
Fundamentals – (cont.)
Bi-level versus Grey-scale Morphology
• Binary morphological operations are typically
applied to bi-level images (foreground vs
background pixels).

• Grey-scale operations are applied in the same

manner but to grey-scale images with each pixel
represented in the 3D integer space Z2 by vector
(x,y,g), where g is the gray level of pixel coordinate
(x,y).
Fundamentals – (cont.)
• Fundamentally, morphological image
processing is very much like spatial filtering.

• A structuring element is moved across every

pixel in the original image to give a pixel in a
new processed image.

• The value (black or white) of this new pixel

depends on the operation performed.
Fundamentals – (cont.)
Structuring Element
• A SE can be any size and any shape.
• For simplicity we use rectangular structuring
elements with the origin at the middle pixel.

0 0 1 0 0
1 1 1 0 1 0 0 1 1 1 0
1 1 1 1 1 1 1 1 1 1 1
1 1 1 0 1 0 0 1 1 1 0
0 0 1 0 0
Fundamentals – (cont.)
Structuring Element
Fundamentals – (cont.)
Structuring Element – Hits and Fits

SE

Fit: All on pixels in the

structuring element cover
on pixels in the image
Hit: Any on pixel in the
structuring element covers
an on pixel in the image
All morphological processing operations are
based on these simple ideas.
Fundamentals – (cont.)
Structuring Element – Hits and Fits

SE

Fit: All on pixels in the

structuring element cover
on pixels in the image
A Hit: Any on pixel in the
structuring element covers
an on pixel in the image
All morphological processing operations are
based on these simple ideas.
Fundamentals – (cont.)
Structuring Element – Hits and Fits

B SE

Fit: All on pixels in the

structuring element cover
on pixels in the image
A Hit: Any on pixel in the
structuring element covers
an on pixel in the image
All morphological processing operations are
based on these simple ideas.
Fundamentals – (cont.)
Structuring Element – Hits and Fits

B SE

Fit: All on pixels in the

structuring element cover
on pixels in the image
A Hit: Any on pixel in the
C structuring element covers
an on pixel in the image
All morphological processing operations are
based on these simple ideas.
Fundamentals – (cont.)
Structuring Element – Hits and Fits
0 0 0 0 0 0 0 0 0 0 0 0
1 1 1
0 0 0 1 1 0 0 0 0 0 0 0
1 1 1
0 0 1 1 1 1 1 0 0 0 0 0
1 1 1
0 1 1 1 1 1 1 1 0 0 0 0 Structuring
0 1 1 1 1 1 1 1 0 0 0 0 Element 1

0 0 1 1 1 1 1 1 0 0 0 0 0 1 0
0 0 1 1 1 1 1 1 1 0 0 0 1 1 1
0 0 1 1 1 1 1 1 1 1 1 0 0 1 0
0 0 0 0 0 1 1 1 1 1 1 0 Structuring
Element 2
0 0 0 0 0 0 0 0 0 0 0 0
Fundamentals – (cont.)
Structuring Element – Hits and Fits
0 0 0 0 0 0 0 0 0 0 0 0
1 1 1
0 0 0 1 1 0 0 0 0 0 0 0
1 1 1
0 0 1 1 1 1 1 0 0 0 0 0
1 1 1
0 1 1 1 1 1 1 1 0 0 0 0 Structuring
0 1 1 1 1 1 1 1 0 0 0 0 Element 1

0 0 1 1 1 1 1 1 0 0 0 0 0 1 0
0 0 1 1 1 1 1 1 1 0 0 0 1 1 1
0 0 1 1 1 1 1 A1 1 1 1 0 0 1 0
0 0 0 0 0 1 1 1 1 1 1 0 Structuring
Element 2
0 0 0 0 0 0 0 0 0 0 0 0
Fundamentals – (cont.)
Structuring Element – Hits and Fits
0 0 0 0 0 0 0 0 0 0 0 0
1 1 1
0 0 0 1 1 0 0 0 0 0 0 0
1 1 1
0 0 1 B1 1 1 1 0 0 0 0 0
1 1 1
0 1 1 1 1 1 1 1 0 0 0 0 Structuring
0 1 1 1 1 1 1 1 0 0 0 0 Element 1

0 0 1 1 1 1 1 1 0 0 0 0 0 1 0
0 0 1 1 1 1 1 1 1 0 0 0 1 1 1
0 0 1 1 1 1 1 A1 1 1 1 0 0 1 0
0 0 0 0 0 1 1 1 1 1 1 0 Structuring
Element 2
0 0 0 0 0 0 0 0 0 0 0 0
Fundamentals – (cont.)
Structuring Element – Hits and Fits
0 0 0 0 0 0 0 0 0 0 0 0
1 1 1
0 0 0 1 1 0 0 0 0 0 0 0
1 1 1
0 0 1 B1 1 1 1 0 C0 0 0 0
1 1 1
0 1 1 1 1 1 1 1 0 0 0 0 Structuring
0 1 1 1 1 1 1 1 0 0 0 0 Element 1

0 0 1 1 1 1 1 1 0 0 0 0 0 1 0
0 0 1 1 1 1 1 1 1 0 0 0 1 1 1
0 0 1 1 1 1 1 A1 1 1 1 0 0 1 0
0 0 0 0 0 1 1 1 1 1 1 0 Structuring
Element 2
0 0 0 0 0 0 0 0 0 0 0 0
Fundamentals – (cont.)
Structuring Element – Reflected/Translated
• The reflection of set B, denoted by B̂ , is
defined as: Bˆ = z | z = −b, for b  B

SE1 ŜE1 SE 2 ŜE 2 SE 3 ŜE 3

The translation of set B, by point z, denoted by
(B)z, is defined as:
( B) z = c | c = b + z, for b  B
Basic Morphological Operations
1- Erosion
• Erosion of image X by structuring element
SE is given by X  SE.
• The structuring element SE is positioned
with its origin at (x, y) and the new pixel
value is determined using the rule:
1 if SE fits X
g ( x, y ) = 
 0 otherwise
• Or in set language:
Erosion
Original Image Processed Image With Eroded Pixels

Structuring Element
Erosion – (cont.)
Original Image Processed Image With Eroded Pixels

Structuring Element
Erosion – (cont.)
Original Image Processed Image With Eroded Pixels

Structuring Element
Erosion – (cont.)
Original Image Processed Image With Eroded Pixels

Structuring Element
Erosion – (cont.)
Original Image Processed Image With Eroded Pixels

Structuring Element
Erosion – (cont.)
Original Image Processed Image With Eroded Pixels

Structuring Element
Erosion – (cont.)
Original Image Processed Image With Eroded Pixels

Structuring Element
Erosion – (cont.)
Original Image Processed Image With Eroded Pixels

Structuring Element
Erosion – (cont.)
Original Image Processed Image With Eroded Pixels

Structuring Element
Erosion – (cont.)
Original Image Processed Image With Eroded Pixels

Structuring Element
Erosion – (cont.)
Original Image Processed Image With Eroded Pixels

Structuring Element
Erosion – (cont.)
Original Image Processed Image With Eroded Pixels

Structuring Element
Erosion – (cont.)
Original Image Processed Image With Eroded Pixels

Structuring Element
Erosion – (cont.)
Original Image Processed Image With Eroded Pixels

Structuring Element
Erosion – (cont.)
Original Image Processed Image With Eroded Pixels

Structuring Element
Erosion – (cont.)

Original image Erosion by 3*3 Erosion by 5*5

square SE square SE

Watch out: In these examples, a 1 refers to a black pixel!

Erosion – (cont.)

eroded image erosion / original original image

SE = Z8 This is a piece of a larger image. Boundary effects are not apparent

Erosion – (cont.)

Original After erosion

image with a disc of
radius 10

After erosion After erosion

with a disc of with a disc of
radius 5 radius 20
Erosion – (cont.)
Erosion can split apart joined objects

Erosion can split apart

Erosion can strip away extrusions

Erosion shrinks objects!

Erosion – (cont.)
Examples of varying SE size

Structuring
Element

Pablo Picasso, Pass with the Cape, 1960

Foreground is BLACK
Qigong Zheng
Basic Morphological Operations
2- Dilation
• Dilation of image X by structuring element
SE is given by X  SE
• The structuring element SE is positioned
with its origin at (x, y) and the new pixel
value is determined using the rule:
1 if SE hits X
g ( x, y ) = 
 0 otherwise
^
Or in set language: X  SE = {z | ( SE ) z  X  }
Dilation
Original Image Processed Image With Dilated Pixels

Structuring Element
Dilation – (cont.)

Original image Dilation by 3*3 square Dilation by 5*5 square

structuring element structuring element

Watch out: In these examples, a 1 refers to a black pixel!

Dilation – (cont.)

dilated image original / dilation original image

SE = Z8 This is a piece of a larger image. Boundary effects are not apparent

Dilation – (cont.)
Original image After dilation

Structuring element
Dilation – (cont.)
Dilation can repair breaks

Dilation can repair intrusions

Dilation enlarges objects!

Dilation – (cont.)
Examples of varying SE size

Structuring
Element

Pablo Picasso, Pass with the Cape, 1960

Foreground is BLACK
Qigong Zheng
Compound Operations
• More interesting morphological operations
can be performed by combinations of
erosions and dilations.
• The most widely used compound operations
are:
- Opening.
- Closing.
Opening
• The opening of image AX by structuring
element SE, denoted X ○ SE is simply an
erosion followed by a dilation:
X ○ SE = (X  SE)  SE

• Opening smoothes the contour of objects,

breaks narrow bridges, and eliminates thin
protrusions.
Opening – (cont.)
Original Image Processed Image

Structuring Element
Opening – (cont.)

opening erosion original

SE = Z8 This is a piece of a larger image. Boundary effects are not apparent
Opening – (cont.)

original erosion opening

Opening – (cont.)
Examples of varying SE size

Structuring
Element

Pablo Picasso, Pass with the Cape, 1960

Foreground is BLACK
Qigong Zheng
Closing
• The closing of image X by structuring
element SE, denoted X • SE is simply a
dilation followed by an erosion:
X • SE = (X  SE)SE

• Closing smoothes sections of contours as

well, but fuses narrow breaks, eliminate
small holes, and fill gaps.
Closing – (cont.)
Original Image Processed Image

Structuring Element
Closing – (cont.)

closing dilation original

SE = Z8 1using the reflected SE, Ž
Closing – (cont.)
Examples of varying SE size

Structuring
Element

Pablo Picasso, Pass with the Cape, 1960

Foreground is BLACK
Qigong Zheng
Compare

closing dilation
original

original

SE = Z8
opening erosion 1using the reflected SE, Ž
Examples

Erosion
Examples – (cont.)

Erosion
Examples – (cont.)

Erosion
Examples – (cont.)

Dilation
Examples – (cont.)

Dilation
Examples – (cont.)

Dilation
Examples – (cont.)

The erosion of the dark-blue square by The dilation of the dark-blue square by a
a disk, resulting in the light-blue square disk, resulting in the light-blue square
with rounded corners
Examples – (cont.)

The closing of the dark-blue shape (union of

The opening of the dark-blue square by a disk, two squares) by a disk, resulting in the union
resulting in the light-blue square with round of the dark-blue shape and the light-blue
corners. areas.
Examples – (cont.) Opening
Examples – (cont.) Closing
Examples – (cont.)
Examples – (cont.)
Examples – (cont.)

Dilation:
- Object thickens
- Intrusions reduce in size
if survived
- Protrusions increase in
size if survived
- Outer corners round
(Disk SE).
Examples – (cont.)
Examples – (cont.)

Opening:
- Outward pointing
corners → rounded
- Inward pointing corners
→ unchanged if survived
- Smoothing effect
Examples – (cont.)
Examples – (cont.)

Closing:
- Inward pointing corners
→ rounded
- Outward pointing corners
→ unchanged
- Intrusions reduces/breaks
if B doesn’t fit
- Smoothing effect
Examples – (cont.)
Examples – (cont.)

Erosion:
- Object shrinks by SE size
- Protrusions where B didn’t
fit were eliminated
Some Basic Morphological Algorithms
• Using the simple techniques we have looked
at so far we can begin to consider some more
interesting morphological algorithms:
– Hit-or-miss transform - Top-Hat
– Boundary Extraction - Hole Filling
– Connected components - Convex Hull
– Thinning/thickening - Skeletonziation
– Pruning
Top-Hat Transform
• Top-hat transformation of an image X is
defined as the difference between the
original image X and its opening
• TH = X - (X ○ SE)

X X ○ SE Top Hat

Y. Tarabalka, J. A. Benediktsson, J. Chanussot

Morphological Gradient
• The basic morphological gradient of an image
X is defined as the arithmetic difference
between the dilation and the erosion of X by
the elementary SE
• TH = (X  SE) - (X  SE)

Y. Tarabalka, J. A. Benediktsson, J. Chanussot

Boundary Extraction

Original Image Extracted Boundary

Boundary Extraction – (cont.)
• The boundary can be given simply as
β(A) = A – (AB)
Hole Filling
• What is a hole?
• Given a pixel inside a boundary, region filling
attempts to fill that boundary with object
pixels (1s).

Given a point inside

here, can we fill the
whole circle?
Hole Filling – (cont.)
• The key equation for region filling is:
X k = ( X k −1  B)  A c
k = 1,2,3.....
• Where X0 is a zero image except the starting
point inside the holes, B is a simple SE, and
Ac is the complement of A.
• This equation is applied repeatedly until
Xk=Xk-1.
• Finally the result is unioned with the original
boundary.
Hole Filling – (cont.)
• ??
X k = ( X k −1  B)  A c
k = 1,2,3.....

The intersection with Ac

limits the result inside the
region of interest.

Note here that the foreground is

dark and the background is white.
Hole Filling – (cont.)

Original Image One Region All Regions

Filled Filled
Connected Components Extraction
X k = ( X k −1  B)  A k = 1,2,3.....
Connected Components Extraction
X k = ( X k −1  B)  A k = 1,2,3.....

Would it differ if we used

Connected Components Extraction

dilated
tag tag

masked
dilated tags
tag

masked
dilated
dilated
tags
tags
Connected Components Extraction

dilated
tags tags

masked dilated
tags tags

masked
connected
dilated
component
tags
Connected Components Extraction
• Automated inspection of processed food
• X-ray image of chicken filet.
• Identify bone fragments.
Connected Components Extraction
• Automated inspection of processed food
• X-ray image of chicken filet.
• Identify bone fragments.
1. Thresholded image
Connected Components Extraction
• Automated inspection of processed food
• X-ray image of chicken filet.
• Identify bone fragments.
1. Thresholded image
2. Eroded image (size?)
Connected Components Extraction
• Automated inspection of processed food
• X-ray image of chicken filet.
• Identify bone fragments.
1. Thresholded image
2. Eroded image (size?)
3. Size of connected component
Helpful Reading
• Convex Hull • MR: morphological
• Thinning/Thickening reconstruction
• Sekeletonization
• Pruning
Gray Scale Morphology
Basic Operations
Gray Scale Morphology – (cont.)
Dilation

SE, Z, is a flat disk.

Gray Scale Morphology – (cont.)
Erosion

SE, Z, is a flat disk.

Gray Scale Morphology – (cont.)
Opening

SE, Z, is a flat disk.

Gray Scale Morphology – (cont.)
Closing

SE, Z, is a flat disk.

Next Lecture
Revision + Quiz 1

Assignment
Check book sections and associated problems
Chapter 9 1, 2, 3, 5(1,2,3)

Associated 1, 5, 6, 7, 8, 17, 18, 20, 21, 22, 24, 36, 37

problems

11/21/2022 134
References
- Gonzalez and Woods, Digital Image Processing.
- Peters, Richard Alan, II, “Image Morphology", Lectures on Image Processing,
Vanderbilt University, Nashville, TN, April 2008, Available on the web at the Internet
Archive, http://www.archive.org/details/Lectures_on_Image_Processing.
-Yuliya Tarabalka, Jón Atli Benediktsson, Jocelyn Chanussot, “Mathematical
morphology”, presentation, University of Iceland/Grenoble Institute of Technology,
France.
- Qigong Zheng, “Mathematical Morphology”, University of Maryland College Park.

11/21/2022 135