Introduction to Computer Vision

(ME-4201)

Segmentation
Phung Thanh Huy
Department of Mechatronics
Ho Chi Minh City University of Technology
Vietnam National University Ho Chi Minh City
Ver2024.1

Ho Chi Minh City– 2024

Segmentation
Segments
• Image segmentation partitions an image into regions, called segments.
• Purposes: for further image analysis, improved efficiency of image
compression, or just for visualization effects.
• Image segmentation creates segments of connected pixels by analyzing
some similarity criteria.
• Segmentation aims at identifying “meaningful” segments that can be
used for describing the contents of an image, such as a segment for
“background”, segments for “objects”, or particular object categories.

Segmentation |2
Segmentation
Segments and segmentation

Image is partitioned into

segments and six segments
of this partition.

The image mapped into a

(nearly binary) image defined
by Winnemöller stylization.
Connected dark or bright
regions define segments

Segmentation of the
image by mean-shift for radii r
= 12 (middle) and r = 24
(right),

Segmentation |3
Segmentation
Segments and segmentation

Segmentation |4
Segmentation
Segments and segmentation

Segmentation |5
Segmentation
Segments and segmentation

Segmentation |6
Segmentation by binarization
Intensity Thresholding
Global thresholding Multiple thresholding

• When the value of T changes over an image, we use the term variable
thresholding
• If T depends on the spatial coordinates (x, y) themselves, then variable
thresholding is often referred to as dynamic or adaptive thresholding
Segmentation |7
Segmentation by binarization
Intensity Thresholding
• The thresholding results are affected by noise and illumination.

Segmentation |8
Segmentation by binarization
Intensity Thresholding
• The thresholding results are affected by noise and illumination.

Segmentation |9
Segmentation by binarization
Intensity Thresholding
• Global Thresholding Algorithm

Segmentation |10
Segmentation by binarization
Otsu Binarization
Principle: maximizing the between-class variance
• Consider normalized histogram 𝑝𝑖 of an n-bit image. The intensity level: 0 → (2𝑛 − 1).
Let 𝐿 = 2𝑛

• Support 𝑇 𝑘 = 𝑘 is threshold

• The mean intensities

Segmentation |11
Segmentation by binarization
Otsu Binarization
• The cumulative mean (average intensity) up to level k

and the average intensity of the entire image (i.e., the global mean)

• The effectiveness of the process is justified by:

• Final result
Find

Segmentation |12
Segmentation by binarization
Otsu Binarization

𝑖 0 1 2 … 2𝑛 − 2 2𝑛 − 1
𝑝𝑖
𝑖𝑝𝑖
𝑃1 (𝑘)
𝑚 𝑘 𝑚𝐺
𝜎𝐵2 𝑘 𝜎𝐺2

• Obtain the Otsu threshold, 𝑘 ∗ , as the value of 𝑘 for which 𝜎𝐵2 𝑘 is maximum.
• If the maximum is not unique, obtain k* by averaging the values of k corresponding
to the various maxima detected.

Segmentation |13
Segmentation by binarization
Otsu Binarization

Segmentation |14
Segmentation by binarization
Improving the binarization
Using image smoothing

Segmentation |15
Segmentation by binarization
Improving the binarization
Using Edges

Segmentation |16
Segmentation by binarization
Improving the binarization
Using Edges

1. Compute an edge image as either the magnitude of the gradient, or

absolute value of the Laplacian of f(x,y)
2. Specify a threshold value T
3. Threshold the image and produce a binary image, which is used as
a mask image; and select pixels from f(x,y) corresponding to
“strong” edge pixels
4. Compute a histogram using only the chosen pixels in f(x,y)
5. Use the histogram from step 4 to segment f(x,y) globally

Segmentation |17
Segmentation by binarization
Multiple Thresholding
• In the case of classes, , , ..., , the between-class variance is:

• The optimum threshold values, 𝑘1∗ , 𝑘2∗ , … , 𝑘𝐾−1

∗
, that maximize

• Thresholding

• Efficiency

Segmentation |18
Segmentation by binarization
Multiple Thresholding Region 3 (b)

Region 1 (a)
Region 2 (b)

Segmentation |19
Segmentation by binarization
Variable Thresholding Based on Local Image Properties
At each point (x,y), consider a neighborhood 𝑆𝑥𝑦 . Inside the neighborhood area:
Need to choose:
If the background is nearly constant + Neighborhood size.
+a
+b
Global mean

Improvement:

Segmentation |20
Segmentation by binarization
Variable Thresholding Based on Moving Averages
• Computing a moving average along scan lines of an image.
• The scanning typically is carried out line by line in a zigzag pattern to reduce
illumination bias
𝑧1
𝑚 1 =
𝑛 n: number of points for
𝑘
1 1 averaging
𝑚 𝑘 = ෍ 𝑧𝑖 = 𝑚 𝑘 − 1 + 𝑧𝑘 − 𝑧𝑘−𝑛+1
𝑛 𝑛
𝑖=𝑘−𝑛+1

Image is padded with (n-1) zeros pixels

n=3

Segmentation |21
Segmentation by binarization
Variable Thresholding Based on Moving Averages
• For each point

Segmentation |22
Segmentation by binarization
Variable Thresholding Based on Moving Averages

• Thresholding based on moving averages works well when the objects are small with
respect to the image size.
• Quite useful in document processing

Segmentation |23
Regional-based Segmentation
Basic Topology and Relationships of Images
Neighbors

Segmentation |24
Regional-based Segmentation
Basic Topology and Relationships of Images
Adjacency

Segmentation |25
Regional-based Segmentation
Basic Topology and Relationships of Images
Adjacency

Segmentation |26
Regional-based Segmentation
Basic Topology and Relationships of Images
Path (Curve)

Segmentation |27
Regional-based Segmentation
Basic Topology and Relationships of Images
Connectivity
Let S represent a subset of pixels in an image.
• Two pixels p and q are said to be connected in S if there exists a path between
them consisting entirely of pixels in S.
• For any pixel p in S, the set of pixels that are connected to it in S is called a
connected component of S.
• If it only has one connected component, then set S is called a connect set.
Region
Let R be a subset of pixels in an image.
• R is a region of the image if R is a connected set.
• The boundary (also called border or contour) of a region R is the set of pixels in
the region that have one or more neighbors that are not in R.

Segmentation |28
Regional-based Segmentation
Basic Topology and Relationships of Images
Distance

Segmentation |29
Regional-based Segmentation
Region Growing
• Region growing is a procedure that groups pixels or subregions into
larger regions.
• The simplest of these approaches is pixel aggregation, which starts
with a set of “seed” points and from these grows regions by
appending to each seed points those neighboring pixels that have
similar properties (such as gray level, texture, color, shape).
• Region growing based techniques are better than the edge-based
techniques in noisy images where edges are difficult to detect.

Example: Region Growing based on 8-connectivity

Segmentation |30
Regional-based Segmentation
Region Growing
• Algorithm:

Segmentation |31
Regional-based Segmentation
Region Growing

𝑆(𝑥, 𝑦)

T=5
4-connectivity 8-connectivity

Segmentation |32
Regional-based Segmentation
Region Growing

𝑆(𝑥, 𝑦)

T=5
4-connectivity 8-connectivity

Segmentation |33
Regional-based Segmentation
Region Growing Image Seed

Seed (erode)

Seed growing

Segmentation |34
Regional-based Segmentation
Region splitting and Merging

◼ Region Splitting
❑ Region growing starts from a set of seed points or whole
image (𝑅).
❑ Subdivide the regions (𝑅 → 𝑅1 , … , 𝑅𝑛 ) that do not satisfy a
condition of homogeneity (Q).
◼ Region Merging
❑ Region merging is the opposite of region splitting.
❑ Start with small regions 𝑅𝑖 (e.g. 2x2 or 4x4 regions) and
merge the regions that have similar characteristics (such as
gray level, variance) (𝑅𝑖 , … , 𝑅𝑗 → 𝑅).
❑ Typically, splitting and merging approaches are used
iteratively.

Segmentation |35
Regional-based Segmentation
Region splitting and Merging

Segmentation |36
Regional-based Segmentation
Region splitting and Merging

Segmentation |37
Regional-based Segmentation
Watershed Segmentation Algorithm

◼ Visualize an image in 3D: spatial coordinates and gray levels.

◼ In such a topographic interpretation, there are 3 types of points:
❑ Points belonging to a regional minimum

❑ Points at which a drop of water would fall to a single minimum.

(→The catchment basin or watershed of that minimum.)
❑ Points at which a drop of water would be equally likely to fall to more
than one minimum. (→The divide lines or watershed lines.)

Watershed lines

Segmentation |38
Regional-based Segmentation
Watershed Segmentation Algorithm
◼ The objective is to find watershed lines.
◼ The idea is simple:
❑ Suppose that a hole is punched in each regional minimum and that the entire
topography is flooded from below by letting water rise through the holes at a
uniform rate.
❑ When rising water in distinct catchment basins is about the merge, a dam is
built to prevent merging. These dam boundaries correspond to the watershed
lines.

Segmentation |39
Regional-based Segmentation
Watershed Segmentation Algorithm

Segmentation |40
Regional-based Segmentation
Watershed Segmentation Algorithm
• Start with all pixels with the lowest possible value.
• These form the basis for initial watersheds
• For each intensity level k:
• For each group of pixels of intensity k
• If adjacent to exactly one existing region, add these pixels to that region
• Else if adjacent to more than one existing regions, mark as boundary
• Else start a new region

Segmentation |41
Watershed Segmentation Algorithm
Watershed algorithm might be used on the gradient image instead of the original
image.

Segmentation |42
Watershed Segmentation Algorithm
Due to noise and other local irregularities of the gradient, oversegmentation
might occur.

A solution is to limit the number of regional minima. Use markers to specify the
only allowed regional minima.

Segmentation |43
Watershed Segmentation Algorithm

Segmentation |44
Use of Motion
Take the difference between a reference image and a subsequent image to determine
the stationary elements and nonstationary image components.

Segmentation |45
Clustering
K-Means Clustering
1. Partition the data points into K clusters randomly. Find the
centroids of each cluster.
2. For each data point:
❑ Calculate the distance from the data point to each cluster.
❑ Assign the data point to the closest cluster.
3. Recompute the centroid of each cluster.
4. Repeat steps 2 and 3 until there is no further change in
the assignment of data points (or in the centroids).

Segmentation |46
K-means Clustering
K-Means Clustering

Segmentation |47
Clustering
K-Means Clustering

◼ RGB vector

 2
K-means clustering minimizes    x j − i 
iclusters  jelements of i' th cluster 

Segmentation |48
Clustering
K-Means Clustering

Original K=5 K=11

Segmentation |49
Clustering
K-Means Clustering

Segmentation |50
Clustering
K-Means Clustering and feature scaling

◼ Features of different types may have different scales.

❑ For example, pixel coordinates on a 100x100 image vs. RGB
color values in the range [0,1].
◼ Problem: Features with larger scales dominate
clustering.
◼ Solution: Scale the features.

Segmentation |51
Clustering
Mean shift Clustering

Segmentation |52
Clustering
Mean shift Clustering
Step 1: Find features for clustering.
Image: usually a, b in Lab color space

Segmentation |53
Clustering
Mean shift Clustering
Step 2: Transformation

Parzen method

Segmentation |54
Clustering
Mean shift Clustering
Step 3: Find peaks and cluster

Segmentation |55
Clustering
Mean shift Clustering

Segmentation |56
Clustering
Hierarchical clustering

Segmentation |57
Active contours

Segmentation |58
Normalized cuts

Segmentation |59
Graph cuts

Segmentation |60
Deep Learning
Autoencoder – U-net

Image Segmentation: Deep Learning vs Traditional [Guide] (v7labs.com)

Segmentation |61
Deep Learning
Autoencoder – U-net

Image Segmentation: Deep Learning vs Traditional [Guide] (v7labs.com)

Segmentation |62
Use of Morphological for regions
Erode

Dilate

Segmentation |63
Use of Morphological for regions
Opening = Erose → Dilate

Closing

Segmentation |64
Advanced terms in segmentation

Image Segmentation: Deep Learning vs Traditional [Guide] (v7labs.com)

Segmentation |65
Advanced terms in segmentation

Modern: Deep Learning

Image Segmentation: Deep Learning vs Traditional [Guide] (v7labs.com)

Segmentation |66