Image Segmentation

Before reading this presentation

file, go through our
class recorded content
 Image Segmentation

• Image segmentation divides an image into regions that are

connected and have some similarity within the region and
some difference between adjacent regions.
• The goal is usually to find individual objects in an image.
• For the most part there are fundamentally two kinds of
approaches to segmentation: discontinuity and similarity.
– Similarity may be due to pixel intensity, color or texture.
– Differences are sudden changes (discontinuities) in any of these, but
especially sudden changes in intensity along a boundary line, which is
called an edge.
 Detection of Discontinuities

• There are three kinds of discontinuities of intensity: points,

lines and edges.
• The most common way to look for discontinuities is to scan a
small mask over the image. The mask determines which kind
of discontinuity to look for.
9
R = w1 z1 + w2 z2 + ... + w9 z9 =  wi zi
i =1
 Detection of Discontinuities
Point Detection

R T
where T : a nonnegative threshold
 Detection of Discontinuities
Line Detection

• Only slightly more common than point detection is to find a

one pixel wide line in an image.
• For digital images the only three point straight lines are only
horizontal, vertical, or diagonal (+ or –45).
Detection of Discontinuities
Line Detection
Detection of Discontinuities
Edge Detection
Detection of Discontinuities
Edge Detection
Detection of Discontinuities
Edge Detection
Detection of Discontinuities
Edge Detection
 Detection of Discontinuities
Gradient Operators

• First-order derivatives:
– The gradient of an image f(x,y) at location (x,y) is defined
as the vector:
 x
G  f
x

f =   =  f 
 y   y 
G
– The magnitude of this vector: f = mag (f ) = G + G 2
x
2
y 
1
2

 Gx 
– The direction of this vector:  ( x, y ) = tan  
−1

 Gy 
 Detection of Discontinuities
Gradient Operators

Roberts cross-gradient operators

Prewitt operators

Sobel operators
 Detection of Discontinuities
Gradient Operators

Prewitt masks for

detecting diagonal edges

Sobel masks for

detecting diagonal edges
 Detection of Discontinuities
Gradient Operators: Example

f  G x + G y
 Detection of Discontinuities
Gradient Operators: Example
 Detection of Discontinuities
Gradient Operators: Example
 Detection of Discontinuities
Gradient Operators

• Second-order derivatives: (The Laplacian)

– The Laplacian of an 2D function f(x,y) is defined as
2 f 2 f
 f = 2 + 2
2

x y
– Two forms in practice:
 Detection of Discontinuities
Gradient Operators

• Consider the function: A Gaussian function

r2
−
h( r ) = −e 2 2
where r 2 = x 2 + y 2
and  : the standard deviation
• The Laplacian of h is
r2
 r −   − 2 2
2 2 The Laplacian of a
 h( r ) = − 
2
e Gaussian (LoG)
 
4

• The Laplacian of a Gaussian sometimes is called the Mexican
hat function. It also can be computed by smoothing the image
with the Gaussian smoothing mask, followed by application of
the Laplacian mask.
Detection of Discontinuities
Gradient Operators
 Detection of Discontinuities
Gradient Operators: Example

Sobel gradient

Gaussian smooth function Laplacian mask

 Detection of Discontinuities
Gradient Operators: Example
 Edge Linking and Boundary Detection
Local Processing

• Two properties of edge points are useful for edge linking:

– the strength (or magnitude) of the detected edge points
– their directions (determined from gradient directions)
• This is usually done in local neighborhoods.
• Adjacent edge points with similar magnitude and direction are
linked.
• For example, an edge pixel with coordinates (x0,y0) in a
predefined neighborhood of (x,y) is similar to the pixel at (x,y)
if
f ( x, y ) − ( x0 , y0 )  E , E : a nonnegativ e threshold
 ( x, y ) −  ( x0 , y0 )  A, A : a nonegative angle threshold
 Edge Linking and Boundary Detection
Local Processing: Example

In this example,
we can find the
license plate
candidate after
edge linking
process.
 Edge Linking and Boundary Detection
Global Processing via the Hough Transform

• Hough transform: a way of finding edge points in an image

that lie along a straight line.
• Example: xy-plane v.s. ab-plane (parameter space)
yi = axi + b
 Thresholding

• Assumption: the range of intensity levels covered by objects

of interest is different from the background.
1 if f ( x, y )  T
g ( x, y ) = 
0 if f ( x, y )  T

Single threshold Multiple threshold

 Thresholding
The Role of Illumination
 Thresholding
The Role of Illumination

r ( x, y ) (a) (c) i ( x, y )

(d) (e)

f ( x, y ) = i ( x, y ) r ( x, y )
 Thresholding
Basic Global Thresholding
 Thresholding
Basic Global Thresholding
 Thresholding
Basic Adaptive Thresholding
 Thresholding
Basic Adaptive Thresholding

How to solve this problem?

 Thresholding
Basic Adaptive Thresholding

Answer: subdivision
 Thresholding
Optimal Global and Adaptive Thresholding

• This method treats pixel values as probability density functions.

• The goal of this method is to minimize the probability of
misclassifying pixels as either object or background.
• There are two kinds of error:
– mislabeling an object pixel as background, and
– mislabeling a background pixel as object.
 Thresholding
Use of Boundary Characteristics
 Thresholding
Thresholds Based on Several Variables

Color image
 Region-Based Segmentation

• Edges and thresholds sometimes do not give good

results for segmentation.
• Region-based segmentation is based on the
connectivity of similar pixels in a region.
– Each region must be uniform.
– Connectivity of the pixels within the region is very
important.
• There are two main approaches to region-based
segmentation: region growing and region splitting.
 Region-Based Segmentation
Basic Formulation

• Let R represent the entire image region.

• Segmentation is a process that partitions R into subregions,
R1,R2,…,Rn, such that
n
(a)  Ri = R
i =1
(b) Ri is a connected region, i = 1,2,..., n
(c) Ri  R j =  for all i and j , i  j
(d) P( Ri ) = TRUE for i = 1,2,..., n
(e) P( Ri  R j ) = FALSE for any adjacent regions Ri and R j
where P(Rk): a logical predicate defined over the points in set Rk
For example: P(Rk)=TRUE if all pixels in Rk have the same gray
level.
Region-Based Segmentation
Region Growing
 Region-Based Segmentation
Region Growing

• Fig. 10.41 shows the histogram of Fig. 10.40 (a). It is difficult to

segment the defects by thresholding methods. (Applying region
growing methods are better in this case.)

Figure 10.40(a) Figure 10.41

 Region-Based Segmentation
Region Splitting and Merging

• Region splitting is the opposite of region growing.

– First there is a large region (possible the entire image).
– Then a predicate (measurement) is used to determine if
the region is uniform.
– If not, then the method requires that the region be split
into two regions.
– Then each of these two regions is independently tested by
the predicate (measurement).
– This procedure continues until all resulting regions are
uniform.
 Region-Based Segmentation
Region Splitting

• The main problem with region splitting is determining where to

split a region.
• One method to divide a region is to use a quadtree structure.
• Quadtree: a tree in which nodes have exactly four descendants.