Chapter 10:

Image Segmentation
 Preview
 Segmentation is to subdivide an image
into its constituent regions or objects.
 Segmentation should stop when the
objects of interest in an application have
been isolated.

2
 Principal approaches
 Segmentation algorithms generally are
based on one of 2 basis properties of
intensity values
 discontinuity : to partition an image based
on abrupt changes in intensity (such as
edges)
 similarity : to partition an image into
regions that are similar according to a set
of predefined criteria.

3
 Detection of Discontinuities
 detect the three basic types of gray-
level discontinuities
 points , lines , edges
 the common way is to run a mask through
the image

4
 Point Detection
 a point has been detected at the location
on which the mark is centered if
|R|  T
 where
 T is a nonnegative threshold
 R is the sum of products of the coefficients
with the gray levels contained in the region
encompassed by the mark.
5
 Point Detection
 Note that the mark is the same as the
mask of Laplacian Operation (in chapter 3)
 The only differences that are considered
of interest are those large enough (as
determined by T) to be considered
isolated points.
|R|  T

6
Example

7
 Line Detection

 Horizontal mask will result with max response when a

line passed through the middle row of the mask with a
constant background.
 the similar idea is used with other masks.
 note: the preferred direction of each mask is weighted
with a larger coefficient (i.e.,2) than other possible
directions.
8
 Line Detection
 Apply every masks on the image
 let R1, R2, R3, R4 denotes the response of
the horizontal, +45 degree, vertical and -
45 degree masks, respectively.
 if, at a certain point in the image
|Ri| > |Rj|,
 for all ji, that point is said to be more
likely associated with a line in the
direction of mask i.
9
 Line Detection
 Alternatively, if we are interested in
detecting all lines in an image in the
direction defined by a given mask, we
simply run the mask through the image and
threshold the absolute value of the result.
 The points that are left are the strongest
responses, which, for lines one pixel thick,
correspond closest to the direction
defined by the mask.
10
Example

11
 Edge Detection
 the most common approach for detecting
meaningful discontinuities in gray level.
 we discuss approaches for implementing
 first-order derivative (Gradient operator)
 second-order derivative (Laplacian operator)
 Here, we will talk only about their properties for
edge detection.
 we have introduced both derivatives in chapter 3

12
 Basic Formulation
 an edge is a set of connected pixels that
lie on the boundary between two regions.
 an edge is a “local” concept whereas a
region boundary, owing to the way it is
defined, is a more global idea.

13
Ideal and Ramp Edges

because of optics,
sampling, image
acquisition
imperfection 14
 Thick edge
 The slope of the ramp is inversely proportional to the
degree of blurring in the edge.
 We no longer have a thin (one pixel thick) path.
 Instead, an edge point now is any point contained in the
ramp, and an edge would then be a set of such points
that are connected.
 The thickness is determined by the length of the ramp.
 The length is determined by the slope, which is in turn
determined by the degree of blurring.
 Blurred edges tend to be thick and sharp edges
tend to be thin

15
 First and Second derivatives

the signs of the derivatives

would be reversed for an edge
that transitions from light to
dark 16
 Second derivatives
 produces 2 values for every edge in an
image (an undesirable feature)
 an imaginary straight line joining the
extreme positive and negative values of
the second derivative would cross zero
near the midpoint of the edge. (zero-
crossing property)

17
 Zero-crossing
 quite useful for locating the centers of
thick edges
 we will talk about it again later

18
 Noise Images
 First column: images and
gray-level profiles of a
ramp edge corrupted by
random Gaussian noise of
mean 0 and  = 0.0, 0.1,
1.0 and 10.0,
respectively.
 Second column: first-
derivative images and
gray-level profiles.
 Third column : second-
derivative images and
gray-level profiles.

19
 Keep in mind
 fairly little noise can have such a
significant impact on the two key
derivatives used for edge detection in
images
 image smoothing should be serious
consideration prior to the use of
derivatives in applications where noise is
likely to be present.
20
 Edge point
 to determine a point as an edge point
 the transition in grey level associated with
the point has to be significantly stronger
than the background at that point.
 use threshold to determine whether a value
is “significant” or not.
 the point’s two-dimensional first-order
derivative must be greater than a specified
threshold.

21
 Segmentation Problem
 to assemble edge segments into longer
edges.
 we will talk about it later.

22
  f 
Gx   x 
f      f 
Gradient Operator G y   
 y 

 first derivatives are implemented using

the magnitude of the gradient.
1
f  mag (f )  [G  G ]
2
x
2
y
2

1
commonly approx.
 f  2  f  2  2

      
 x   y  
f  G x  G y
the magnitude becomes nonlinear
23
 Gradient direction
 Let  (x,y) represent the direction angle
of the vector f at (x,y)

 (x,y) = tan-1(Gy/Gx)
 The direction of an edge at (x,y) is
perpendicular to the direction of the
gradient vector at that point

24
Gradient Masks

25
Diagonal edges with Prewitt
and Sobel masks

Sobel masks have

slightly superior
noise-suppression
characteristics which
is an important issue
when dealing with
derivatives.
26
Example

27
Example

28
Example

29
 Laplacian
Laplacian operator
 f ( x, y )  f ( x, y )
2 2

(linear operator)  f 
2

x 2
y 2

 f  [ f ( x  1, y )  f ( x  1, y )
2

 f ( x, y  1)  f ( x, y  1)  4 f ( x, y )]

30
 Laplacian of Gaussian
 Laplacian combined with smoothing as a
precursor to find edges via zero-
crossing.
where r2 = x2+y2, and
r2  is the standard deviation

h( r )  e 2 2

r2
 r 2
  2
 
 h( r )   
2
e
2 2

 
4
 31
positive central term
surrounded by an adjacent negative region (a function of distance)
zero outer region
Mexican hat

the coefficient must be sum to zero 32

 Linear Operation
 second derivation is a linear operation
 thus, 2f is the same as convolving the
image with Gaussian smoothing function
first and then computing the Laplacian of
the result

33
 Example
a). Original image
b). Sobel Gradient
c). Spatial Gaussian
smoothing function
d). Laplacian mask
e). LoG
f). Threshold LoG
g). Zero crossing

34
 Zero crossing & LoG
 Approximate the zero crossing from LoG
image
 to threshold the LoG image by setting all
its positive values to white and all
negative values to black.
 the zero crossing occur between positive
and negative values of the thresholded
LoG.

35
 Zero crossing vs. Gradient
 Attractive
 Zero crossing produces thinner edges
 Noise reduction
 Drawbacks
 Zero crossing creates closed loops.
(spaghetti effect)
 sophisticated computation.
 Gradient is more frequently used.

36
 Edge Linking and Boundary
Detection
 edge detection algorithm are followed by
linking procedures to assemble edge
pixels into meaningful edges.
 Basic approaches
 Local Processing
 Global Processing via the Hough Transform
 Global Processing via Graph-Theoretic
Techniques

37
 Local Processing
 analyze the characteristics of pixels in a
small neighborhood (say, 3x3, 5x5) about
every edge pixels (x,y) in an image.
 all points that are similar according to a
set of predefined criteria are linked,
forming an edge of pixels that share
those criteria.

38
 Criteria
1. the strength of the response of the
gradient operator used to produce the
edge pixel
 an edge pixel with coordinates (x0,y0) in a
predefined neighborhood of (x,y) is similar
in magnitude to the pixel at (x,y) if

|f(x,y) - f (x0,y0) |  E
39
 Criteria
2. the direction of the gradient vector
 an edge pixel with coordinates (x0,y0) in a
predefined neighborhood of (x,y) is similar
in angle to the pixel at (x,y) if

|(x,y) -  (x0,y0) | < A

40
 Criteria
 A point in the predefined neighborhood of (x,y)
is linked to the pixel at (x,y) if both magnitude
and direction criteria are satified.
 the process is repeated at every location in the
image
 a record must be kept
 simply by assigning a different gray level to
each set of linked edge pixels.

41
 find rectangles whose sizes makes
them suitable candidates for license
Example plates

• use horizontal and

vertical Sobel
• operators
eliminate isolated short
segments
• link conditions:
gradient value > 25
42
gradient direction differs < 15
 Hough Transformation (Line)

xy-plane ab-plane or parameter space

yi =axi + b b = - axi + yi
all points (xi ,yi) contained on the same line must have lines in
parameter space that intersect at (a’,b’) 43
 Accumulator cells
 (amax, amin) and (bmax, bmin) are
the expected ranges of slope
and intercept values.
 all are initialized to zero
 if a choice of ap results in
solution bq then we let
A(p,q) = A(p,q)+1
 at the end of the procedure,
value Q in A(i,j) corresponds
to Q points in the xy-plane
lying on the line y = aix+bj b = - axi + yi
44
 -plane

-plane

x cos + y sin = 
q = 90 measured with respect to x-axis
 problem of using equation y = ax + b is that value of a is infinite
for a vertical line.
 To avoid the problem, use equation x cos + y sin  =  to
represent a line instead.
 vertical line has  = 90 with  equals to the positive y-intercept
or  = -90 with  equals to the negative y-intercept
45
range of    2 D
where D is the distance
between corners in the
image

46
 Generalized Hough
Transformation
 can be used for any function of the form
g(v,c) = 0
 v is a vector of coordinates
 c is a vector of coefficients

47
 Hough Transformation (Circle)
 equation:
(x-c1)2 + (y-c2)2 = c32
 three parameters (c1, c2, c3)
 cube like cells
 accumulators of the form A(i, j, k)
 increment c1 and c2 , solve of c3 that satisfies
the equation
 update the accumulator corresponding to the
cell associated with triplet (c1, c2, c3)
48
 Edge-linking based on Hough
Transformation
1. Compute the gradient of an image and
threshold it to obtain a binary image.
2. Specify subdivisions in the -plane.
3. Examine the counts of the accumulator
cells for high pixel concentrations.
4. Examine the relationship (principally
for continuity) between pixels in a
chosen cell.

49
 Continuity
 based on computing the distance between
disconnected pixels identified during
traversal of the set of pixels
corresponding to a given accumulator cell.
 a gap at any point is significant if the
distance between that point and its
closet neighbor exceeds a certain
threshold.
50
link criteria:
1). the pixels belonged to one of the set of pixels
linked according to the highest count
2). no gaps were longer than 5 pixels
51
Thresholding
image with dark image with dark
background and background and
a light object two light objects

52
 Multilevel thresholding
 a point (x,y) belongs to
 to an object class if T1 < f(x,y)  T2
 to another object class if f(x,y) > T2
 to background if f(x,y)  T1
 T depends on
 only f(x,y) : only on gray-level values  Global
threshold
 both f(x,y) and p(x,y) : on gray-level values and its
neighbors  Local threshold

53
 easily use global thresholding
object and background are separated

The Role of Illumination

f(x,y) = i(x,y) r(x,y)

a). computer generated

reflectance function
b). histogram of
reflectance function
c). computer generated
illumination function
(poor)
d). product of a). and c).
e). histogram of product
image
difficult to segment 54
 Illumination compensation
 compensate nonuniformity by projecting
the illumination pattern onto a constant,
white reflective surface.
 thus,
g(x,y) = ki(x,y)
 where k is a constant that depends on
the surface and i(x,y) is the illumination
pattern

55
 Illumination compensation
 for any image f(x,y) = i(x,y) r(x,y)
obtained with the same illumination
function
 simply dividing f(x,y) by g(x,y)
 yields a normalized function h(x,y)
h(x,y) = f(x,y)/g(x,y) = r(x,y)/k
 thus, we can use a single threshold T/k
to segment h(x,y)

56
Basic Global Thresholding

use T midway
between the max
and min gray levels
generate binary image
57
 Basic Global Thresholding
 based on visual inspection of histogram
1. Select an initial estimate for T.
2. Segment the image using T. This will produce two
groups of pixels: G1 consisting of all pixels with gray
level values > T and G2 consisting of pixels with gray
level values  T
3. Compute the average gray level values 1 and 2 for
the pixels in regions G1 and G2
4. Compute a new threshold value
5. T = 0.5 (1 + 2)
6. Repeat steps 2 through 4 until the difference in T in
successive iterations is smaller than a predefined
parameter To.
58
Example: Heuristic method
note: the clear valley
of the histogram and
the effective of the
segmentation
between object and
background

T0 = 0
3 iterations
with result T = 125

59
 Basic Adaptive Thresholding
 subdivide original image into small areas.
 utilize a different threshold to segment
each subimages.
 since the threshold used for each pixel
depends on the location of the pixel in
terms of the subimages, this type of
thresholding is adaptive.

60
Example : Adaptive Thresholding

61
 Further subdivision
a). Properly and improperly
segmented subimages from previous
example
b)-c). corresponding histograms
d). further subdivision of the
improperly segmented subimage.
e). histogram of small subimage at
top
f). result of adaptively segmenting d).

62
Optimal Global and Adaptive
Thresholding

p ( z )  P1 p1 ( z )  P2 p2 ( z )
P1  P2  1 63
Probability of erroneously
T
E1 (T )  

p 2 ( z ) dz

T
E2 (T )  

p1 ( z ) dz

E (T )  P2 E1 (T )  P1 E2 (T )
64
 Minimum error
Differentiating E(T) with respect to T (using Leibniz’s
rule) and equating the result to 0

dE (T ) d ( P2 E1 (T )  P1 E2 (T ))
 0
dT dT if P = P then 1 2
the optimum threshold
is where the curve
find T which makes p1(z) and p2(z) intersect
P1 p1 (T )  P2 p2 (T )
65
 Gaussian density
Example: use PDF = Gaussian density : p1(z) and p2(z)

p ( z )  P1 p1 ( z )  P2 p2 ( z )
( z  1 ) 2 ( z 2 )2
 
P1 2 12 P2 2 22
 e  e
where 2  1 2  2
• 1 and 12 are the mean and variance of the Gaussian
density of one object
• 2 and 22 are the mean and variance of the Gaussian
density of the other object 66
 Quadratic equation
AT  BT  C  0
2

where A   2
1
2
2

B  2( 1   2 )
2
2
2
1

C        2 2 ln( 2 P1 /  1 P2 )
2
1
2
2
2
2
2
1
2
1
2
2
if P1 = P2 or  = 0
1   2  2
 P2  then the optimal
T  ln   threshold is the
2 1   2  P1  average of the
means 67
Example

68
Histogram of the example

69
Example:
Boundary superimposed

70
 Boundary Characteristic for
Histogram Improvement and Local
Thresholding
light object of dark background

0 if f  T

s ( x , y )   if f  T and  f  0
2

 if f  T and  2 f  0

 local processing due to using gradient of each small area
 (…)(-,+)(0 or +)(+,-)(…)
 all pixels that are not on an edge are labeled 0
 all pixels that are on the dark side of an edge are
labeled +
 all pixels that are on the light side an edge are labeled -
71
Example

72
 Histogram of Gradient

select T at the midpoint of the valley

73
Result of applying T

74
 Region-Based Segmentation
 Basicn Formulation
(a )  Ri  R
i 1

(b) Ri is a connected region, i  1, 2 , ..., n

(c) Ri  R j   for all i and j, i  j
(d ) P(Ri )  TRUE for i  1, 2, ..., n
(e) P(Ri  R j )  FALSE for i  j
P(Ri) is a logical predicate property defined over the points in set R i
ex. P(Ri) = TRUE if all pixel in Ri have the same gray level 75
 Region Growing
 start with a set of “seed” points
 growing by appending to each seed those
neighbors that have similar properties
such as specific ranges of gray level

76
 select all seed points with gray level 255

Region Growing
criteria:
1. the absolute gray-
level difference
between any pixel
and the seed has to
be less than 65
2. the pixel has to be
8-connected to at
least one pixel in
that region (if more,
the regions are
merged) 77
Histogram of fig 10.40 a)

used to find the

criteria of the
difference gray-level
between each pixels
and the seeds

78
 Region splitting and merging

Quadtree
1. Split into 4 disjoint quadrants any region Ri for which
P(Ri) = FALSE
2. Merge any adjacent region Rj and Rk for which
P(Ri  Rk ) = TRUE
3. Stop when no further merging or splitting is possible. 79
 Example

P(Ri) = TRUE if at least 80% of the pixels in Ri have the

property |zj-mi|  2i,
where
zj is the gray level of the jth pixel in Ri
mi is the mean gray level of that region
 is the standard deviation of the gray levels in80R