Image Segmentation

1
What is segmentation?
 Dividing image into its constituent regions
or objects

2
 Preview

 Segmentation is to subdivide an image

into its component regions or objects.
Segmentation should stop when
the objects of interest in an application
have been isolated.

3
 Segmentation and Grouping:
• Basic Segmentation
– Thresholding
– Connected component algorithm
• Contour Detection
– Active Contors- Snakes
• Split and Merge
– Region Segmentation (Split and Merge)
• Motion Segmentation
– Frame Differencing- 3 Frame Differencing
• Grouping
– Kmeans clustering, K- Nearest Neighbour
• Graph cut and Energy based Methods
– Graph Cut Segmentation
4
 Thresholding

image with dark image with dark

background and background and
a light object two light objects

5
 Multilevel thresholding

 a point (x,y) belongs to

 to an object class if T1 < f(x,y)  T2
 to another object class if f(x,y) > T2
 to background if f(x,y)  T1
 T depends on
 only f(x,y) : only on gray-level values  Global
threshold
 both f(x,y) and p(x,y) : on gray-level values and its
neighbors  Local threshold
6
 Basic Global Thresholding
based on visual inspection of histogram

1. Select an initial estimate for T.

2. Segment the image using T. This will produce two
groups of pixels: G1 consisting of all pixels with gray
level values > T and G2 consisting of pixels with gray
level values  T
3. Compute the average gray level values 1 and  for
the pixels in regions G1 and G2 2

4. Compute a new threshold value

5. T = 0.5 (1 + 2)
6. Repeat steps 2 through 4 until the difference in T in
successive iterations is smaller than a predefined
parameter To.
7
 Principal approaches

 Segmentation algorithms generally are

based on one of 2 basis properties of
intensity values
 discontinuity : to partition an image based
on sharp changes in intensity (such as edges)
similarity : to partition an image into
regions that are similar according to a set
of predefined criteria.

8
 Detection of Discontinuities
 detect the three basic types of
gray-
level discontinuities
 points , lines , edges
 the common way is to run a mask
through the image

9
 Keep in mind

 fairly little noise can have such a

significant impa on the two key
derivatives usedct for edge detection in
images
 image smoothing should be serious
consideration prior to the use of
derivatives in applications where noise is
likely to be present.
10
 Edge point

 to determine a point as an edge point

 the transition in grey level associated with
the point has to be significantly stronger
than the background at that point.
 use threshold to determine whether a
value is “significant” or not.
 the point’s two-dimensional first-
order derivative must be greater than a
specified threshold.

11
easily use global thresholding
object and background are separated

The Role of
Illumination

12
40
 Thresholding
The Role of Illumination

r(x, y) i(x, y)
(a) (c)
(d (e)
)

f (x, y)  i(x, y)r(x, y)

13
 Thresholding
Basic Adaptive Thresholding

14
 Thresholding
Basic Adaptive Thresholding

How to solve this problem?

15
 Thresholding
Basic Adaptive Thresholding

Answer: subdivision

16
Basic Global Thresholding

use T midway
between the max and
min gray levels
generate binary image
17
 Basic Global Thresholding

 based on visual inspection of histogram

1. Select an initial estimate for T.
2. Segment the image using T. This will produce two
groups of pixels: G1 consisting of all pixels with gray
level values > T and G2 consisting of pixels with gray
level values  T
3. Compute the average gray level values 1 and  for
the pixels in regions G1 and G2 2

4. Compute a new threshold value

5. T = 0.5 (1 + 2)
6. Repeat steps 2 through 4 until the difference in T in
successive iterations is smaller than a predefined
parameter To.
18
Example: Heuristic method

note: the clear valley

of the histogram and
the effective of the
segmentation
between object and
background

T0 = 0
3 iterations
with result T = 125

19
 Basic Adaptive Thresholding

 subdivide original image into small areas.

 utilize a different threshold to segment

each subimages.
 since the threshold used for each pixel

depends on the location of the pixel in

terms of the subimages, this type of
thresholding is adaptive.

20
Example : Adaptive Thresholding

21
 Further subdivision
50
a). Properly and improperly segmented
subimages from previous example
b)-c). corresponding histograms d).
further subdivision of the improperly
segmented subimage. e). histogram of
small subimage at top
f). result of adaptively segmenting d).

22
 Simple Thresholding

This simple algorithm works well for histograms with a reasonably clear valley
between peaks.

23
Using the averaging method, find the threshold of the 4 × 4 8-bit image. Let the initial
value of the threshold be the average of the gray levels of the image. The iteration
stops when the difference between two consecutive threshold values becomes less
than 0.5.

24
Connected Component Labelling

25
 One component at a time

It is assumed that the input image is a binary image, with pixels being either
background or foreground and that the connected components in the foreground
pixels are desired. The algorithm steps can be written as:

1.Start from the first pixel in the image. Set current label to 1. Go to (2).

2.If this pixel is a foreground pixel and it is not already labelled, give it the current
label and add it as the first element in a queue, then go to (3). If it is a background
pixel or it was already labelled, then repeat (2) for the next pixel in the image.

3.Pop out an element from the queue, and look at its neighbours (based on any
type of connectivity). If a neighbour is a foreground pixel and is not already labelled,
give it the current label and add it to the queue. Repeat (3) until there are no more
elements in the queue.

4.Go to (2) for the next pixel in the image and increment current label by 1.

26
 Two-pass

Conditions to check:
Does the pixel to the left (West) have the same value as the current pixel?
Yes – We are in the same region. Assign the same label to the current pixel
No – Check next condition

Do both pixels to the North and West of the current pixel have the same value as the
current pixel but not the same label?
Yes – We know that the North and West pixels belong to the same region and
must be merged. Assign the current pixel the minimum of the North and West
labels, and record their equivalence relationship
No – Check next condition

Does the pixel to the left (West) have a different value and the one to the North the
same value as the current pixel?
Yes – Assign the label of the North pixel to the current pixel
No – Check next condition

Do the pixel's North and West neighbors have different pixel values than current
pixel?
Yes – Create a new label id and assign it to the current pixel
27
28
 Labelling connected components - Example

Suppose the binary image is the following:

29
 The first pass
Here is the same image, represented as a
matrix.

check the top and left pixels

30
check the pixel in row 1, column 2, or simply pixel (1, 2). It does
has a pixel to its left. 
copy its label. The next pixel, (1, 3) is a background pixel. So
simply skip it and let it be black (a label of 0).

Next comes the pixel (1, 4). There are no pixels above it. But
the pixel to its left is a background pixel. So create a new
label. Mark (1, 4) with label 2 (shown as dark yellow)

31
Now the next two pixels, (1, 5) and(1, 6) will have pixel to their left.
So, they get the label 2 as well. (1, 7) is background so ignore it. At
(1,8) create a new label again (pixel above does not exist, and pixel
to left is a background pixel).
So, after completing row 1, result looks like this:

After row 1 is complete

32
Now the pixel (2, 1) does not have anything to its left. But it does
have a pixel just above it. Its label is '1', so copy that. Similarly for
(2, 2). In fact, the same thing holds for the entire row. All pixels in
the second row have a pixel just above them. So the result looks
like this:

Completion of row 2

33
In the third row, pixels (3, 1) (3,2 ) and (3, 3) are quite straight forward. They get
the label '1'. But pixel (3, 4) is a tricky one. You have pixel both above and to the
left. And both have different labels.
Which one do you choose? How do you handle the sudden "realization" of the
connectedness of labels 1 and 2.
Take the smaller label (in this case '1') and put that on (3, 4). And, also store
that 2 (the numerically larger label) is a child of 1 (using the union-find data
structure).

Resolving the dilemma at (3, 4)

34
final result of pass 1

After 4 rows

Completing 5 rows

35
 Row 6, done

Completing row 7, note the new label

36
First pass complete. Note the new label.

37
 The second pass

It starts with pixel (1, 1). It checks the union-find data structure for the label
'1'. On checking, it figures out that '1' is not a child of any other labels. It is a
root itself. So it moves on.
Similarly for (1, 2). For the pixel (1, 3) it is not concerned because it is a
background pixel.
Now on pixel (1,4) it checks the data structure for '2'. It notices that '2' is a
child of '1'. Then, it checks for '1'. It notices '1' is a root. So, it replaces the
label of (1, 4) with '1':

Converting a '2' into a '1' 38

Similarly, the process goes on for the entire row. The result after
completing 4 rows is:

Completing till row 4

39
Now, row 7 undergoes changes:

Completing row 7

And after doing row 8, here's the final result:

The final result

40
Example

41
42
Otsu’s Thresholding

B.Sathya Bama
• Otsu's method, named after Nobuyuki Otsu , is
used to automatically perform clustering-based
image thresholding, or, the reduction of a
graylevel image to a binary image.

• The algorithm assumes that the image contains

two classes of pixels following bi-modal
histogram (foreground pixels and background
pixels), it then calculates the optimum threshold
separating the two classes so that their combined
spread (intra-class variance) is minimal, or
equivalently (because the sum of pairwise squared
distances is constant), so that their inter-class
variance is maximal.
In Otsu's method we exhaustively search for the
threshold that minimizes the intra-class variance (the
variance within the class), defined as a weighted sum of
variances of the two classes:
1. Compute histogram and probabilities of each intensity level.
2. Set up initial class probability and initial class means.
3. Step through all possible thresholds maximum intensity.
4. Update qi and μi.
5. Compute between class variance.
6. Desired threshold corresponds to the maximum value of between class
variance.
 Otsu’s
• Compute histogram and probabilities of each
intensity level.
• Compute weight and variance of each class
 calculate the 'Within-Class Variance'. This is simply the sum of
the two variances multiplied by their associated weights.

• This final value is the 'sum of weighted variances' for the

threshold value 3. This same calculation needs to be
performed for all the possible threshold values 0 to 5.
for the threshold equal to 3, as well as being
used for the example, also has the lowest sum
of weighted variances. Therefore, this is the
final selected threshold. All pixels with a level
less than 3 are background, all those with a
level equal to or greater than 3 are
foreground. As the images in the table show,
this threshold works well.