Image and Video Processing

Image Segmentation-01

Computer Vision & Biometrics Lab, Indian Institute of Information Technology, Allahabad
 Image and Video Processing

Fundamentals
• Let R represent the entire spatial region
occupied by an image. Image segmentation is a
process that partitions R into n sub-regions, R1,
R2, …, Rn, such that
n
(a)  Ri  R.
i 1

(b) Ri is a connected set. i  1, 2, ..., n.

(c) Ri  R j  .
(d) ( Ri )  TRUE for i  1, 2, ..., n.
(e) ( Ri  R j )  FALSE for any adjacent regions
Ri and R j . 2

Computer Vision & Biometrics Lab, Indian Institute of Information Technology, Allahabad
 Image and Video Processing

Computer Vision & Biometrics Lab, Indian Institute of Information Technology, Allahabad
 Image and Video Processing

Background

• First-order derivative

f
 f '( x)  f ( x  1)  f ( x)
x

• Second-order derivative
2 f
 f ( x  1)  f ( x  1)  2 f ( x)
x 2

Computer Vision & Biometrics Lab, Indian Institute of Information Technology, Allahabad
 Image and Video Processing

Computer Vision & Biometrics Lab, Indian Institute of Information Technology, Allahabad
 Image and Video Processing

Characteristics of First and Second Order Derivatives

• First-order derivatives generally produce thicker edges in
image

• Second-order derivatives have a stronger response to fine

detail, such as thin lines, isolated points, and noise

• Second-order derivatives produce a double-edge response at

ramp and step transition in intensity

• The sign of the second derivative can be used to determine

whether a transition into an edge is from light to dark or dark
to light
6

Computer Vision & Biometrics Lab, Indian Institute of Information Technology, Allahabad
 Image and Video Processing

Detection of Isolated Points

• The Laplacian
 2
f  2
f
 f ( x, y )  2  2
2

x y
 f ( x  1, y )  f ( x  1, y )  f ( x, y  1)  f ( x, y  1)
4 f ( x, y )
9

1 if | R( x, y ) | T R   wk zk
g ( x, y )   k 1

0 otherwise
7

Computer Vision & Biometrics Lab, Indian Institute of Information Technology, Allahabad
 Image and Video Processing

Computer Vision & Biometrics Lab, Indian Institute of Information Technology, Allahabad
 Image and Video Processing

Line Detection
• Second derivatives to result in a stronger
response and to produce thinner lines than first
derivatives

• Double-line effect of the second derivative must

be handled properly

Computer Vision & Biometrics Lab, Indian Institute of Information Technology, Allahabad
 Image and Video Processing

10

Computer Vision & Biometrics Lab, Indian Institute of Information Technology, Allahabad
 Image and Video Processing

Detecting Line in Specified Directions

• Let R1, R2, R3, and R4 denote the responses of the masks in Fig. 10.6.
If, at a given point in the image, |Rk|>|Rj|, for all j≠k, that point is
said to be more likely associated with a line in the direction of mask
k.
11

Computer Vision & Biometrics Lab, Indian Institute of Information Technology, Allahabad
 Image and Video Processing

12

Computer Vision & Biometrics Lab, Indian Institute of Information Technology, Allahabad
 Image and Video Processing

Edge Detection
• Edges are pixels where the brightness function
changes abruptly
• Edge models

13

Computer Vision & Biometrics Lab, Indian Institute of Information Technology, Allahabad
 Image and Video Processing

14

Computer Vision & Biometrics Lab, Indian Institute of Information Technology, Allahabad
 Image and Video Processing

15

Computer Vision & Biometrics Lab, Indian Institute of Information Technology, Allahabad
 Image and Video Processing

16

Computer Vision & Biometrics Lab, Indian Institute of Information Technology, Allahabad
 Image and Video Processing

Basic Edge Detection by Using First-Order Derivative

 f 
 g x   x 
f  grad ( f )      
 g y   f 
 y 
The magnitude of f
M ( x, y )  mag(f )  g x 2  g y 2
The direction of f
 gx 
 ( x, y )  tan   1

 g y 
The direction of the edge
   - 90 17

Computer Vision & Biometrics Lab, Indian Institute of Information Technology, Allahabad
 Image and Video Processing

Basic Edge Detection by Using First-Order Derivative

 f 
 g x   x 
Edge normal: f  grad ( f )      
 g y   f 
 y 
Edge unit normal: f / mag(f )

In practice,sometimes the magnitude is approximated by

f f  f f 
mag(f )= + or mag(f )=max  | |,| | 
x y  x y 
18

Computer Vision & Biometrics Lab, Indian Institute of Information Technology, Allahabad
 Image and Video Processing

19

Computer Vision & Biometrics Lab, Indian Institute of Information Technology, Allahabad
 Image and Video Processing

20

Computer Vision & Biometrics Lab, Indian Institute of Information Technology, Allahabad
 Image and Video Processing

21

Computer Vision & Biometrics Lab, Indian Institute of Information Technology, Allahabad
 Image and Video Processing

22

Computer Vision & Biometrics Lab, Indian Institute of Information Technology, Allahabad
 Image and Video Processing

23

Computer Vision & Biometrics Lab, Indian Institute of Information Technology, Allahabad
 Image and Video Processing

24

Computer Vision & Biometrics Lab, Indian Institute of Information Technology, Allahabad
 Image and Video Processing

25

Computer Vision & Biometrics Lab, Indian Institute of Information Technology, Allahabad
 Image and Video Processing

26

Computer Vision & Biometrics Lab, Indian Institute of Information Technology, Allahabad
 Image and Video Processing

27

Computer Vision & Biometrics Lab, Indian Institute of Information Technology, Allahabad
 Image and Video Processing

28

Computer Vision & Biometrics Lab, Indian Institute of Information Technology, Allahabad
 Image and Video Processing

Advanced Techniques for Edge Detection

• The Marr-Hildreth edge detector
x2  y 2

G ( x, y )  e 2 ,  : space constant.
2

Laplacian of Gaussian (LoG)

 2 G ( x, y )  2 G ( x, y )
 G ( x, y ) 
2

x 2
y 2
   x  2 2     y  x  y2 
x y 2 2 2 2

  2e    2 e 2 
x    y   
x2  y 2 x2  y 2
x 2
1   y 1 
2 
  4  2 e 2 2
  4  2 e 2 2

     
x2  y 2
 x2  y 2   2  
  e 2 2

  4

29

Computer Vision & Biometrics Lab, Indian Institute of Information Technology, Allahabad
 Image and Video Processing

30

Computer Vision & Biometrics Lab, Indian Institute of Information Technology, Allahabad
 Image and Video Processing

Marr-Hildreth Algorithm
1. Filter the input image with an nxn Gaussian
lowpass filter. N is the smallest odd integer
greater than or equal to 6
2.
1. Compute the Laplacian of the image resulting
from step1
g ( x, y )    G ( x, y ) f ( x, y ) 
2

2. Find the zero crossing of the image from step

31

Computer Vision & Biometrics Lab, Indian Institute of Information Technology, Allahabad
 Image and Video Processing

32

Computer Vision & Biometrics Lab, Indian Institute of Information Technology, Allahabad
 Image and Video Processing

The Canny Edge Detector

• Optimal for step edges corrupted by white noise.

• The Objective

1. Low error rate

The edges detected must be as close as possible to the true edge

2. Edge points should be well localized

The edges located must be as close as possible to the true edges

3. Single edge point response

The number of local maxima around the true edge should be
33
minimum
Computer Vision & Biometrics Lab, Indian Institute of Information Technology, Allahabad
 Image and Video Processing

The Canny Edge Detector: Algorithm (1)

Let f ( x, y ) denote the input image and
G ( x, y ) denote the Gaussian function:
x2  y 2

G ( x, y )  e 2 2

We form a smoothed image, f s ( x, y ) by

convolving G and f :
f s ( x , y )  G ( x, y ) f ( x, y)
34

Computer Vision & Biometrics Lab, Indian Institute of Information Technology, Allahabad
 Image and Video Processing

The Canny Edge Detector: Algorithm(2)

Compute the gradient magnitude and direction (angle):
M ( x, y )  g x 2  g y 2
and
 ( x, y )  arctan( g y / g x )
where g x  f s / x and g y  f s / y

Note: any of the filter mask pairs in Fig.10.14 can be used

to obtain g x and g y
35

Computer Vision & Biometrics Lab, Indian Institute of Information Technology, Allahabad
 Image and Video Processing

The Canny Edge Detector: Algorithm(3)

The gradient M ( x, y ) typically contains wide ridge around
local maxima. Next step is to thin those ridges.
Nonmaxima suppression:
Let d1 , d 2 , d3 , and d 4 denote the four basic edge directions for
a 3  3 region: horizontal, -45o , vertical,+45 , respectively.
1. Find the direction d k that is closest to  ( x, y ).
2. If the value of M ( x, y ) is less than at least one of its two
neighbors along d k , let g N ( x, y )  0 (suppression);
otherwise, let g N ( x, y )  M ( x, y ) 36

Computer Vision & Biometrics Lab, Indian Institute of Information Technology, Allahabad
 Image and Video Processing

37

Computer Vision & Biometrics Lab, Indian Institute of Information Technology, Allahabad
 Image and Video Processing

The Canny Edge Detector: Algorithm(4)

The final operation is to threshold g N ( x, y ) to reduce

false edge points.

Hysteresis thresholding:
g NH ( x, y )  g N ( x, y )  TH
g NL ( x, y )  g N ( x, y )  TL
and
g NL ( x, y )  g NL ( x, y )  g NH ( x, y )
38

Computer Vision & Biometrics Lab, Indian Institute of Information Technology, Allahabad
 Image and Video Processing

The Canny Edge Detector: Algorithm(5)

Depending on the value of TH , the edges in g NH ( x, y )
typically have gaps. Longer edges are formed using
the following procedure:
(a). Locate the next unvisited edge pixel, p, in g NH ( x, y ).
(b). Mark as valid edge pixel all the weak pixels in g NL ( x, y)
that are connected to p using 8-connectivity.
(c). If all nonzero pixel in g NH ( x, y) have been visited go to
step (d), esle return to (a).
(d). Set to zero all pixels in g NL ( x, y) that were not marked as
valid edge pixels. 39

Computer Vision & Biometrics Lab, Indian Institute of Information Technology, Allahabad
 Image and Video Processing

The Canny Edge Detection: Summary

• Smooth the input image with a Gaussian filter

• Compute the gradient magnitude and angle

images

• Apply nonmaxima suppression to the gradient

magnitude image

• Use double thresholding and connectivity

analysis to detect and link edges 40

Computer Vision & Biometrics Lab, Indian Institute of Information Technology, Allahabad
 Image and Video Processing

41
TL  0.04;TH  0.10;  4 and a mask of size 25  25
Computer Vision & Biometrics Lab, Indian Institute of Information Technology, Allahabad
 Image and Video Processing

TL  0.05; TH  0.15;  2 and a mask of size 13 13

42

Computer Vision & Biometrics Lab, Indian Institute of Information Technology, Allahabad
 Image and Video Processing

43

Computer Vision & Biometrics Lab, Indian Institute of Information Technology, Allahabad
 Image and Video Processing

44

Computer Vision & Biometrics Lab, Indian Institute of Information Technology, Allahabad
 Image and Video Processing

Topic

Computer Vision & Biometrics Lab, Indian Institute of Information Technology, Allahabad 45