Unsupervised Texture Segmentation

Via
Gaussian Markov Random Field (GMRF) Model
And
Filter Bank Decomposition

by

Zhou Shaohua
March 15, 2001

A project report for partial requirement of ENEE739J, Image Understanding

Unsupervised Texture Segmentation via GMRF Model and Filter Bank Decomposition

Abstract
This report deals with texture segmentation under no supervision. With absence
of training patterns in unsupervised segmentation, typically, K-Means clustering
technique is invoked to make the segmentation in the feature space. Two kinds of features
are extracted from the mosaic texture in this study. GMRF model is used to capture the
statistical properties of textures, while filter bank decomposition to attain the frequency
properties.

Unsupervised Texture Segmentation via GMRF Model and Filter Bank Decomposition

Chapter 1: Introduction
Due to its widespread presence all over the world, texture is served as an
important cue in human visual system (HVS). Texture-related research topics, such as
texture analysis, classification, and segmentation, attract a significant amount of attention
in the literature. In this report, we focus on the problem of texture segmentation.
Despite of long history of research effort on texture, no exact definition of texture
can be found in the literature, because texture has multi-fold characteristics, which are
hard to be described in one single sentence. For instance, some textures have strong
structural organization, some have random natures, and some have both properties. Thus,
approaches to texture analysis, according to their emphases, can be roughly divided into
three groups: structural, statistical [1, 2, 3], and spectral [4, 5] ones. Also, there are
alternative approaches, such as normalized cut [6], available in the literature. In this
report, we study both statistical and spectral approaches. In particular, GRMF model is a
statistical approach, and filter bank decomposition is a spectral one.
Texture segmentation can be achieved under no supervision. With absence of
training patterns in unsupervised segmentation, unsupervised clustering techniques such
as K-means algorithm are common practices. However, clustering image intensities
produces nothing meaningful, especially for textural images. Therefore, feature should be
extracted before clustering. The ideal feature, which is hard to extract in practice, should
possess the capability of minimizing within-cluster distance while maximizing betweencluster one. Two features proposed in this report are (i) GMRF parameters, and (ii) postprocessed results of filter responses after filter bank decomposition.
The rest of this report is organized as follows. Chapter 2 introduces the GMRF
model and its implemental details. Chapter 3 describes the filter bank approach and
discusses the issues of filter selection and feature extraction. Experimental results are also
included in according chapters. Chapter 4 concludes the report.

Unsupervised Texture Segmentation via GMRF Model and Filter Bank Decomposition

Chapter 2: Gaussian Markov Random Field (GMRF) Model

Markov random field (MRF) is an extension of 1-D causal Markov chain to 2-D,
unnecessarily causal any more. An MRF { f ( m, n), ( m, n) } is depicted in terms of
local conditional probability density function (PDF), i.e.,

p( f (m, n) | f (k, l), (k, l) (m, n), (k, l) } p( f (m, n) | f (k, l), (k, l) (m,n) )
where ( m , n ) is the neighborhood of pixel ( m, n) . If this PDF happens to be a Gaussian
distribution, we call this MRF as a GMRF. Fig. 2.1 illustrates the order of GMRF, which
is related to the location of neighbors.

Figure 2.1: Illustration of the order of GMRF.

In this report, we use a second-order GMRF, which can be expressed as follows:
f (m, n)

(t , s) f (m t , n s) e(m, n)

( t , s )

where

{(1,1), (1,0), (1,1), (0,1), (0,1), ( 1,1), ( 1,0), ( 1,1)} ,

and

e(m, n) ~ N (0, 2 ) with zero mean and variance 2 . For each pixel, we train the mean

(which is defined in the covariance matrix), and the parameters,

{ (t , s ), (t , s ) } , over a window W centered on this, by Least Square Estimation

(LSE), i.e.,

Unsupervised Texture Segmentation via GMRF Model and Filter Bank Decomposition

r (0,0)
r (0,1)

r (0,2)

r (1,0)

r (0,1)
r (0,0)

r (0,2)
r (0,1)

r (0,1)
r (1,1)

r (0,0)
r (1,2)

r (1,0) (1,1)
r (1,1)

r (1,0)
r (1,1) (1,0)

r (1,1)
r (1,2) (1,1)

r (0,0) (0,1)
r (0,1)

2 r (0,0)

(t , s)r (t , s)

( t , s )

r (t , s )

1
Nw

f (m, n) f (m t , n s)

( m , n )W

1
Nw

f (m, n)

( m , n )W

where N w is the number of pixel in the window W . Note that, due to the symmetry of
correlation function, only four s are estimated. We now form the feature space
{ , , (t , s ), (t , s ) } , and K-Means clustering algorithm is then applied to make

segmentation in this space.

Fig. 2.2 shows two mosaic images used in the experiment, Fig. 2.3 the
corresponding feature spaces of Fig. 2.2(a), and Fig. 2.4 the segmented results. From the
given figures, we observe the following:

A reasonable unsupervised segmentation is achieved. See, for example, Fig. 2.3

where the feature spaces are distinguishable, and Fig 2.4(a) where five patches are
obtained. Also, the upper part of Fig. 2.4(b) is not bad.

However, misclassifications are there due to (i) that some regions of the given
mosaic images are too non-stationary to be captured by GMRF model, or in other
word, the trained parameters have large variations, and (ii) that estimation from a
window centered at the pixel has de facto smoothing effect, which blurs the
boundary.

In addition, Fig. 2.2(b) is much more different to segment than Fig. 2.2(a) because
of the small patch of each class in the down part, which make it extremely hard to
train GMRF parameters.

Unsupervised Texture Segmentation via GMRF Model and Filter Bank Decomposition

(a)

(b)
Figure 2.2: Two mosaic images used in the experiment.

Unsupervised Texture Segmentation via GMRF Model and Filter Bank Decomposition

Figure 2.3: Feature spaces of Fig. 2.2(a).

Unsupervised Texture Segmentation via GMRF Model and Filter Bank Decomposition

(a)

(b)
Figure 2.4: Segmented results using GMRF model.

Unsupervised Texture Segmentation via GMRF Model and Filter Bank Decomposition

Chapter 3: Filter Bank Decomposition

A general filter bank scheme is presented in Fig. 3.1. A mosaic image f ( m, n) is
first decomposed using a filter bank {hk (m, n), k 1,2,..., N } , then useful features are
extracted from the filter responses by nonlinear operations, and finally clustering
algorithm is applied to produce segmentation. Given such a scheme, two issues should be
addressed: (i) filter selection and (ii) feature extraction.

Figure 3.1: a general scheme of filter bank analysis of image.

Three basic guidelines on how to choose a good filter bank are as follows. (1) It
should have a good localization in both spatial and spectral domains, though there is an
uncertainty between them. (2) It should be able to cover multi-channels or multifrequency components in order to provide a kaleidoscopic view. (3) It is hopefully HVSmotivated because HVS performs much better than any computer system does in visual
processing. Based on the above guidelines, we choose Gabor wavelets (or Gabor jets of
[7]) as the filter bank used in this report. It is well known that Gabor function minimizes
joint uncertainty, and simulates some functionalities of primary visual cortex in HVS,
while wavelets are typical multi-resolution filters. One more advantage of Gabor function
is that it is also orientation-selective.
The Gabor wavelets as defined as follows:

Unsupervised Texture Segmentation via GMRF Model and Filter Bank Decomposition

k u2
k u2
2
2
2
GW[u ,v ] (m, n) 2 exp(
(
m

n
)){cos(
k
(
m
cos

n
sin

))

exp(

)}
u
v
v
2

2 2
/2
v
k u u / 2 , u 0,1,2; v
, v 0,1,...,7
8
2

Fig. 3.2 shows some examples of filters and their spectrums. There are 24 filters in this
wavelet family, which covers almost all high-frequency components. In addition, a
simple Gaussian averaging filter is introduced in order to cover low-frequency
components.
Filter responses themselves are features. However, as argued in [4], purely linear
mechanism such as convolution is inadequate for discrimination, and thus nonlinear
operations are needed. In this report, we introduce two nonlinear operations: (i) full-wave
rectification, and (ii) adaptive pooling. Full-wave rectification is mathematically defined
as taking the absolute operation, i.e.,
rk ( m, n) ( f hk )(m, n)

Adaptive pooling is defined as follows:

s k ( m, n) rk (m, n) * g k ( m, n) ,

where g k (m, n) is a Gaussian averaging function with its variance proportional to the
sample variance of rk (m, n) . The idea of adaptive pooling stems from the fact: the filter
responses are non-stationary even within one class, thus need pooling before feeding to
K-Means clustering.
Fig. 3.3 shows the segmented results using Gabor filter bank decomposition. We
can observer the following:

Filter bank approach seems to produce a better segmentation than GMRF model
algorithm. The benefit may arise from the adaptive pooling to overcome the
irregularity within one texture class.

There exist boundary artifacts (as shown in Fig. 3.3), which are inherent to
filtering approaches.

It is still very difficult to segment the down part of Fig. 2.2(b).

Unsupervised Texture Segmentation via GMRF Model and Filter Bank Decomposition

Figure 3.2 Gabor filters and their spectrum

(a)

Unsupervised Texture Segmentation via GMRF Model and Filter Bank Decomposition

(a)

(b)
Figure 3.3 Segmented results of Gabor Filter Bank

Unsupervised Texture Segmentation via GMRF Model and Filter Bank Decomposition

Chapter 4: Conclusion
In this report, texture segmentation is achieved in unsupervised way. GMRF
model is fitted on the mosaic images, and the model parameters are used to form a multidimensional feature space. In filter bank approach, nonlinear operations are applied to
filter responses to extract features. K-Means clustering algorithm is then invoked to
perform the segmentation. Experimental results are presented and observations are made
to demonstrate the efficiency of our approaches.

Reference
[1] B. S. Manjunath, T. Simchony, and R. Chellappa, Stochastic and Deterministic
Networks for Texture Segmentation, IEEE Acoustic, Speech, and Signal Processing,
Vol. 38, No. 5, July 1990.
[2] Fure-Ching Jeng, J. W. Woods, and S. Rastogi, Compound Gauss-Markov Random
Fields for Parallel Image Processing, Markov Random Fields Theory and Application,
Academic Press, 1993.
[3] S. Krishnamachari, and R. Chellappa, Multiresolution Gauss-Markov Random Field
Models for Texture Segmentation, IEEE Trans. Image Processing, Vol. 6, No. 2,
February 1997.
[4] J. Malik and P. Perona, Preattentive texture discrimination with early vision
mechanisms, J. Opt. Soc. Am. A, Vol. 7, No. 5, May 1990
[5] B. S. Manjunath and W. Ma, Texture features for browsing and retrieval of image
data, IEEE Trans. on Pattern Analysis and Machine Intelligence, Vol. 18, No. 8, August
1996.
[6] Jianbo Shi and J Malik, Normalized Cuts and Image Segmentation, IEEE
Transactions on Pattern Analysis and Machine Intelligence, Vol. 22, No. 8, August 2000.

Unsupervised Texture Segmentation via GMRF Model and Filter Bank Decomposition

[7] L. Wiskott, J.-M. Fellous, N. Kruger, and C. von der Malsburg, Face recognition by
elastic graph matching, IEEE Trans. Pattern Analysis and Machine Intelligence, Vol. 19,
No. 7, July 1997.
[8] Part of Matlab source is attached in the appendix.

Unsupervised Texture Segmentation via GMRF Model and Filter Bank Decomposition

Appendix: Part of Matlab Source Code

% GMRF model
function para = gmrf(in)
[M,N]= size(in);
S = 1;
T = 32;
R = T/S-1;
mno = M/S;
nno = N/S;
for m=1:mno-R
for n=1:nno-R
left = (m-1)*S+1;
right = (m-1)*S+T;
top = (n-1)*S+1;
bottom = (n-1)*S+T;
a = in(left:right, top:bottom);
u = mean2(a);
a = a - mean2(a);
r00 = corrcoef2(a, 0, 0);
r10 = corrcoef2(a, 1, 0);
r01 = corrcoef2(a, 0, 1);
r11 = corrcoef2(a, 1 ,1);
r1_1 = corrcoef2(a, 1, -1);
r02 = corrcoef2(a, 0, 2);
r1_2 = corrcoef2(a, 1, -2);
b = [r00, r01, r02, r10; ...
r01, r00, r01, r1_1; ...
r02, r01, r00, r1_2; ...
r10, r1_1, r1_2, r00];
c = [r11; r10; r1_1; r01];
d = b\c;
para(m,n,1:4) = d;
para(m,n,5) = u;
para(m,n,6) = sqrt(r00-d(1)*r11-d(2)*r10-d(3)*r1_1-d(4)*r01);
end
end
for m=mno-R:mno
for n=nno-R:nno
if m ~= mno-R | n~= nno-R
para(m,n,:) = para(mno-R,nno-R,:);
end
end
end
if (mno>R+1)
for m=1:mno-R-1
for n=nno-R+1:nno
para(m,n,:) = para(m, nno-R, :);
end
end
end

Unsupervised Texture Segmentation via GMRF Model and Filter Bank Decomposition
if (nno>R+1)
for m=mno-R+1:mno
for n=1:nno-R-1
para(m,n,:) = para(mno-R, n, :);
end
end
end
%CORRCOEF2 Correlation coefficient for the image
% CORRCOEF2(X, DX, DY) computes the correlation coefficient
% r(dx, dy).
function r = corrcoef2(x, dx, dy)
[M,N] = size(x);
if dx >= 0 & dy >= 0
x1 = x(dx+1:M, dy+1:N);
x2 = x(1:M-dx, 1:N-dy);
elseif dx >= 0 & dy < 0
x1 = x(dx+1:M, 1:N+dy);
x2 = x(1:M-dx, -dy+1:N);
elseif dx < 0 & dy >= 0
x1 = x(1:M+dx, dy+1:N);
x2 = x(-dx+1:M, 1:N-dy);
else
x1 = x(1:M+dx, 1:N+dy);
x2 = x(-dx+1:M, -dy+1:N);
end
r = sum(sum(x1.*x2))/(M-abs(dx))/(N-abs(dy));
%function gw = gaborw(v, u, sigma, scale, sz)
% u -- controls the center freq
% v -- controls the orientaion
% sigma -- controls the standard variation
% scale -- set to 1 by default
% sz -- filter size
function gw = gaborw(v, u, sigma, scale, sz)
k_v = pi/2/power(2, v/2);
phi_u = u*pi/8;
s = sigma;
y = scale;
rows = sz(1);
cols = sz(2);
gw = zeros([rows,cols]);
for r=1:rows
for c = 1:cols
r1 = (r-1-rows/2)*y;
c1 = (c-1-cols/2)*y;
a = y*y;
a = a*(k_v*k_v)/(s*s)*exp(-k_v*k_v*(r1*r1+c1*c1)/(2.0*s*s));
a = a*(cos(k_v*(r1*cos(phi_u)-c1*sin(phi_u)))-exp(-s*s/2.0));
gw(r,c) = a;
end
end

Unsupervised Texture Segmentation via GMRF Model and Filter Bank Decomposition
% out = JETS(in) -- compute gabor jets for input image
function out = jets(in)
[M,N] = size(in);
out = zeros([M,N,24]);
fft2_in = fft2(in);
for u=0:2
for v=0:7
fprintf('jet (%d %d)\n', u, v);
k = u*8+v+1;
gw = gaborw(u,v,2*pi,1.0, [M,N]);
out(:,:,k) = abs(fftshift(real(ifft2(fft2_in.*fft2(gw)))));
end
end
x = [1:M] - M/2 -1;
y = gauss(x, 0, 2*pi);
gw = y'*y;
out(:,:,25) = abs(fftshift(real(ifft2(fft2_in.*fft2(gw)))));
% jets pooling
function out = jpool(in)
[M,N,L] = size(in);
for l=1:L
s = std2(in(:,:,l));
if s>4
K = floor(s);
x = -K:K;
y = gauss(x, 0, s);
B = y'*y;
else
B = 1;
end
out(:,:,l) = conv2(in(:,:,l), B, 'same');
end
%function cmap = kmeans(in, cno, error)
% cmap -- cluster map
% in -- input matrix
% cno -- class number
% epsilon -- convengence error
function cmap = kmeans(in, cno, epsilon)
if nargin ~= 3
error('Usage: kmeans(in, cno, epsilon)');
end
[M,N,L] = size(in);
cmap = zeros([M,N]);
for k=1:cno
a(k,:) = in(floor(rand(1)*M), floor(rand(1)*N), :);
end
for i=1:L

Unsupervised Texture Segmentation via GMRF Model and Filter Bank Decomposition
s(i) = std2(in(:,:,i));

end

prev_error = 0;
while (1)
disp(a);
curr_error = 0;
for m=1:M
for n=1:N
v(1,:) = in(m,n,:);
[id, dist] = nnb(v, a, s); %nearest neighbor
cmap(m,n)=id;
curr_error = curr_error + dist;
end
end
curr_error = curr_error/(M*N);
error = abs(curr_error - prev_error);
fprintf('curr_error = %f, prev_error = %f, error=%f\n', ...
curr_error, prev_error, error);
if error < epsilon
break;
else
prev_error = curr_error;
a(:,:)= 0.0;
b(1:cno)= 0;
for m=1:M
for n=1:N
k = cmap(m,n);
b(k)=b(k)+1;
v(1,:) = in(m,n,:);
a(k,:) = a(k,:) + v;
end
end
for k=1:cno
a(k,:)=a(k,:)/b(k);
end
end
end
% nearest neighbor
function [id,dist] = nnb(in, a, s)
dist = 1000;
[cno,x] = size(a);
for i=1:cno
d = sum(abs((in-a(i,:))./s));
if dist>d
id = i;
dist = d;
end
end