CLUSTERING AND FUZZY

INFERENCE SYSTEMS [FIS]

BY RONALD EKYALIMPA
CLUSTERING

2
 CLUSTERING
• Clustering is grouping data or things.
• In fuzzy logic, clustering is used to define membership functions.
• The figure below to the left shows data clustered into two – red
colored cluster and blue colored cluster.
• The figure below to the right shows membership functions generated
from clusters.

Triangular MFs for Variable y

Variable y Cluster 3
Cluster 1
MFy,3
MFy,1
Cluster 2

MFy,2

Variable x
MFx,1 MFx,2 MFx,3

Gaussian MFs for Variable x 3

 DATA-DRIVEN CLUSTERING
• Steps for clustering:
• Collect data
• Choose a clustering algorithm to use within an environment you are familiar with
• Write script to perform clustering.
• This is done here but for the Matlab environment
• Collect data
• Data instances should have values for input variables and output variables
• Input variables are clustered separately, preferably in pairs
• Output variables are clustered separately, preferably in pairs
• Cluster the data to generate membership functions
• Fuzzy C-means clustering [FCM].[Our focus in this class]
• K-means clustering
• Subtractive clustering
4
 BACKGROUND ON MATLAB FUNCTIONS TO HELP WITH
CLUSTERING
[MATLAB MAX FUNCTION FOR MATRICES]
How the max function works:
• The max function looks at each column in a matrix
• It gets the maximum value of each column and puts it in the first row
• It then returns this single row of maximum values

16 2 3 13
A= 5 11 10 8
9 7 6 12
4 14 15 1

M= max(A)

M= 16 14 15 13

5
 BACKGROUND ON MATLAB FUNCTIONS TO HELP WITH
CLUSTERING
[MATLAB FIND FUNCTION FOR MATRICES]
How the find function works:
• It takes two row vectors in a Boolean equality expression
• Compares corresponding elements in the two vectors.
• If the Boolean expression returns true, it returns and saves that index or
subscript otherwise it skips that index or subscript
• Finally it returns indices that fulfill that criteria

16 2 3 13
A= 5 11 10 8 M= max(A) M= 16 14 15 13
9 7 6 12
4 14 15 1

Index1 = find(A(1,:)==M) Index3 = find(A(3,:)==M)

Index1 = (1, 4) Index3 = ()
Index2 = find(A(2,:)==M) Index4 = find(A(4,:)==M)
Index2 = () Index4 = (2, 3)
6
 FUZZY PARTITION MATRIX [U]
Details of a partition matrix:
• A partition matrix encodes the graphical representation of data instances and their
interaction [degree of belonging] with cluster centers. See the figures below.
• This is a matrix that has Nc rows and Nd columns[CD: C – Clusters; D- Data instances]
• The clusters are the rows and the data instances are the columns [Opposite to the way
things are stored in MS. Excel]
• The rows and columns are setup this way to facilitate the use of the max function we have
described previously, i.e., to get the maximum degree of belonging for each data instance
hence the cluster that the data instance belongs to
• Each element [uij] indicates the degree of belonging of data instance j to cluster I
• An example of a partition matrix from 3 clusters and n data instances is shown below
d1 d2 d3 dn
X1 X2 X3 Xn
Variable y
Cluster 1
Cluster 3
Data instances Y1 Y2 Y3 . . Yn
Clusters Z1 Z2 Z3 Zn
Cluster 2
C1(X1,Y1,Z1) u11 u12 u13 . . u1n

Variable x
U= C2(X2,Y2,Z2) u21 u22 u23 . . u2n

C3(X3,Y3,Z3) u31 u32 u33 . . 7 u3n

 FUZZY C-MEANS CLUSTERING IN MATLAB

• Typically cluster input data alone and output data alone

• Structure of input parameters for the fcm function:
• data – is a matrix and the structure is similar to the way data is stored in excel. The rows are the
data instances while the columns are the attributes
• Nc - is the number of clusters. It is an integer
• Structure of output parameters for the fcm function:
• Centers – It is a matrix that is similar to the way data is stored in excel. The rows are the data
instances, i.e. centers and the columns are the attributes
• U – Is the partition matrix. The structure is different than that in excel to allow us to filter data
instances by cluster. The rows are cluster centers. The columns are data instances
8
 WHY THE NEED FOR FURTHER ANALYTICS
• We don’t know the data instances that belong to each cluster center
• Therefore, we use the max function on the partition matrix to obtain
the maximum degree of belonging that each data instance has been
assigned across all cluster centers
• Then we use the find function and the vector returned from the max
function, sequentially with each row of the partition matrix [i.e.
cluster centers] to find all data instances by index, that belong to each
cluster center
• Thereafter we can make a graphical plot of cluster centers, and each
of the data instances that belong that belong to those centers.

9
 FINDING INDICES OF DATA INSTANCES IN THE
DIFFERENT CLUSTER CENTERS IN MATLAB

The matlab find function:

Index1 → represents the indices of data instances that belong to cluster center 1
Index2 → represents the indices of data instances that belong to cluster center 2
• It takes a Boolean expression as its arguments. The expression is built from two-row vectors.
• The function checks if two corresponding elements in the vectors are equal. If they are equal, it returns the index
of that element, if not, it does not return the index of that elements.
• The find function returns the indices of the elements that are equal in a pair of vectors.
• Index1 will have the indices of the elements that were equal between the first row and the maximum values row
vector
• Similarly, Index2 will have the indices of the elements that were equal between the second row and the
maximum values row vector
• U(1,:) is the vector of the first row in the U matrix
• U(2,:) is the vector of the second row in the U matrix
10
 EXTRACTING FEATURE 1 & 2 VALUES
BELONGING TO INDEX1 [CLUSTER 1]
Index1 is a row vector that has the indices of the data instances that belong to cluster center 1

Features/attributes/variables

f1 f2 f3 fn
DI11 DI12 DI13 . . DI1n

DI21 DI22 DI23 . . DI2n

fcmdata = Data instances DI31 DI32 DI33 . . DI3n

. . . . . .
DIm1 DIm2 DIm3 . . DImn

Feature_1_values_in_cluster_1 = fcmdata(index1,1) Feature_1_value_for_cluster_center_1 = centers(1,1)

Feature_2_values_in_cluster_1 = fcmdata(index1,2) Feature_2_value_for_cluster_center_1 = centers(1,2)
11
 EXTRACTING FEATURE 1 & 2 VALUES
BELONGING TO INDEX2 [CLUSTER 2]
Index2 is a row vector that has the indices of the data instances that belong to cluster center 2

Features/attributes/variables

f1 f2 f3 fn
DI11 DI12 DI13 . . DI1n

DI21 DI22 DI23 . . DI2n

fcmdata = Data instances DI31 DI32 DI33 . . DI3n

. . . . . .
DIm1 DIm2 DIm3 . . DImn

Feature_1_values_in_cluster_2 = fcmdata(index2,1) Feature_1_value_for_cluster_center_2 = centers(2,1)

Feature_2_values_in_cluster_2 = fcmdata(index2,2) Feature_2_value_for_cluster_center_2 = centers(2,2)
12
PLOTTING DATA INSTANCES CLUSTERED AND
CLUSTER CENTERS IN MATLAB

• The structure of the fcmdata matrix :

• Index1 has all the rows of instances that belong to cluster 1. They are as a row array. These passed
indicate the rows of the instances that belong to cluster 1. The column indicates the feature or attribute
targeted. Fcmdata(index1,1) returns all feature 1 values that belong to cluster 1. These make the x-values
to be plotted for this cluster. Fcmdata(index1,2) returns all feature 2 values that belong to cluster 1. These
make the y-values to be plotted for this cluster.

13
DATA INSTANCES CLUSTERED AND CLUSTER
CENTERS IN MATLAB

14
CLUSTERING MAPS TO FUZZY MEMBERSHIPS
FUNCTIONS [MFS]
Triangular MFs for Variable y
Variable y Cluster 3
MFy,3 Cluster 1

MFy,1
Cluster 2
MFy,2

Variable x
MFx,1 MFx,2 MFx,3

Gaussian MFs for Variable x

• Clustering is used to generate membership functions from data
• Clustering methods include:
• FCM clustering [FCM: Fuzzy C-Means]
• K-means clustering
• Subtractive clustering 15
CREATING AN FIS [FUZZY INFERENCE SYSTEM]

16
 FUZZY INFERENCE SYSTEM [FIS]
• Collect data
• Data instances should have values for input variables and output variables
• Cluster the data to generate membership functions
• K-means clustering
• Fuzzy C-means clustering [Our focus]
• Subtractive clustering
• Rule base
• Is a set of rules that link the input variables to output variables
• Each rule may have a weight associated with it
• The number of rules is dependent on the number of input variables [dimensionality] in the
problem and membership functions for each variable
• Equation: No. of rules = mn [m - # of membership functions; n - # of input variables]
• The curse of dimensionality. Increasing the number of input variables exponentially increases the
number of rules required to model the problem
• The rules combine on all input membership functions between variables and figures out from the data
what the matching output is.
• Output variables are not used to form the rule-base size/conditional structure of the rules
• Inferencing methods
• Mamdani [Outputs a value from defuzzification – COA, LOM, MOM, MOM]
• Sugeno [Outputs a polynomial]
17
 FUZZY INFERENCE SYSTEM[S]
• Inference is similar to deduction
• Inference is also similar to reasoning
• This is why is classified under Artificial Intelligence [AI] or Machine Learning
[ML]
• FIS is a type of model that maps inputs to outputs
• In order to qualify as a model, there needs to be a mechanism that maps or
links given inputs to outputs.
• In FIS, that mechanism is referred to as a rule-base
• To be able generate a rule-base, each of the input and output variables are
represented using membership functions.
• These membership functions are permutated between variables [inputs-
inputs; inputs-outputs], to obtain a rule-base.
• A rule-base is complete when it has mn. Where m are the number of
membership functions while n is the number of fuzzy variables.
18
RULE-BASE CREATION FROM DATA

19
 FUZZY MEMBERSHIP FUNCTIONS AND RULE-BASE
CREATION
Variable A Variable B Variable C
Membership Degree Membership Degree Membership Degree

MFA,1 MFA,2 MFA,3 MFB,1 MFB,2 MFB,3 MFC,1 MFC,2 MFC,3

Universe of discourse Universe of discourse Universe of discourse

Combination between input variable membership functions Structure of rules in the rule-base [mn =32]:
• A is MFA,1 is combined with each MF in B: Maps/combines Input Variable MFs
• B is MFB,1 1. A is MFA,1 and B is MFB,1 then C is ?
• B is MFB,2 ▪ Formulate the rule
2. A is MFA,1 and B is MFB,2 then C is ?
• B is MFB,3 structure:
o Condition part complete 3. A is MFA,1 and B is MFB,3 then C is ?
• A is MFA,2 is combined with each MF in B: 4. A is MFA,2 and B is MFB,1 then C is ?
o # of rules is complete
• B is MFB,1 5. A is MFA,2 and B is MFB,2 then C is ?
• B is MFB,2 6. A is MFA,2 and B is MFB,3 then C is ?
• B is MFB,3 7. A is MFA,3 and B is MFB,1 then C is ?
• A is MFA,3 is combined with each MF in B: 8. A is MFA,3 and B is MFB,2 then C is ?
• B is MFB,1 9. A is MFA,3 and B is MFB,3 then C is ?
• B is MFB,2
20
• B is MFB,3
 FUZZY MEMBERSHIP FUNCTIONS AND RULE-BASE
Membership Membership Membership
Degree Degree Degree

MFA,1 MFA,2 MFA,3 MFB,1 MFB,2 MFB,3 MFC,1 MFC,2 MFC,3

Variable A Variable B Variable C

Rules in the rule-base [mn =32]: Rules in the rule-base [mn =32]:
• Find the membership
1. A is MFA,1 and B is MFB,1 then C is ? functions of C that match • [C1]->[W1]: A is MFA,1 and B is MFB,1 then C is [MFC,1 ]
2. A is MFA,1 and B is MFB,2 then C is ? the input combinations in • [C2]->[[W2]: A is MFA,1 and B is MFB,2 then C is MFC,2
each rule from the data • [C3]->[[W3]: A is MFA,1 and B is MFB,3 then C is MFC,3
3. A is MFA,1 and B is MFB,3 then C is ?
• Compute the weight from
4. A is MFA,2 and B is MFB,1 then C is ? • [C4]->[[W4]: A is MFA,2 and B is MFB,1 then C is MFC,1
the counts of instances of
5. A is MFA,2 and B is MFB,2 then C is ? the rule that manifested in • [C5]->[[W5]: A is MFA,2 and B is MFB,2 then C is MFC,2
6. A is MFA,2 and B is MFB,3 then C is ? the data • [C6]->[[W6]: A is MFA,2 and B is MFB,3 then C is MFC,3
7. A is MFA,3 and B is MFB,1 then C is ? • [C7]->[[W7]: A is MFA,3 and B is MFB,1 then C is MFC,1
8. A is MFA,3 and B is MFB,2 then C is ? • [C8]->[[W8]: A is MFA,3 and B is MFB,2 then C is MFC,2
9. A is MFA,3 and B is MFB,3 then C is ? • [C9]->[[W9]: A is MFA,3 and B is MFB,3 then C is MFC,3
21
USING/EXPLAINING HOW AN FIS WORKS

22
USING AN FIS: LINGUISTIC INPUT VALUES INTO FIS
Membership Membership Membership
Degree Degree Degree

MFA,1 MFA,2 MFA,3 MFB,1 MFB,2 MFB,3 MFC,1 MFC,2 MFC,3

Variable A Variable B Variable C

Prediction problem with linguistic input:
• A is MFA,2 and B is MFB,2 then ?
Membership Degree
• It fires only the relevant rule and picks the output from that rule.
Rules in the rule-base [mn]: MFC,2
• [W1]: A is MFA,1 and B is MFB,1 then C is MFC,1
• [W2]: A is MFA,1 and B is MFB,2 then C is MFC,2
• [W3]: A is MFA,1 and B is MFB,3 then C is MFC,3 Variable C
• [W4]: A is MFA,2 and B is MFB,1 then C is MFC,1 Defuzzification
• [W5]: A is MFA,2 and B is MFB,2 then C is MFC,2 [This is the only rule fired. The result is a defuzzification of MFC,1 for the output C]
• [W6]: A is MFA,2 and B is MFB,3 then C is MFC,3
• [W7]: A is MFA,3 and B is MFB,1 then C is MFC,1
• [W8]: A is MFA,3 and B is MFB,2 then C is MFC,2
• [W9]: A is MFA,3 and B is MFB,3 then C is MFC,3 23
 USING AN FIS: NUMERIC INPUT VALUES INTO FIS
Membership Membership Membership
Degree Degree Degree

MFA,1 MFA,2 MFA,3 MFB,1 MFB,2 MFB,3 MFC,1 MFC,2 MFC,3

µA,2 µA,2
µA,1 µA,3
µA,3 µA,1
15 20 30 Temperature 30 50 80 Humidity Productivity

Prediction problem with numeric input:

• Temperature is 20 degrees centigrade and B is 50% then productivity is?
• The temperature and humidity values are marked off with a vertical line. The degrees of belonging
are read off
• Find the membership functions that fire rules in the rule base. It is those that don’t have a degree of
belonging of zero
• Fire only the relevant rule, i.e. those related to those membership functions whose degree of
belonging is greater than 0.0.
• Apply the mathematical operator for “AND” or “OR” based on what appears in the rule, on the
degrees of belonging for the variables and obtain the output membership degree. “AND” is a MAX
while “OR” is a MIN.
• Use that degree of belonging to truncate the membership functions of the output
• Deffuzify the truncated output membership functionsfunctions 24
 NUMERIC INPUT VALUES INTO FIS
Membership Membership Membership
Degree Degree Degree

MFA,1 MFA,2 MFA,3 MFB,1 MFB,2 MFB,3 MFC,1 MFC,2 MFC,3

µA,2 µB,2
µA,1 µB,3
µA,3 µB,1
15 20 30 Temperature 30 50 80 Humidity Productivity

Prediction problem with numeric input:

• The membership functions that will trigger rules in the rule base in this example
include:
• Variable 1 [Temperature]: MFA,1[µA,1], MFA,2[µA,2]
• Variable 2 [Humidity]: MFB,2[µB,2], MFB,3[µB,3]
• A combination is made between variable 1 memberships and those of variable 2
• MFA,1[µA,1] & MFB,2[µB,2]
• MFA,1[µA,1] & MFB,3[µB,3]
• MFA,2[µA,2] & MFB,2[µB,2]
• MFA,2[µA,2] & MFB,3[µB,3]
• The following slides show the rules that are fired

25
 RULE FIRING: NUMERIC INPUT VALUES INTO FIS
Membership Membership Membership
Degree Degree Degree

MFA,1 MFA,2 MFA,3 MFB,1 MFB,2 MFB,3 MFC,1 MFC,2 MFC,3

µA,2 µB,2
µA,1 µB,3
µA,3 µB,1
15 20 30 Temperature 30 50 80 Humidity Productivity

• A combination is made between variable 1 memberships and those of variable 2

• MFA,1[µA,1] & MFB,2[µB,2]
• MFA,1[µA,1] & MFB,3[µB,3]
• MFA,2[µA,2] & MFB,2[µB,2]
• MFA,2[µA,2] & MFB,3[µB,3]
• The following slides show the rules that are fired

• [W1]: A is MFA,1 and B is MFB,1 then C is MFC,1

• [W2]: A is MFA,1 and B is MFB,2 then C is MFC,2 [Resulting membership, µR=max(µA,1, µB,2). It truncates the 2nd MF shape, MFc,2]
• [W3]: A is MFA,1 and B is MFB,3 then C is MFC,3 [Resulting membership, µR=max(µA,1, µB,3). It truncates the 3rd MF shape, MFc,3]
• [W4]: A is MFA,2 and B is MFB,1 then C is MFC,1
• [W5]: A is MFA,2 and B is MFB,2 then C is MFC,2 [Resulting membership, µR=max(µA,2, µB,2). It truncates the 2nd MF shape, MFc,2]
• [W6]: A is MFA,2 and B is MFB,3 then C is MFC,3 [Resulting membership, µR=max(µA,2, µB,3). It truncates the 3rd MF shape, MFc,3]
• [W7]: A is MFA,3 and B is MFB,1 then C is MFC,1
• [W8]: A is MFA,3 and B is MFB,2 then C is MFC,2
• [W9]: A is MFA,3 and B is MFB,3 then C is MFC,3
26
 RULE
Membership
FIRING: NUMERIC
Membership
INPUT VALUES
Membership
INTO FIS
Degree Degree Degree

MFA,1 MFA,2 MFA,3 MFB,1 MFB,2 MFB,3 MFC,2

µB,2 µR
µA,1
15 20 30 Temperature 30 50 80 Humidity Productivity

Membership Degree Membership Degree Membership Degree

MFA,1 MFA,2 MFA,3 MFB,1 MFB,2 MFB,3 MFC,3

µA,1 µB,3 µR
15 20 30 Temperature 30 50 80 Humidity Productivity
• [W1]: A is MFA,1 and B is MFB,1 then C is MFC,1
• [W2]: A is MFA,1 and B is MFB,2 then C is MFC,2 [Resulting membership, µR=max(µA,1, µB,2). It truncates the 2nd MF shape, MFc,2]
• [W3]: A is MFA,1 and B is MFB,3 then C is MFC,3 [Resulting membership, µR=max(µA,1, µB,3). It truncates the 3rd MF shape, MFc,3]
• [W4]: A is MFA,2 and B is MFB,1 then C is MFC,1
• [W5]: A is MFA,2 and B is MFB,2 then C is MFC,2
• [W6]: A is MFA,2 and B is MFB,3 then C is MFC,3
• [W7]: A is MFA,3 and B is MFB,1 then C is MFC,1
• [W8]: A is MFA,3 and B is MFB,2 then C is MFC,2
• [W9]: A is MFA,3 and B is MFB,3 then C is MFC,3
27
 RULE FIRING: NUMERIC INPUT Membership
VALUES INTO
Degree
FIS
Membership Degree Membership Degree

MFA,1 MFA,2 MFA,3 MFB,1 MFB,2 MFB,3 MFC,2

µA,2 µB,2 µR

15 20 30 Temperature 30 50 80 Humidity Productivity

Membership Degree Membership Degree Membership Degree

MFA,1 MFA,2 MFA,3 MFB,1 MFB,2 MFB,3 MFC,3

µA,2 µR
µB,3
15 20 30 Temperature 30 50 80 Humidity Productivity

• [W1]: A is MFA,1 and B is MFB,1 then C is MFC,1

• [W2]: A is MFA,1 and B is MFB,2 then C is MFC,2
• [W3]: A is MFA,1 and B is MFB,3 then C is MFC,3
• [W4]: A is MFA,2 and B is MFB,1 then C is MFC,1
• [W5]: A is MFA,2 and B is MFB,2 then C is MFC,2 [Resulting membership, µR=max(µA,2, µB,2). It truncates the 2nd MF shape, MFc,2]
• [W6]: A is MFA,2 and B is MFB,3 then C is MFC,3 [Resulting membership, µR=max(µA,2, µB,3). It truncates the 3rd MF shape, MFc,3]
• [W7]: A is MFA,3 and B is MFB,1 then C is MFC,1
• [W8]: A is MFA,3 and B is MFB,2 then C is MFC,2
• [W9]: A is MFA,3 and B is MFB,3 then C is MFC,3
28
 RULE FIRING: NUMERIC INPUT VALUES INTO FIS
Membership Membership Membership
Degree Degree Degree

MFA,1 MFA,2 MFA,3 MFB,1 MFB,2 MFB,3 MFC,1 MFC,2 MFC,3

µA,2 µB,2
µA,1 µB,3
µA,3 µB,1
15 20 30 Temperature 30 50 80 Humidity Productivity

• A combination is made between variable 1 memberships and those of variable 2

• MFA,1[µA,1] & MFB,2[µB,2]
• MFA,1[µA,1] & MFB,3[µB,3]
• MFA,2[µA,2] & MFB,2[µB,2]
• MFA,2[µA,2] & MFB,3[µB,3]
• The following slides show the rules that are fired

• [W1]: A is MFA,1 and B is MFB,1 then C is MFC,1

• [W2]: A is MFA,1 and B is MFB,2 then C is MFC,2 [Resulting membership, µR=max(µA,1, µB,2). It truncates the 2nd MF shape, MFc,2]
• [W3]: A is MFA,1 and B is MFB,3 then C is MFC,3 [Resulting membership, µR=max(µA,1, µB,3). It truncates the 3rd MF shape, MFc,3]
• [W4]: A is MFA,2 and B is MFB,1 then C is MFC,1
• [W5]: A is MFA,2 and B is MFB,2 then C is MFC,2 [Resulting membership, µR=max(µA,2, µB,2). It truncates the 2nd MF shape, MFc,2]
• [W6]: A is MFA,2 and B is MFB,3 then C is MFC,3 [Resulting membership, µR=max(µA,2, µB,3). It truncates the 3rd MF shape, MFc,3]
• [W7]: A is MFA,3 and B is MFB,1 then C is MFC,1
• [W8]: A is MFA,3 and B is MFB,2 then C is MFC,2
• [W9]: A is MFA,3 and B is MFB,3 then C is MFC,3
29
FIS IMPLEMENTATION IN MATLAB

30
CREATING THE FIS OPTIONS OBJECT
[genfisOptions]

31
FCM CLUSTERING OPTIONS [FCMClustering]

32
SUBTRACTIVE CLUSTERING OPTIONS [SubtractiveClustering]

33
GRID PARTITIONING CLUSTERING OPTIONS [GridPartition]

34
MODELING FIS IN MATLAB – CREATE THE INPUT &
OUTPUT DATA & GENFISOPTIONS

35
 MODELING FIS IN MATLAB – CREATING FIS
[PREDICTIVE MODEL] AND SHOWING RULES

36
MODELING FIS IN MATLAB – PLOT THE
MEMBERSHIP FUNCTIONS FROM FIS
• Subplot function is used to plot multiple
graphs close to each other
• It has three input parameters in its
method.
• The first two specify the dimension of
matrix space reserved to plot the
graphs. It uses the roman-catholic/row-
column convention
• The last input specifies the position or
cell in the matrix where the plot is to be
inserted
• Example: subplot(2,1,1) would mean we
have two rows and one column reserved
for plotting graphs and we are going to
insert our graph in the first position or
cell in that reserved space

37
THE END

38