Frequent Pattern Analysis

CS 822 Data Mining

Anis ur Rahman
Department of Computing
NUST-SEECS
Islamabad

September 13, 2018

1 / 43
 Getting To Know Your Data

Road Map

1 Data Objects and Attribute Types

2 Descriptive Data Summarization
3 Measuring Data Similarity and Dissimilarity

2 / 43
 Getting To Know Your Data

Data Objects and Attribute Types

Types of data sets

Data objects
Attributes and their types

3 / 43
 Getting To Know Your Data

Types of Data Sets

Record. is a collection of fields, possibly of different data types,

typically in fixed number and sequence.

Relational records Data matrix, e.g., numerical matrix, cross

tabulations.
Login First name Last name
Spenders Books Mult. devices
koala John Clemens
lion Mary Stevens Big 30% 70%
Budget 60% 25%
Login phone Tight 10% 5%

koala 039689852639 Document data: text documents:

term-frequency vector
Transaction data
team ball lost timeout
TID Items
Doc 1 3 5 2 2
1 Bred, Cake, Milk Doc 2 0 0 3 0
2 Beer, Bred Doc 3 0 1 0 0

4 / 43
 Getting To Know Your Data

Types of Data Sets

Graph and Network

World Wide Web
Social or information networks
Molecular structures networks

5 / 43
 Getting To Know Your Data

Types of Data Sets

Ordered
Videos
Temporal data
Sequential data
Genetic sequence data

6 / 43
 Getting To Know Your Data

Types of Data Sets

Spatial, image and multimedia

Spatial data
Image data
Video data
Audio Data

7 / 43
 Getting To Know Your Data

Data Objects and Attributes

Datasets are made up of data objects.

A data object ( or sample , example, instance, data point, tuple)
represents an entity.
Examples
Sales database. customers, store items, sales
Medical database. patients, treatments
University database. students, professors, courses
Data objects are described by attributes (or dimension, feature,
variable).
Database rows → data objects; columns → attributes.

Patient_ID Age Height Weight Gender

1569 30 1,76m 70 kg male
2596 26 1,65m 58kg female

8 / 43
 Getting To Know Your Data

Attribute Types

Nominal categories, states, or “names of things”

Hair_color = {black, brown, blond, red, grey, white}
marital status, occupation, ID numbers, zip codes
Binary
Nominal attribute with only 2 states (0 and 1)
Symmetric binary. both outcomes equally important
e.g., gender
Asymmetric binary. outcomes not equally important.
e.g., medical test (positive vs. negative)
Convention. assign 1 to most important outcome (e.g., having
cancer)
Ordinal
Values have a meaningful order (ranking) but magnitude between
successive values is not known.
Size = {small, medium, large}, grades, army rankings

9 / 43
 Getting To Know Your Data

Attributes Types

Numeric. quantity (integer or real-valued)

Interval-scaled
Measured on a scale of equal-sized units
Values have order
e.g., temperature in C ◦ or F ◦ , calendar dates
No true zero-point
can add and subtract degrees e.g., 100◦ is 10◦ warmer than 90◦
cannot multiply values or create ratios 100◦ is not twice as warm as
50◦
Ratio-scaled
Inherent zero-point
e.g., temperature in Kelvin, length, counts, monetary quantities
Can be presented on order of magnitude larger than the unit of
measurement
10 K ◦ is twice as high as 5 K ◦
A 6-foot person is 20% taller than a 5-foot person.
A baseball game lasting 3 hours is 50% longer than a game lasting 2
hours
10 / 43
 Getting To Know Your Data

Discrete vs. Continuous Attributes

Discrete
Has only a finite or countable infinite set of values
e.g., zip codes, profession, or the set of words in a collection of
documents
Sometimes, represented as integer variables
Note. Binary attributes are a special case of discrete attributes
Continuous
Has real numbers as attribute values
e.g., temperature, height, or weight
Practically, real values can only be measured and represented
using a finite number of digits
Typically represented as floating-point variables

11 / 43
 Getting To Know Your Data

Quiz

Q. What is the type of an attribute that describes the height of a person

in centimeters?
Nominal
Ordinal
Interval-scaled
Ratio-scaled
Q. In Olympic games, three types of medals are awarded: bronze, silver,
or gold. To describe these medals, which type of attributes should be
used?
Nominal
Ordinal
Interval-scaled
Ratio-scaled
12 / 43
 Getting To Know Your Data

Road Map

1 Data Objects and Attribute Types

2 Descriptive Data Summarization
3 Measuring Data Similarity and Dissimilarity

13 / 43
 Getting To Know Your Data

Descriptive Data Summarization

Motivation
For data preprocessing, an overall picture of data is essential

Data summarization techniques used to

Define the typical properties of the data
Highlight which data should be treated as noise or outliers
Data properties
Centrality. use measures such as the median
Variance. use measures such as the quantiles

From the data mining point of view it is important to

Examine how these measures are computed efficiently
Introduce the notions of distributive, algebraic and holistic
measures
14 / 43
 Getting To Know Your Data

Measuring the Central Tendency

Mean (algebraic measure)
n
1X
x̄ = xi ; n is sample size
n
i =1

A distributive measure can be computed by partitioning the data

into smaller subsets (e.g., sum, and count)

An algebraic measure can be computed by applying an algebraic

function to one or more distributive measures
e.g., mean = sum/count
Sometimes each value xi is weighted, weighted arithmetic mean
Pn
1 w i xi
x̄ = Pi =
n w
i =1 i

15 / 43
 Getting To Know Your Data

Measuring the Central Tendency

Median (holistic measure)

Middle value if odd number of values, or average of the middle two
values
Measure computed on the entire dataset
Much more expensive to compute than distributive measures
Can be estimated by interpolation (for grouped data):
n − (P freq) !
l
median = L1 + 2 width
freqmedian

Median interval contains the median frequency

L1 : the lower boundary of the median interval
n: the number of values in the entire dataset
P
( freq)l : sum of all freq of intervals below the median interval
Freqmedian and width : frequency & width of the median interval
16 / 43
 Getting To Know Your Data

Measuring the Central Tendency

Age frequency
1-5 200
6-15 450
16-20 300
21-50 1500
51-80 700

17 / 43
 Getting To Know Your Data

Measuring the Central Tendency

Mode
Value that occurs most frequently in the data
It is possible that several different values have the greatest
frequency
unimodal, bimodal, trimodal, multimodal
If each data value occurs only once then there is no mode
Empirical formula:
mean − mode = 3 × (mean − median)

Midrange
Can also be used to assess the central tendency
It is the average of the smallest and the largest value of the set
It is an algebric measure that is easy to compute
18 / 43
 Getting To Know Your Data

Symmetric vs. Skewed Data

Median, mean and mode of symmetric, positively and negatively

skewed data

19 / 43
 Getting To Know Your Data

Quiz

Q. Give an example of something having a positively skewed

distribution
Income is a good example of a positively skewed variable
There will be a few people with extremely high incomes, but most
people will have incomes bunched together below the mean.
Q. Give an example of something having a bimodal distribution
Bimodal distribution has some kind of underlying binary variable
that will result in a separate mean for each value of this variable.
One example can be human weight–the gender is binary and is a
statistically significant indicator of how heavy a person is.

20 / 43
 Getting To Know Your Data

Measuring the Dispersion of Data

The degree in which data tend to spread is called the dispersion, or

variance of the data.
Most common measures for data dispersion are,
range, the five-number summary (based on quartiles), the
inter-quartile range, and standard deviation.

Range
the distance between the largest and the smallest values
Kth percentile
value xi having the property that k% of the data lies at or below xi
the median is 50th percentile
the most popular percentiles other than the median are Quartiles
Q1 (25th percentile), Q3 (75th percentile)

Quartiles + median give some indication of the center, spread, and the
shape of a distribution
21 / 43
 Getting To Know Your Data

Measuring the Dispersion of Data

Inter-quartile range
Distance between the first and the third quartiles
IQR = Q3 − Q1

A simple measure of spread that gives the range covered by the

middle half of the data
Outlier. usually, a value falling at least 1.5 × IQR above the third
quartile or below the first quartile
Five number summary
Provide in addition information about the endpoints (e.g., tails)
min, Q1 , median, Q3 , max
e.g., min = Q1 − 1.5 × IQR , max = Q3 + 1.5 × IQR
Represented by a Boxplot

22 / 43
 Getting To Know Your Data

Measuring the Dispersion of Data

Variance.
n
1X
σ2 = (xi − µ)2
N
i =1

Standard deviation. σ is the square root of variance σ 2

Basic properties of the standard deviation

σ measures spread about the mean and should be used only
when the mean is chosen as the measure of the center
σ = 0 only when there is no spread, that is, when all observations
have the same value. Otherwise σ > 0
Variance and standard deviation are algebraic measures. Thus, their
computation is scalable in large databases.

23 / 43
 Getting To Know Your Data

Graphic Displays

Boxplot. graphic display of five-number summary

Histogram. x-axis are values, y-axis representative frequencies
Scatter plot. each pair of values is a pair of coordinates and
plotted as points in the plane

24 / 43
 Getting To Know Your Data

Boxplot

Recall five-number summary of a distribution

Minimum, Q1, Median, Q3, Maximum

Boxplot. represents data with a box

The ends of the box are at the first and third

quartiles, i.e., the height of the box is IQR
The median is marked by a line within the box
Whiskers. two lines outside the box extended
to min and max
Outliers. points beyond a specified outlier
threshold, plotted individually

25 / 43
 Getting To Know Your Data

Histogram Analysis

Histogram. summarizes the distribution of a given attribute

Partition the data distribution into disjoint subsets, or buckets

Note
If the attribute is nominal → bar chart
If the attribute is numeric → histogram

26 / 43
 Getting To Know Your Data

Histograms Often Tell More than Boxplots

The two histograms shown in the left may have the same boxplot
representation
The same values for: min, Q1, median, Q3, max, but rather
different data distributions

27 / 43
 Getting To Know Your Data

Scatter plot

Provides a first look at bivariate data to see clusters of points,

outliers, etc.
Each pair of values is treated as a pair of coordinates and plotted
as points in the plane

28 / 43
 Getting To Know Your Data

Positively & Negatively Correlated Data

Leftmost is positively correlated

Middle is negatively correlated
Rightmost is uncorrelated

29 / 43
 Getting To Know Your Data

Road Map

1 Data Objects and Attribute Types

2 Descriptive Data Summarization
3 Measuring Data Similarity and Dissimilarity

30 / 43
 Getting To Know Your Data

Data Similarity and Dissimilarity

Similarity. Numerical measure of how alike two data objects are.

value is higher when objects are more alike
often falls in the range [0,1]
Dissimilarity. (e.g., distance) Numerical measure of how different two
data objects are
value is lower when objects are more alike
minimum dissimilarity is often 0
upper limit varies
Proximity. refers to a similarity or dissimilarity

31 / 43
 Getting To Know Your Data

Data Matrix and Dissimilarity Matrix

Data matrix

x
 11 ··· x1f ··· x1p 
 . .. .. 

.. ..
n data points with p dimensions  .
 . . . . . 
··· ···
 
 xi 1 xif xip 
Two modes. rows and columns represent 
 . .. .. 
 .. .. ..
different entities  . . . . 
xn1 ··· xnf ··· xnp

Dissimilarity matrix

n data points, but registers only the distance

 0 
 
 d (2, 1) 0 
 
A triangular matrix  d (3, 1)
 d (3, 2) 

 .. . ..
 
Single mode. row and columns represent  . .. .


 
the same entity d (n, 1) d (n, 2) ··· 0

32 / 43
 Getting To Know Your Data

Nominal Attributes

Can take 2 or more states

generalization of a binary attribute
e.g., red, yellow, blue, green

Method 1. Simple matching

m: # of matches, p: total # of variables
p −m
d (i , j ) =
p

Method 2. Use a large number of binary attributes

creating a new binary attribute for each of the M nominal states

33 / 43
 Getting To Know Your Data

Binary Attributes

A contingency table for binary data,

Obj i
1 0 sum
1 q r q+r
Obj j
0 s t s+t
sum q+s r+t p

Distance measure for symmetric binary variables

r +s
d (i , j ) =
q +r +s +t
Distance measure for asymmetric binary variables
r +s
d (i , j ) =
q +r +s
Jaccard coefficient. similarity measure for asymmetric binary
variables
q
sim(i , j ) = = 1 − d (i , j )
q +r +s
34 / 43
 Getting To Know Your Data

Example

feat = {Sphere shape, Sweet, Sour, Crunchy}

Obj i = Apple = {Y,Y,Y,Y} Obj j = Banana = {N,Y,N,N}

35 / 43
 Getting To Know Your Data

Numeric Attributes

Problem. The measurement unit used for interval-scale attributes can

effect similarity, e.g., kilograms vs. pounds for weight
Need of standardizing the data; convert the original
measurements to unit-less variables
For measurements of each variable f , calculate the mean absolute
deviation, sf
1
sf = (|x − mf | + |x2f − mf | + · · · + |xnf − mf |)
n 1f

mf : the mean of f
x1f , · · · , xnf : measurements of f
Calculate the standardized measurement, or z-score
x − mf
zif = if
sf
Using the mean absolute deviation reduces the effect of outliers
Outliers remain detectable (non squared deviation)
36 / 43
 Getting To Know Your Data

Distance on Numeric Data

Minkowski distance. A popular distance measure
q
d (i , j ) = h (|xi 1 − xj 1 |h + |xi 2 − xj 2 |h + · · · + |xip − xjp |h )

where i = (xi 1 , xi 2 , · · · , xip ) and j = (xj 1 , xj 2 , · · · , xjp ) are two

p-dimensional data objects, and h is the order
Properties
d (i , j ) > 0 if i , j , and d (i , i ) = 0 (Positive definiteness)
d (i , j ) = d (j , i ) (Symmetry)
d (i , j ) ≤ d (i , k ) + d (k , j ) (Triangle Inequality)

A distance that satisfies these properties is a metric.

37 / 43
 Getting To Know Your Data

Special Cases of Minkowski Distance

h = 1. Manhattan (city block, L1 norm) distance

e.g., the Hamming distance: the number of bits that are different
between two binary vectors

d (i , j ) = |xi 1 − xj 1 | + |xi 2 − xj 2 | + · · · + |xip − xjp |

h = 2. (L2 norm) Euclidean distance

q
d (i , j ) = (|xi 1 − xj 1 |2 + |xi 2 − xj 2 |2 + · · · + |xip − xjp |2 )

38 / 43
 Getting To Know Your Data

Example: Minkowski Distance

point attribute 1 attribute 2

x1 1 2
x2 3 5
x3 2 0
x4 4 5

Dissimilarity Matrices

x1 x2 x3 x4 
 
 
x 0
 1 

Manhattan(L1 ) = x2 5 0
 

x3 3 6 0
 
 
x4 6 1 7 0

x1 x2 x3 x4 
 
 
x 0
 1 

Euclidean(L2 ) = x2 3.61 0
 

x3 2.24 5.1 0
 
 
x4 4.24 1 5.39 0
39 / 43
 Getting To Know Your Data

Ordinal Variables

An ordinal variable can be discrete or continuous

Order is important, e.g., rank
rif ∈ {1, · · · , Mf }

Can be treated like interval-scaled

replace xif by their rank
map the range of each variable onto [0, 1] by replacing i-th object in
the f-th variable by
r −1
zif = if
Mf − 1
compute the dissimilarity using methods for interval-scaled
variables

40 / 43
 Getting To Know Your Data

Vector Objects

A document can be represented by thousands of attributes,

each recording the frequency of a particular word (such as
keywords) or phrase in the document.
team coach baseball soccer penalty score win loss
Doc1 5 0 0 2 0 0 2 0
Doc2 3 0 2 1 0 0 3 0
Doc3 0 7 0 1 0 0 3 0
Doc4 0 1 0 1 2 2 0 3

Other vector objects. gene features in micro-arrays, ...

Cosine measure. If d1 and d2 are two vectors (e.g., term-frequency
vectors), then
d1 · d2
cos(d1 , d2 ) =
||d1 ||||d2 ||

where (·) indicates vector dot product, and ||d || is the length of vector d
41 / 43
 Getting To Know Your Data

Cosine Similarity
Example. Find the similarity between documents 1 and 2
d1 = (5, 0, 3, 0, 2, 0, 0, 2, 0, 0) and d2 = (3, 0, 2, 0, 1, 1, 0, 1, 0, 1)
Compute d 1 · d 2
d 1 · d 2 = (5)(3) + 0 + (3)(2) + 0 + (2)(1) + (0)(1) + (0)(1) + (2)(1) + 0 + (0)(1)
= 25
Compute ||d1 ||
p
||d1 || = (5)(5) + 0 + (3)(3) + 0 + (2)(2) + 0 + 0 + (2)(2) + 0 + 0
√
= 42 = 6.481
Compute ||d2 ||
p
||d2 || = (3)(3) + 0 + (2)(2) + 0 + (1)(1) + (1)(1) + 0 + (1)(1) + 0 + (1)(1)
√
= 17 = 4.12
Compute cosine
d1 · d2 25
cos(d1 , d2 ) = = = 0.94
||d1 ||||d2 || 6.481 × 4.12
42 / 43
 Getting To Know Your Data

Summary

Data attribute types. nominal, binary, ordinal, interval-scaled,

ratio-scaled
Many types of data sets, e.g., numerical, text, graph, Web, image
Gain insight into the data by:
Basic statistical data description. central tendency, dispersion,
graphical displays
Data visualization. map data onto graphical primitives
Measure data similarity

Note
Above steps are the beginning of data preprocessing
Many methods have been developed but still an active area of
research.

43 / 43