Data Mining

Unit – 2: Association Analysis

Dr. M. Kumara Swamy

1
 Unit - II

◼ Association Analysis: Basic Concepts, Market

Basket Analysis, Apriori Algorithm, FP-growth,
From Association Analysis to Correlation Analysis,
Pattern Mining in Multilevel Associations and
Multidimensional Associations

2
 Association Analysis

◼ Basic Concepts

◼ Frequent Itemset Mining Methods

◼ Apriori Algorithm

◼ FP-growth

◼ From Association Analysis to Correlation Analysis

◼ Pattern Mining in Multilevel Associations and

◼ Multidimensional Associations

◼ Summary
3
 What Is Frequent Pattern Analysis?
◼ Frequent pattern: a pattern (a set of items, subsequences, substructures,
etc.) that occurs frequently in a data set
◼ First proposed by Agrawal, Imielinski, and Swami [AIS93] in the context
of frequent itemsets and association rule mining
◼ Motivation: Finding inherent regularities in data
◼ What products were often purchased together?— Beer and diapers?!
◼ What are the subsequent purchases after buying a PC?
◼ What kinds of DNA are sensitive to this new drug?
◼ Can we automatically classify web documents?
◼ Applications
◼ Basket data analysis, cross-marketing, catalog design, sale campaign
analysis, Web log (click stream) analysis, and DNA sequence analysis.
4
 Why Is Freq. Pattern Mining Important?

◼ Freq. pattern: An intrinsic and important property of

datasets
◼ Foundation for many essential data mining tasks
◼ Association, correlation, and causality analysis

◼ Sequential, structural (e.g., sub-graph) patterns

◼ Pattern analysis in spatiotemporal, multimedia, time-

series, and stream data

◼ Classification: discriminative, frequent pattern analysis

◼ Cluster analysis: frequent pattern-based clustering

◼ Data warehousing: iceberg cube and cube-gradient

◼ Semantic data compression: fascicles

◼ Broad applications

5
 Basic Concepts: Frequent Patterns

Tid Items bought ◼ itemset: A set of one or more

10 Beer, Nuts, Diaper items
20 Beer, Coffee, Diaper ◼ k-itemset X = {x1, …, xk}
30 Beer, Diaper, Eggs
◼ (absolute) support, or, support
40 Nuts, Eggs, Milk count of X: Frequency or
50 Nuts, Coffee, Diaper, Eggs, Milk occurrence of an itemset X
Customer Customer
◼ (relative) support, s, is the
buys both buys diaper fraction of transactions that
contains X (i.e., the probability
that a transaction contains X)
◼ An itemset X is frequent if X’s
support is no less than a minsup
Customer
buys beer
threshold

6
 Basic Concepts: Association Rules
Tid Items bought ◼ Find all the rules X → Y with
10 Beer, Nuts, Diaper
minimum support and confidence
20 Beer, Coffee, Diaper
30 Beer, Diaper, Eggs ◼ support, s, probability that a
40 Nuts, Eggs, Milk transaction contains X  Y
50 Nuts, Coffee, Diaper, Eggs, Milk
◼ confidence, c, conditional
probability that a transaction
Customer
Customer
buys both
having X also contains Y
buys
diaper
Let minsup = 50%, minconf = 50%
Freq. Pat.: Beer:3, Nuts:3, Diaper:4, Eggs:3,
Customer {Beer, Diaper}:3
buys beer ◼ Association rules: (many more!)
◼ Beer → Diaper (60%, 100%)
◼ Diaper → Beer (60%, 75%)
7
 Closed Patterns and Max-Patterns
◼ A long pattern contains a combinatorial number of sub-
patterns, e.g., {a1, …, a100} contains (1001) + (1002) + … +
(110000) = 2100 – 1 = 1.27*1030 sub-patterns!
◼ Solution: Mine closed patterns and max-patterns instead
◼ An itemset X is closed if X is frequent and there exists no
super-pattern Y ‫ כ‬X, with the same support as X
(proposed by Pasquier, et al. @ ICDT’99)
◼ An itemset X is a max-pattern if X is frequent and there
exists no frequent super-pattern Y ‫ כ‬X (proposed by
Bayardo @ SIGMOD’98)
◼ Closed pattern is a lossless compression of freq. patterns
◼ Reducing the # of patterns and rules
8
Computational Complexity of Frequent Itemset
Mining
◼ How many itemsets are potentially to be generated in the worst case?
◼ The number of frequent itemsets to be generated is senstive to the
minsup threshold
◼ When minsup is low, there exist potentially an exponential number of
frequent itemsets
◼ The worst case: MN where M: # distinct items, and N: max length of
transactions
◼ The worst case complexty vs. the expected probability
◼ Ex. Suppose Walmart has 104 kinds of products
◼ The chance to pick up one product 10-4
◼ The chance to pick up a particular set of 10 products: ~10-40
◼ What is the chance this particular set of 10 products to be frequent
103 times in 109 transactions?

9
 Association Analysis

◼ Basic Concepts

◼ Frequent Itemset Mining Methods

◼ Apriori Algorithm

◼ FP-growth

◼ From Association Analysis to Correlation Analysis

◼ Pattern Mining in Multilevel Associations and

◼ Multidimensional Associations

◼ Summary
10
The Downward Closure Property and Scalable
Mining Methods
◼ The downward closure property of frequent patterns
◼ Any subset of a frequent itemset must be frequent

◼ If {beer, diaper, nuts} is frequent, so is {beer,

diaper}
◼ i.e., every transaction having {beer, diaper, nuts} also

contains {beer, diaper}

◼ Scalable mining methods: Three major approaches
◼ Apriori (Agrawal & Srikant@VLDB’94)

◼ Freq. pattern growth (FPgrowth—Han, Pei & Yin

@SIGMOD’00)
◼ Vertical data format approach (Charm—Zaki & Hsiao

@SDM’02)
11
Apriori: A Candidate Generation & Test Approach

◼ Apriori pruning principle: If there is any itemset which is

infrequent, its superset should not be generated/tested!
(Agrawal & Srikant @VLDB’94, Mannila, et al. @ KDD’ 94)
◼ Method:
◼ Initially, scan DB once to get frequent 1-itemset
◼ Generate length (k+1) candidate itemsets from length k
frequent itemsets
◼ Test the candidates against DB
◼ Terminate when no frequent or candidate set can be
generated

12
 The Apriori Algorithm—An Example
Supmin = 2 Itemset sup
Itemset sup
Database TDB {A} 2
L1 {A} 2
Tid Items C1 {B} 3
{B} 3
10 A, C, D {C} 3
1st scan {C} 3
20 B, C, E {D} 1
{E} 3
30 A, B, C, E {E} 3
40 B, E
C2 Itemset sup C2 Itemset
{A, B} 1
L2 Itemset sup
{A, C} 2
2nd scan {A, B}
{A, C} 2 {A, C}
{A, E} 1
{B, C} 2
{B, C} 2 {A, E}
{B, E} 3
{B, E} 3 {B, C}
{C, E} 2
{C, E} 2 {B, E}
{C, E}

C3 Itemset L3 Itemset sup

3rd scan
{B, C, E} {B, C, E} 2

13
The Apriori Algorithm (Pseudo-Code)
Ck: Candidate itemset of size k
Lk : frequent itemset of size k

L1 = {frequent items};
for (k = 1; Lk !=; k++) do begin
Ck+1 = candidates generated from Lk;
for each transaction t in database do
increment the count of all candidates in Ck+1 that
are contained in t
Lk+1 = candidates in Ck+1 with min_support
end
return k Lk;
14
 Implementation of Apriori

◼ How to generate candidates?

◼ Step 1: self-joining Lk
◼ Step 2: pruning
◼ Example of Candidate-generation
◼ L3={abc, abd, acd, ace, bcd}
◼ Self-joining: L3*L3
◼ abcd from abc and abd
◼ acde from acd and ace
◼ Pruning:
◼ acde is removed because ade is not in L3
◼ C4 = {abcd}
15
Candidate Generation: An SQL Implementation
◼ SQL Implementation of candidate generation
◼ Suppose the items in Lk-1 are listed in an order
◼ Step 1: self-joining Lk-1
insert into Ck
select p.item1, p.item2, …, p.itemk-1, q.itemk-1
from Lk-1 p, Lk-1 q
where p.item1=q.item1, …, p.itemk-2=q.itemk-2, p.itemk-1 <
q.itemk-1
◼ Step 2: pruning
forall itemsets c in Ck do
forall (k-1)-subsets s of c do
if (s is not in Lk-1) then delete c from Ck
◼ Use object-relational extensions like UDFs, BLOBs, and Table functions for
efficient implementation [See: S. Sarawagi, S. Thomas, and R. Agrawal.
Integrating association rule mining with relational database systems:
Alternatives and implications. SIGMOD’98]
17
 Association Analysis

◼ Basic Concepts

◼ Frequent Itemset Mining Methods

◼ Apriori Algorithm

◼ FP-growth

◼ From Association Analysis to Correlation Analysis

◼ Pattern Mining in Multilevel Associations and

◼ Multidimensional Associations

◼ Summary
18
 Pattern-Growth Approach: Mining Frequent
Patterns Without Candidate Generation
◼ Bottlenecks of the Apriori approach
◼ Breadth-first (i.e., level-wise) search
◼ Candidate generation and test
◼ Often generates a huge number of candidates
◼ The FPGrowth Approach (J. Han, J. Pei, and Y. Yin, SIGMOD’ 00)
◼ Depth-first search
◼ Avoid explicit candidate generation
◼ Major philosophy: Grow long patterns from short ones using local
frequent items only
◼ “abc” is a frequent pattern
◼ Get all transactions having “abc”, i.e., project DB on abc: DB|abc
◼ “d” is a local frequent item in DB|abc → abcd is a frequent pattern
19
Construct FP-tree from a Transaction Database

TID Items bought (ordered) frequent items

100 {f, a, c, d, g, i, m, p} {f, c, a, m, p}
200 {a, b, c, f, l, m, o} {f, c, a, b, m}
300 {b, f, h, j, o, w} {f, b} min_support = 3
400 {b, c, k, s, p} {c, b, p}
500 {a, f, c, e, l, p, m, n} {f, c, a, m, p} {}
Header Table
1. Scan DB once, find
frequent 1-itemset (single Item frequency head f:4 c:1
item pattern) f 4
c 4 c:3 b:1 b:1
2. Sort frequent items in a 3
frequency descending b 3 a:3 p:1
order, f-list m 3
p 3
3. Scan DB again, construct m:2 b:1
FP-tree
F-list = f-c-a-b-m-p p:2 m:1
20
 Partition Patterns and Databases

◼ Frequent patterns can be partitioned into subsets

according to f-list
◼ F-list = f-c-a-b-m-p

◼ Patterns containing p

◼ Patterns having m but no p

◼ …

◼ Patterns having c but no a nor b, m, p

◼ Pattern f

◼ Completeness and non-redundency

21
Find Patterns Having P From P-conditional Database

◼ Starting at the frequent item header table in the FP-tree

◼ Traverse the FP-tree by following the link of each frequent item p
◼ Accumulate all of transformed prefix paths of item p to form p’s
conditional pattern base

{}
Header Table
f:4 c:1 Conditional pattern bases
Item frequency head
f 4 item cond. pattern base
c 4 c:3 b:1 b:1 c f:3
a 3
b 3 a:3 p:1 a fc:3
m 3 b fca:1, f:1, c:1
p 3 m:2 b:1 m fca:2, fcab:1

p:2 m:1 p fcam:2, cb:1

22
From Conditional Pattern-bases to Conditional FP-trees

◼ For each pattern-base

◼ Accumulate the count for each item in the base

◼ Construct the FP-tree for the frequent items of the

pattern base

m-conditional pattern base:

{} fca:2, fcab:1
Header Table
Item frequency head All frequent
f:4 c:1 patterns relate to m
f 4 {}
c 4 c:3 b:1 b:1 m,

a 3 f:3  fm, cm, am,
b 3 a:3 p:1 fcm, fam, cam,
m 3 c:3 fcam
p 3 m:2 b:1
p:2 m:1 a:3
m-conditional FP-tree
23
 Recursion: Mining Each Conditional FP-tree
{}

{} Cond. pattern base of “am”: (fc:3) f:3

c:3
f:3
am-conditional FP-tree
c:3 {}
Cond. pattern base of “cm”: (f:3)
a:3 f:3
m-conditional FP-tree
cm-conditional FP-tree

{}

Cond. pattern base of “cam”: (f:3) f:3

cam-conditional FP-tree

24
A Special Case: Single Prefix Path in FP-tree

◼ Suppose a (conditional) FP-tree T has a shared

single prefix-path P
◼ Mining can be decomposed into two parts
{}
◼ Reduction of the single prefix path into one node
a1:n1 ◼ Concatenation of the mining results of the two
a2:n2 parts
a3:n3
{} r1

b1:m1 C1:k1 a1:n1

 r1 = + b1:m1 C1:k1
a2:n2
C2:k2 C3:k3
a3:n3 C2:k2 C3:k3
25
 Benefits of the FP-tree Structure

◼ Completeness
◼ Preserve complete information for frequent pattern
mining
◼ Never break a long pattern of any transaction
◼ Compactness
◼ Reduce irrelevant info—infrequent items are gone
◼ Items in frequency descending order: the more
frequently occurring, the more likely to be shared
◼ Never be larger than the original database (not count
node-links and the count field)

26
The Frequent Pattern Growth Mining Method

◼ Idea: Frequent pattern growth

◼ Recursively grow frequent patterns by pattern and

database partition
◼ Method
◼ For each frequent item, construct its conditional

pattern-base, and then its conditional FP-tree

◼ Repeat the process on each newly created conditional

FP-tree
◼ Until the resulting FP-tree is empty, or it contains only

one path—single path will generate all the

combinations of its sub-paths, each of which is a
frequent pattern

27
Performance of FPGrowth in Large Datasets

100

90 D1 FP-grow th runtime
D1 Apriori runtime
80

70
Run time(sec.)
60
50 Data set T25I20D10K
40

30

20

10
0
0 0.5 1 1.5 2 2.5 3
Support threshold(%)

FP-Growth vs. Apriori

30
Advantages of the Pattern Growth Approach

◼ Divide-and-conquer:
◼ Decompose both the mining task and DB according to the
frequent patterns obtained so far
◼ Lead to focused search of smaller databases
◼ Other factors
◼ No candidate generation, no candidate test
◼ Compressed database: FP-tree structure
◼ No repeated scan of entire database
◼ Basic ops: counting local freq items and building sub FP-tree, no
pattern search and matching
◼ A good open-source implementation and refinement of FPGrowth
◼ FPGrowth+ (Grahne and J. Zhu, FIMI'03)

31
 Association Analysis

◼ Basic Concepts

◼ Frequent Itemset Mining Methods

◼ Apriori Algorithm

◼ FP-growth

◼ From Association Analysis to Correlation Analysis

◼ Pattern Mining in Multilevel Associations and

◼ Multidimensional Associations

◼ Summary
32
 Interestingness Measure: Correlations (Lift)
◼ play basketball  eat cereal [40%, 66.7%] is misleading
◼ The overall % of students eating cereal is 75% > 66.7%.
◼ play basketball  not eat cereal [20%, 33.3%] is more accurate,
although with lower support and confidence
◼ Measure of dependent/correlated events: lift

Basketball Not basketball Sum (row)

Cereal 2000 1750 3750

Not cereal 1000 250 1250

2000 / 5000
lift ( B, C ) = = 0.89 Sum(col.) 3000 2000 5000
3000 / 5000 * 3750 / 5000
1000 / 5000
lift ( B, C ) = = 1.33
3000 / 5000 *1250 / 5000

33
 Are lift and 2 Good Measures of Correlation?

◼ “Buy walnuts  buy

milk [1%, 80%]” is
misleading if 85% of
customers buy milk
◼ Support and confidence
are not good to indicate
correlations
◼ Over 20 interestingness
measures have been
proposed (see Tan,
Kumar, Sritastava
@KDD’02)
◼ Which are good ones?

34
Null-Invariant Measures

35
 Comparison of Interestingness Measures
◼ Null-(transaction) invariance is crucial for correlation analysis
◼ Lift and 2 are not null-invariant
◼ 5 null-invariant measures

Milk No Milk Sum (row)

Coffee m, c ~m, c c
No Coffee m, ~c ~m, ~c ~c
Sum(col.) m ~m 

Null-transactions Kulczynski
w.r.t. m and c measure (1927) Null-invariant

September 12, 2024 Data Mining: Concepts and Techniques Subtle: They disagree36
 Analysis of DBLP Coauthor Relationships
Recent DB conferences, removing balanced associations, low sup, etc.

Advisor-advisee relation: Kulc: high,

coherence: low, cosine: middle
◼ Tianyi Wu, Yuguo Chen and Jiawei Han, “Association Mining in Large
Databases: A Re-Examination of Its Measures”, Proc. 2007 Int. Conf.
Principles and Practice of Knowledge Discovery in Databases
(PKDD'07), Sept. 2007
37
 Which Null-Invariant Measure Is Better?
◼ IR (Imbalance Ratio): measure the imbalance of two
itemsets A and B in rule implications

◼ Kulczynski and Imbalance Ratio (IR) together present a

clear picture for all the three datasets D4 through D6
◼ D4 is balanced & neutral

◼ D5 is imbalanced & neutral

◼ D6 is very imbalanced & neutral

 Association Analysis

◼ Basic Concepts

◼ Frequent Itemset Mining Methods

◼ Apriori Algorithm

◼ FP-growth

◼ From Association Analysis to Correlation Analysis

◼ Pattern Mining in Multilevel Associations and

◼ Multidimensional Associations

◼ Summary
39
 Research on Pattern Mining: A Road Map

40
 Mining Multiple-Level Association Rules

◼ Items often form hierarchies

◼ Flexible support settings
◼ Items at the lower level are expected to have lower
support
◼ Exploration of shared multi-level mining (Agrawal &
Srikant@VLB’95, Han & Fu@VLDB’95)

uniform support reduced support

Level 1 Milk
min_sup = 5% Level 1
[support = 10%] min_sup = 5%

Level 2 2% Milk Skim Milk Level 2

min_sup = 5% [support = 6%] [support = 4%] min_sup = 3%

41
 Multi-level Association: Flexible Support and
Redundancy filtering
◼ Flexible min-support thresholds: Some items are more valuable but
less frequent
◼ Use non-uniform, group-based min-support
◼ E.g., {diamond, watch, camera}: 0.05%; {bread, milk}: 5%; …
◼ Redundancy Filtering: Some rules may be redundant due to
“ancestor” relationships between items
◼ milk  wheat bread [support = 8%, confidence = 70%]
◼ 2% milk  wheat bread [support = 2%, confidence = 72%]
The first rule is an ancestor of the second rule
◼ A rule is redundant if its support is close to the “expected” value,
based on the rule’s ancestor

42
 Mining Multi-Dimensional Association
◼ Single-dimensional rules:
buys(X, “milk”)  buys(X, “bread”)
◼ Multi-dimensional rules:  2 dimensions or predicates
◼ Inter-dimension assoc. rules (no repeated predicates )
age(X,”19-25”)  occupation(X,“student”)  buys(X, “coke”)
◼ hybrid-dimension assoc. rules (repeated predicates)
age(X,”19-25”)  buys(X, “popcorn”)  buys(X, “coke”)
◼ Categorical Attributes: finite number of possible values, no
ordering among values—data cube approach
◼ Quantitative Attributes: Numeric, implicit ordering among
values—discretization, clustering, and gradient approaches

43
 Advanced Frequent Pattern Mining
◼ Pattern Mining in Multi-Level, Multi-Dimensional Space
◼ Mining Multi-Level Association
◼ Mining Multi-Dimensional Association
◼ Mining Quantitative Association Rules
◼ Mining Rare Patterns and Negative Patterns

44
 Mining Quantitative Associations

Techniques can be categorized by how numerical attributes,

such as age or salary are treated
1. Static discretization based on predefined concept
hierarchies (data cube methods)
2. Dynamic discretization based on data distribution
(quantitative rules, e.g., Agrawal & Srikant@SIGMOD96)
3. Clustering: Distance-based association (e.g., Yang &
Miller@SIGMOD97)
◼ One dimensional clustering then association
4. Deviation: (such as Aumann and Lindell@KDD99)
Sex = female => Wage: mean=$7/hr (overall mean = $9)

45
Static Discretization of Quantitative Attributes

◼ Discretized prior to mining using concept hierarchy.

◼ Numeric values are replaced by ranges
◼ In relational database, finding all frequent k-predicate sets
will require k or k+1 table scans
◼ Data cube is well suited for mining ()

◼ The cells of an n-dimensional

(age) (income) (buys)
cuboid correspond to the
predicate sets
(age, income) (age,buys) (income,buys)
◼ Mining from data cubes
can be much faster
(age,income,buys)
46
Quantitative Association Rules Based on Statistical
Inference Theory [Aumann and Lindell@DMKD’03]
◼ Finding extraordinary and therefore interesting phenomena, e.g.,
(Sex = female) => Wage: mean=$7/hr (overall mean = $9)
◼ LHS: a subset of the population
◼ RHS: an extraordinary behavior of this subset
◼ The rule is accepted only if a statistical test (e.g., Z-test) confirms the
inference with high confidence
◼ Subrule: highlights the extraordinary behavior of a subset of the pop.
of the super rule
◼ E.g., (Sex = female) ^ (South = yes) => mean wage = $6.3/hr
◼ Two forms of rules
◼ Categorical => quantitative rules, or Quantitative => quantitative rules
◼ E.g., Education in [14-18] (yrs) => mean wage = $11.64/hr
◼ Open problem: Efficient methods for LHS containing two or more
quantitative attributes
47
 Advanced Frequent Pattern Mining
◼ Pattern Mining in Multi-Level, Multi-Dimensional Space
◼ Mining Multi-Level Association
◼ Mining Multi-Dimensional Association
◼ Mining Quantitative Association Rules
◼ Mining Rare Patterns and Negative Patterns

48
 Negative and Rare Patterns
◼ Rare patterns: Very low support but interesting
◼ E.g., buying Rolex watches
◼ Mining: Setting individual-based or special group-based
support threshold for valuable items
◼ Negative patterns
◼ Since it is unlikely that one buys Ford Expedition (an
SUV car) and Toyota Prius (a hybrid car) together, Ford
Expedition and Toyota Prius are likely negatively
correlated patterns
◼ Negatively correlated patterns that are infrequent tend to
be more interesting than those that are frequent
49
Defining Negative Correlated Patterns (I)
◼ Definition 1 (support-based)
◼ If itemsets X and Y are both frequent but rarely occur together, i.e.,
sup(X U Y) < sup (X) * sup(Y)
◼ Then X and Y are negatively correlated
◼ Problem: A store sold two needle 100 packages A and B, only one
transaction containing both A and B.
◼ When there are in total 200 transactions, we have
s(A U B) = 0.005, s(A) * s(B) = 0.25, s(A U B) < s(A) * s(B)
◼ When there are 105 transactions, we have
s(A U B) = 1/105, s(A) * s(B) = 1/103 * 1/103, s(A U B) > s(A) * s(B)
◼ Where is the problem? —Null transactions, i.e., the support-based
definition is not null-invariant!

50
Defining Negative Correlated Patterns (II)
◼ Definition 2 (negative itemset-based)
◼ X is a negative itemset if (1) X = Ā U B, where B is a set of positive
items, and Ā is a set of negative items, |Ā|≥ 1, and (2) s(X) ≥ μ
◼ Itemsets X is negatively correlated, if

◼ This definition suffers a similar null-invariant problem

◼ Definition 3 (Kulzynski measure-based) If itemsets X and Y are
frequent, but (P(X|Y) + P(Y|X))/2 < є, where є is a negative pattern
threshold, then X and Y are negatively correlated.
◼ Ex. For the same needle package problem, when no matter there are
200 or 105 transactions, if є = 0.01, we have
(P(A|B) + P(B|A))/2 = (0.01 + 0.01)/2 < є
51
 Association Analysis

◼ Basic Concepts

◼ Frequent Itemset Mining Methods

◼ Apriori Algorithm

◼ FP-growth

◼ From Association Analysis to Correlation Analysis

◼ Pattern Mining in Multilevel Associations and

◼ Multidimensional Associations

◼ Summary
52
 Mining Colossal Frequent Patterns
◼ F. Zhu, X. Yan, J. Han, P. S. Yu, and H. Cheng, “Mining Colossal
Frequent Patterns by Core Pattern Fusion”, ICDE'07.
◼ We have many algorithms, but can we mine large (i.e., colossal)
patterns? ― such as just size around 50 to 100? Unfortunately, not!
◼ Why not? ― the curse of “downward closure” of frequent patterns
◼ The “downward closure” property
◼ Any sub-pattern of a frequent pattern is frequent.
◼ Example. If (a1, a2, …, a100) is frequent, then a1, a2, …, a100, (a1,
a2), (a1, a3), …, (a1, a100), (a1, a2, a3), … are all frequent! There
are about 2100 such frequent itemsets!
◼ No matter using breadth-first search (e.g., Apriori) or depth-first
search (FPgrowth), we have to examine so many patterns
◼ Thus the downward closure property leads to explosion!
53
 Colossal Patterns: A Motivating Example
Let’s make a set of 40 transactions Closed/maximal patterns may
T1 = 1 2 3 4 ….. 39 40 partially alleviate the problem but not
T2 = 1 2 3 4 ….. 39 40 really solve it: We often need to mine
: . scattered large patterns!
: . Let the minimum support threshold
: . σ= 20
: .
T40=1 2 3 4 ….. 39 40
Then delete the items on the diagonal
T1 = 2 3 4 ….. 39 40
T2 = 1 3 4 ….. 39 40
: .
: .
: .
: .
T40=1 2 3 4 …… 39
54
 Colossal Pattern Set: Small but Interesting

◼ It is often the case that

only a small number of
patterns are colossal,
i.e., of large size

◼ Colossal patterns are

usually attached with
greater importance than
those of small pattern
sizes

55
 Mining Colossal Patterns: Motivation and
Philosophy
◼ Motivation: Many real-world tasks need mining colossal patterns
◼ Micro-array analysis in bioinformatics (when support is low)

◼ Biological sequence patterns

◼ Biological/sociological/information graph pattern mining

◼ No hope for completeness

◼ If the mining of mid-sized patterns is explosive in size, there is no
hope to find colossal patterns efficiently by insisting “complete set”
mining philosophy
◼ Jumping out of the swamp of the mid-sized results
◼ What we may develop is a philosophy that may jump out of the
swamp of mid-sized results that are explosive in size and jump to
reach colossal patterns
◼ Striving for mining almost complete colossal patterns
◼ The key is to develop a mechanism that may quickly reach colossal
patterns and discover most of them
56
 Alas, A Show of Colossal Pattern Mining!

Let the min-support threshold σ= 20

T1 = 2 3 4 ….. 39 40  40 
Then there are 
 20 
 closed/maximal
T2 = 1 3 4 ….. 39 40  
: . frequent patterns of size 20
: .
: . However, there is only one with size
: . greater than 20, (i.e., colossal):
T40=1 2 3 4 …… 39
T41= 41 42 43 ….. 79 α= {41,42,…,79} of size 39
T42= 41 42 43 ….. 79
: . The existing fastest mining algorithms
: . (e.g., FPClose, LCM) fail to complete
T60= 41 42 43 … 79 running
Our algorithm outputs this colossal
pattern in seconds
57
 Methodology of Pattern-Fusion Strategy
◼ Pattern-Fusion traverses the tree in a bounded-breadth way
◼ Always pushes down a frontier of a bounded-size candidate
pool
◼ Only a fixed number of patterns in the current candidate pool
will be used as the starting nodes to go down in the pattern tree
― thus avoids the exponential search space
◼ Pattern-Fusion identifies “shortcuts” whenever possible
◼ Pattern growth is not performed by single-item addition but by
leaps and bounded: agglomeration of multiple patterns in the
pool
◼ These shortcuts will direct the search down the tree much more
rapidly towards the colossal patterns
58
Observation: Colossal Patterns and Core Patterns

Transaction Database D
A colossal pattern α
α D

α1 Dαk
α2
D
Dα1
α
Dα2

αk

Subpatterns α1 to αk cluster tightly around the colossal pattern α by

sharing a similar support. We call such subpatterns core patterns of α

59
 Robustness of Colossal Patterns
◼ Core Patterns
Intuitively, for a frequent pattern α, a subpattern β is a τ-core
pattern of α if β shares a similar support set with α, i.e.,

where τ is called the core ratio

◼ Robustness of Colossal Patterns
A colossal pattern is robust in the sense that it tends to have much
more core patterns than small patterns

60
 Colossal Patterns Correspond to Dense Balls

◼ Due to their robustness,

colossal patterns correspond to
dense balls
◼ Ω( 2^d) in population
◼ A random draw in the pattern
space will hit somewhere in the
ball with high probability

61
 Idea of Pattern-Fusion Algorithm

◼ Generate a complete set of frequent patterns up to a

small size
◼ Randomly pick a pattern β, and β has a high probability to
be a core-descendant of some colossal pattern α
◼ Identify all α’s descendants in this complete set, and
merge all of them ― This would generate a much larger
core-descendant of α
◼ In the same fashion, we select K patterns. This set of
larger core-descendants will be the candidate pool for the
next iteration

62
 Pattern-Fusion: The Algorithm

◼ Initialization (Initial pool): Use an existing algorithm to

mine all frequent patterns up to a small size, e.g., 3
◼ Iteration (Iterative Pattern Fusion):
◼ At each iteration, k seed patterns are randomly picked

from the current pattern pool

◼ For each seed pattern thus picked, we find all the

patterns within a bounding ball centered at the seed

pattern
◼ All these patterns found are fused together to generate

a set of super-patterns. All the super-patterns thus

generated form a new pool for the next iteration
◼ Termination: when the current pool contains no more
than K patterns at the beginning of an iteration
63
 Why Is Pattern-Fusion Efficient?

◼ A bounded-breadth pattern
tree traversal
◼ It avoids explosion in

mining mid-sized ones

◼ Randomness comes to help

to stay on the right path

◼ Ability to identify “short-cuts”
and take “leaps”
◼ fuse small patterns

together in one step to

generate new patterns of
significant sizes
◼ Efficiency

64
Pattern-Fusion Leads to Good Approximation

◼ Gearing toward colossal patterns

◼ The larger the pattern, the greater the chance it will
be generated
◼ Catching outliers
◼ The more distinct the pattern, the greater the chance
it will be generated

65
 Association Analysis

◼ Basic Concepts

◼ Frequent Itemset Mining Methods

◼ Apriori Algorithm

◼ FP-growth

◼ From Association Analysis to Correlation Analysis

◼ Pattern Mining in Multilevel Associations and

◼ Multidimensional Associations

◼ Summary
66
 Association Analysis

◼ Basic Concepts

◼ Frequent Itemset Mining Methods

◼ Apriori, FP Growth

◼ Correlation Analysis

◼ Multi level association rules

◼ High dimensional data

67
End Unit - 2

68