Lecture 1

Introduction to Data Mining

15-Jan-19 Abu Ahmed Ferdaus, CSE, DU. 1

This lesson includes:

o 1.2 What Is Data Mining?

o 1.4 What Kinds of Patterns Can Be Mined?
o 1.5 What Kinds of Technologies Are Used?

15-Jan-19 Abu Ahmed Ferdaus, CSE, DU. 2

1.2 What is Data Mining?
 Data mining is the process of discovering or extracting
interesting (non-trivial, implicit, previously unknown and
potentially useful) patterns and knowledge from
huge/large/massive amounts of data.
 The data sources can include databases, data warehouses,
the Web, other information repositories, or data that are
streamed into the system dynamically.

15-Jan-19 Abu Ahmed Ferdaus, CSE, DU. 3

1.2 Data Mining – a misnomer?
 To refer to the mining of gold from rocks or sand, we say
gold mining instead of rock or sand mining. Analogously,
data mining should have been more appropriately named
“knowledge mining from data,” which is unfortunately
somewhat long.
 However, the shorter term, knowledge mining may not
reflect the emphasis on mining from large amounts of data.
 Nevertheless, mining is a vivid term characterizing the
process that finds a small set of precious nuggets from a
great deal of raw material. Thus, such a misnomer carrying
both “data” and “mining” became a popular choice.
 Alternative names: Knowledge discovery (mining) in
databases (KDD), knowledge extraction, data/pattern
analysis, data archaeology, data dredging, information
harvesting, business intelligence etc.

15-Jan-19 Abu Ahmed Ferdaus, CSE, DU. 4

1.2 KDD Process and Data Mining
 Many people treat data mining as a synonym for another
popularly used term, knowledge discovery in database,
or KDD, while others view data mining as merely an
essential step in the process of knowledge discovery.
 The knowledge discovery process includes following steps:
 1. Data cleaning (to remove noise and inconsistent data)
 2. Data integration (where multiple data sources may be
combined)
 3. Data selection (where data relevant to the analysis
task are retrieved from the database)
 4. Data transformation (where data are transformed and
consolidated into forms appropriate for mining by
performing summary or aggregation operations)
 5. Data mining (an essential process where intelligent
methods are applied to extract data patterns)

15-Jan-19 Abu Ahmed Ferdaus, CSE, DU. 5

1.2 KDD Process and Data Mining
 6. Pattern evaluation (to identify the truly interesting
patterns representing knowledge based on interestingness
measures)
 7. Knowledge presentation (where visualization and
knowledge representation techniques are used to present
mined knowledge to users)
 Steps 1 through 4 are different forms of data
preprocessing, where data are prepared for mining.

15-Jan-19 Abu Ahmed Ferdaus, CSE, DU. 6

1.2 KDD Process

15-Jan-19 Abu Ahmed Ferdaus, CSE, DU. 7

 Knowledge Discovery (KDD) Process
 This is a view from typical
database systems and data
warehousing communities
Pattern Evaluation

Data Mining

Task-relevant Data

Data Warehouse Selection

Data Cleaning

Data Integration

8
Databases
1.4 What Kinds of Patterns can be Mined?
 Data mining functionalities are used to specify the kinds of
patterns to be found in data mining tasks. There are a
number of data mining functionalities. These include:
 The mining of frequent patterns, associations, and
Correlations
 Classification and regression
 Clustering analysis and
 Outlier analysis

15-Jan-19 Abu Ahmed Ferdaus, CSE, DU. 9

1.4.2 Mining Frequent Patterns, Associations, and Correlations

 Frequent patterns, as the name suggests, are patterns

that occur frequently in data.
 There are many kinds of frequent patterns, including
 Frequent itemsets,
 Frequent subsequences (known as sequential patterns), and
 Frequent substructures.
 A frequent itemset typically refers to a set of items that
often appear together in a transactional data set—for
example, milk and bread, which are frequently bought
together in grocery stores by many customers.
 A frequently occurring subsequence, such as the pattern
that customers, tend to purchase first a laptop, followed by
a digital camera, and then a memory card, is a (frequent)
sequential pattern.

15-Jan-19 Abu Ahmed Ferdaus, CSE, DU. 10

1.4.2 Mining Frequent Patterns, Associations, and Correlations

 A substructure can refer to different structural forms (e.g.,

graphs, trees, or lattices) that may be combined with
itemsets or subsequences. If a substructure occurs
frequently, it is called a (frequent) structured pattern.
 Mining frequent patterns leads to the discovery of
interesting associations and correlations within data.

15-Jan-19 Abu Ahmed Ferdaus, CSE, DU. 11

1.4.2 Association Analysis
 Suppose that, you to know which items are frequently
purchased together (i.e., within the same transaction).
 An example of such a rule, mined from a transactional
database, is

where X is a variable representing a customer. A

confidence, or certainty, of 50% means that if a customer
buys a computer, there is a 50% chance that she will buy
software as well.
 A 1% support means that 1% of all the transactions under
analysis show that computer and software are purchased
together.
 This association rule involves a single attribute or predicate
(i.e., buys) that repeats. Association rules that contain a
single predicate are referred to as single-dimensional
association rules.
15-Jan-19 Abu Ahmed Ferdaus, CSE, DU. 12
1.4.2 Association Analysis
 Dropping the predicate notation, the rule can be written
simply as

 Suppose, instead, that we are given the relational database

related to purchases. A data mining system may find
association rules like

 The rule indicates that of the all customers under study,

2% are 20 to 29 years old with an income of $40,000 to
$49,000 and have purchased a laptop (computer).
 There is a 60% probability that a customer in this age and
income group will purchase a laptop.

15-Jan-19 Abu Ahmed Ferdaus, CSE, DU. 13

1.4.2 Association Analysis
 Note that this is an association involving more than one
attribute or predicate (i.e., age, income, and buys).
Adopting the terminology used in multidimensional
databases, where each attribute is referred to as a
dimension, the above rule can be referred to as a
multidimensional association rule.
 Typically, association rules are discarded as uninteresting if
they do not satisfy both a minimum support threshold
and a minimum confidence threshold.
 Additional analysis can be performed to uncover interesting
statistical correlations between associated attribute–value
pairs.
 Frequent itemset mining is a fundamental form of frequent
pattern mining.

15-Jan-19 Abu Ahmed Ferdaus, CSE, DU. 14

Association Analysis : Support & Conf.
 Let I = { i1,i2,…,im } be a set of items
 Let D = { t1,t2…,tn } be a set of transactions where every
ti is a set of items such that ti Є I
 Each transaction is associated with an id TID

 An association rule is of the form

A B, where A Є I, B Є I and A ∩ B = Ø
 A rule A B, has support s means
- the percentage of transactions in D that contains A U B
s=P(AUB)
 A rule A B, has confidence c means
- the percentage of transaction in D containing A that also
contains B,
c = P ( B | A ) = P(A U B) / P(A)

15-Jan-19 Abu Ahmed Ferdaus, CSE, DU. 15

Association Analysis : Support & Conf.

 Support of A C is
s = n(AUC)/n(D) TID Itemset
s = 2/4 X 100% = 50%
100 A,B,C
 Confidence of A C is
c = n(AUC) / n(A) 200 A,C
c = 2/3 X 100% = 66.6%

 Confidence of C A is
300 A,D
c = n(AUC) / n(C)
c = 2/2 X 100% = 100% 400 B,E,F

15-Jan-19 Abu Ahmed Ferdaus, CSE, DU. 16

1.4.3 Classification and Regression for Predictive Analysis
 Classification is the process of finding a model (or
function) that describes and distinguishes data classes or
concepts.
 The model are derived based on the analysis of a set of
training data (i.e., data objects for which the class labels
are known). The model is used to predict the class label of
objects for which the class label is unknown.
 The derived model may be represented in various forms:
 classification rules (i.e., IF-THEN rules),
 decision trees,
 mathematical formulae, or
 neural networks
 A decision tree is a flowchart-like tree structure, where
each node denotes a test on an attribute value, each
branch represents an outcome of the test, and tree leaves
represent classes or class distributions.
15-Jan-19 Abu Ahmed Ferdaus, CSE, DU. 17
1.4.3 Classification and Regression for Predictive Analysis

15-Jan-19 Abu Ahmed Ferdaus, CSE, DU. 18

1.4.3 Classification and Regression for Predictive Analysis

15-Jan-19 Abu Ahmed Ferdaus, CSE, DU. 19

1.4.3 Classification and Regression for Predictive Analysis

15-Jan-19 Abu Ahmed Ferdaus, CSE, DU. 20

1.4.3 Classification and Regression for Predictive Analysis
 Decision trees can easily be converted to classification
rules.
 A neural network, when used for classification, is
typically a collection of neuron-like processing units with
weighted connections between the units.
 There are many other methods for constructing
classification models, such as naïve Bayesian classification,
support vector machines (SVM), and k-nearest-neighbor
classification.
 Whereas classification predicts categorical (discrete,
unordered) labels, regression models continuous-valued
functions.
 That is, regression is used to predict missing or unavailable
numerical data values rather than (discrete) class labels.
The term prediction refers to both numeric prediction and
class label prediction.
15-Jan-19 Abu Ahmed Ferdaus, CSE, DU. 21
1.4.3 Classification and Regression for Predictive Analysis
 Regression analysis is a statistical methodology that is
most often used for numeric prediction, although other
methods exist as well.
 Regression also encompasses the identification of
distribution trends based on the available data.
 Classification and regression may need to be preceded by
relevance analysis, which attempts to identify attributes
that are significantly relevant to the classification and
regression process. Such attributes will be selected for the
classification and regression process. Other attributes,
which are irrelevant, can then be excluded from
consideration.

15-Jan-19 Abu Ahmed Ferdaus, CSE, DU. 22

1.4.4 Cluster Analysis
 Unlike classification and regression, which analyze class-
labeled (training) data sets, clustering analyzes data
objects without consulting class labels.
 In many cases, class labeled data may simply not exist at
the beginning. Clustering can be used to generate class
labels for a group of data.
 The objects are clustered or grouped based on the principle
of maximizing the intraclass similarity and minimizing the
interclass similarity.
 That is, clusters of objects are formed so that objects
within a cluster have high similarity in comparison to one
another, but are rather dissimilar to objects in other
clusters.
 Each cluster so formed can be viewed as a class of objects,
from which rules can be derived. Clustering can also
facilitate taxonomy formation, that is, the organization
15-Jan-19 Abu Ahmed Ferdaus, CSE, DU. 23
1.4.4 Cluster Analysis
of observations into a hierarchy of classes that group
similar events together.
 Example of Cluster analysis: Cluster analysis can be
performed on customer data to identify homogeneous
subpopulations of customers. These clusters may represent
individual target groups for marketing.
 Figure 1.10 shows a 2-D plot of customers with respect to
customer locations in a city. Three clusters of data points
are evident.

15-Jan-19 Abu Ahmed Ferdaus, CSE, DU. 24

1.4.4 Cluster Analysis

15-Jan-19 Abu Ahmed Ferdaus, CSE, DU. 25

1.4.5 Outlier Analysis
 A data set may contain objects that do not comply with the
general behavior or model of the data. These data objects
are outliers.
 Many data mining methods discard outliers as noise or
exceptions. However, in some applications (e.g., fraud
detection) the rare events can be more interesting than the
more regularly occurring ones. The analysis of outlier data
is referred to as outlier analysis or anomaly mining.
 Outliers may be detected using statistical tests that
assume a distribution or probability model for the data, or
using distance measures where objects that are remote
from any other cluster are considered outliers.
 Outlier analysis may uncover fraudulent usage of credit
cards by detecting purchases of unusually large amounts
for a given account number in comparison to regular
charges incurred by the same account. Outlier values may
also be detected with respect to the locations and types of
purchase, or the purchase
15-Jan-19 frequency.
Abu Ahmed Ferdaus, CSE, DU. 26
1.4.6 Are All Patterns Interesting?
 A data mining system has the potential to generate
thousands or even millions of patterns, or rules.
 “Are all of the patterns interesting?”
 Typically, the answer is no—only a small fraction of the
patterns potentially generated would actually be of interest
to a given user.
 This raises some serious questions for data mining:
 1. What makes a pattern interesting?
 2. Can a data mining system generate all of the interesting
patterns? Or,
 3. Can the system generate only the interesting ones?
 A pattern is interesting if it is
 (1) easily understood by humans,
 (2) valid on new or test data with some degree of certainty,
 (3) potentially useful, and
 (4) novel.
15-Jan-19 Abu Ahmed Ferdaus, CSE, DU. 27
1.4.6 Are All Patterns Interesting?
 A pattern is also interesting if it validates a hypothesis that
the user sought (required) to confirm. An interesting
pattern represents knowledge.
 Interestingness measures are: 1. Objective and 2.
Subjective
 Several objective measures of pattern interestingness
exist. These are based on the structure of discovered
patterns and the statistics underlying them.
 An objective measure for association rules of the form X 
Y is rule support, representing the percentage of
transactions from a transaction database that the given
rule satisfies.
 This is taken to be the probability P(X  Y), where X  Y
indicates that a transaction contains both X and Y, that is,
the union of itemsets X and Y.

15-Jan-19 Abu Ahmed Ferdaus, CSE, DU. 28

1.4.6 Are All Patterns Interesting?
 Another objective measure for association rules is
confidence, which assesses the degree of certainty of the
detected association.
 This is taken to be the conditional probability P(Y | X), that
is, the probability that a transaction containing X also
contains Y.
 More formally, support and confidence are defined as

 In general, each interestingness measure is associated with

a threshold, which may be controlled by the user.
 For example, rules that do not satisfy a confidence
threshold of, say, 50% can be considered uninteresting.
Rules below the threshold likely reflect noise, exceptions,
or minority cases and are probably of less value.

15-Jan-19 Abu Ahmed Ferdaus, CSE, DU. 29

1.4.6 Are All Patterns Interesting?
 Other objective interestingness measures include accuracy
and coverage for classification (IF-THEN) rules.
 In general terms, accuracy tells us the percentage of data
that are correctly classified by a rule. Coverage is similar to
support, in that it tells us the percentage of data to which a
rule applies.
 Although objective measures help identify interesting
patterns, they are often insufficient unless combined with
subjective measures that reflect a particular user’s needs
and interests.
 For example, patterns describing the characteristics of
customers who shop frequently should be interesting to the
marketing manager, but may be of little interest to other
analysts studying the same database for patterns on
employee performance.

15-Jan-19 Abu Ahmed Ferdaus, CSE, DU. 30

1.4.6 Subjective Interestingness
 Furthermore, many patterns that are interesting by
objective standards may represent common sense and,
therefore, are actually uninteresting.
 Subjective interestingness measures are based on user
beliefs in the data. These measures find patterns
interesting if the patterns are unexpected (contradicting a
user’s belief) or offer strategic information on which the
user can act.
 In the latter case, such patterns are referred to as
actionable. For example, patterns like “a large earthquake
often follows a cluster of small quakes” may be highly
actionable if users can act on the information to save lives.
 Patterns that are expected can be interesting if they
confirm a hypothesis that the user wishes to validate or
they resemble a user’s hunch (guess or intuition).

15-Jan-19 Abu Ahmed Ferdaus, CSE, DU. 31

1.4.6 Are All Patterns Interesting?
 The second question—“Can a data mining system generate
all of the interesting patterns?”
 It refers to the completeness of a data mining algorithm.
It is often unrealistic and inefficient for data mining
systems to generate all possible patterns.
 Instead, user provided constraints and interestingness
measures should be used to focus the search.
 For some mining tasks, such as association, this is often
sufficient to ensure the completeness of the algorithm.
 Association rule mining is an example where the use of
constraints and interestingness measures can ensure the
completeness of mining.

15-Jan-19 Abu Ahmed Ferdaus, CSE, DU. 32

1.4.6 Are All Patterns Interesting?
 Finally, the third question—“Can a data mining system
generate only interesting patterns?”—
 It is an optimization problem in data mining. It is highly
desirable for data mining systems to generate only
interesting patterns.
 This would be efficient for users and data mining systems
because neither would have to search through the patterns
generated to identify the truly interesting ones.
 Progress has been made in this direction; however, such
optimization remains a challenging issue in data mining.
 Measures of pattern interestingness are essential for
filtering out the uninteresting ones.
 More important, such measures can be used to guide and
constrain the discovery process, improving the search
efficiency by pruning away subsets of the pattern space
that do not satisfy pre-specified interestingness constraints.
15-Jan-19 Abu Ahmed Ferdaus, CSE, DU. 33
1.5 Which Technologies Are Used?
 As a highly application-driven domain, data mining has
incorporated many techniques from other domains such as
 Statistics,
 Machine learning,
 Pattern recognition,
 Database and data warehouse systems,
 Information retrieval,
 Visualization,
 Algorithms,
 High-performance computing etc.
 The interdisciplinary nature of data mining research and
development contributes significantly to the success of
data mining and its extensive applications.

15-Jan-19 Abu Ahmed Ferdaus, CSE, DU. 34

1.5 Which Technologies Are Used?

15-Jan-19 Abu Ahmed Ferdaus, CSE, DU. 35

 1.5.1 Statistics
 Statistics studies the collection, analysis, interpretation or
explanation, and presentation of data. Data mining has an
inherent connection with statistics.
 A statistical model is a set of mathematical functions that
describe the behavior of the objects in a target class in
terms of random variables and their associated probability
distributions. Statistical models are widely used to model
data and data classes.
 For example, in data mining tasks like data characterization
and classification, statistical models of target classes can be
built. In other words, such statistical models can be the
outcome of a data mining task. Alternatively, data mining
tasks can be built on top of statistical models.
 For example, we can use statistics to model noise and
missing data values. Then, when mining patterns in a large
data set, the data mining process can use the model to help
identify and handle noisy or missing values in the data.
15-Jan-19 Abu Ahmed Ferdaus, CSE, DU. 36
 1.5.2 Machine Learning
 Machine learning investigates how computers can learn
(or improve their performance) based on data.
 A main research area is for computer programs to
automatically learn to recognize complex patterns and
make intelligent decisions based on data.
 For example, a typical machine learning problem is to
program a computer so that it can automatically recognize
handwritten postal codes on mail after learning from a set
of examples.
 Machine learning is a fast-growing discipline. The learning
process can be:
 Supervised learning
 Unsupervised learning
 Semi-supervised learning
 Active learning

15-Jan-19 Abu Ahmed Ferdaus, CSE, DU. 37

 1.5.2 Machine Learning
 Supervised learning is basically a synonym for
classification. The supervision in the learning comes from
the labeled examples in the training data set.
 For example, in the postal code recognition problem, a set
of handwritten postal code images and their corresponding
machine-readable translations are used as the training
examples, which supervise the learning of the classification
model.
 Unsupervised learning is essentially a synonym for
clustering. The learning process is unsupervised since the
input examples are not class labeled. Typically, we may use
clustering to discover classes within the data.
 For example, an unsupervised learning method can take, as
input, a set of images of handwritten digits. Suppose that it
finds 10 clusters of data. These clusters may correspond to
the 10 distinct digits of 0 to 9, respectively.

15-Jan-19 Abu Ahmed Ferdaus, CSE, DU. 38

 1.5.2 Machine Learning
 However, since the training data are not labeled, the
learned model cannot tell us the semantic meaning of the
clusters found.
 Semi-supervised learning is a class of machine learning
techniques that make use of both labeled and unlabeled
examples when learning a model.
 In one approach, labeled examples are used to learn class
models and unlabeled examples are used to refine the
boundaries between classes.
 For a two-class problem, we can think of the set of
examples belonging to one class as the positive examples
and those belonging to the other class as the negative
examples.
 In Figure 1.12, if we do not consider the unlabeled
examples, the dashed line is the decision boundary that
best partitions the positive examples from the negative
examples.
15-Jan-19 Abu Ahmed Ferdaus, CSE, DU. 39
 1.5.2 Machine Learning
 Using the unlabeled examples, we can refine the decision
boundary to the solid line. Moreover, we can detect that the
two positive examples at the top right corner, though
labeled, are likely noise or outliers.

15-Jan-19 Abu Ahmed Ferdaus, CSE, DU. 40

 1.5.2 Machine Learning
 Active learning is a machine learning approach that lets
users play an active role in the learning process.
 An active learning approach can ask a user (e.g., a domain
expert) to label an example, which may be from a set of
unlabeled examples or synthesized by the learning program.
 The goal is to optimize the model quality by actively
acquiring knowledge from human users, given a constraint
on how many examples they can be asked to label.
 There are many similarities between data mining and
machine learning. For classification and clustering tasks,
machine learning research often focuses on the accuracy of
the model.
 In addition to accuracy, data mining research places strong
emphasis on the efficiency and scalability of mining methods
on large data sets, as well as on ways to handle complex
types of data and explore new, alternative methods.
15-Jan-19 Abu Ahmed Ferdaus, CSE, DU. 41
 1.5.3 Database Systems and Data Warehouses
 Database systems research focuses on the creation,
maintenance, and use of databases for organizations and
end-users.
 Particularly, database systems researchers have established
highly recognized principles in data models, query
languages, query processing and optimization methods,
data storage, and indexing and accessing methods.
 Database systems are often well known for their high
scalability in processing very large, relatively structured data
sets.
 Many data mining tasks need to handle large data sets or
even real-time, fast streaming data.
 Therefore, data mining can make good use of scalable
database technologies to achieve high efficiency and
scalability on large data sets. Moreover, data mining tasks
can be used to extend the capability of existing database
15-Jan-19 Abu Ahmed Ferdaus, CSE, DU. 42
 1.5.3 Database Systems and Data Warehouses
systems to satisfy advanced users’ sophisticated data
analysis requirements.
 Recent database systems have built systematic data
analysis capabilities on database data using data
warehousing and data mining facilities.
 A data warehouse integrates data originating from
multiple sources and various timeframes. It consolidates
data in multidimensional space to form partially materialized
data cubes.
 The data cube model not only facilitates OLAP in multi-
dimensional databases but also promotes multidimensional
data mining.

15-Jan-19 Abu Ahmed Ferdaus, CSE, DU. 43

 1.5.4 Information Retrieval
 Information retrieval (IR) is the science of searching for
documents or information in documents.
 Documents can be text or multimedia, and may reside on
the Web. The differences between traditional information
retrieval and database systems are twofold:
 Information retrieval assumes that (1) the data under
search are unstructured; and (2) the queries are formed
mainly by keywords, which do not have complex structures.
 Increasingly large amounts of text and multimedia data
have been accumulated and made available online due to
the fast growth of the Web and applications such as digital
libraries, digital governments, and health care information
systems.
 Their effective search and analysis have raised many
challenging issues in data mining. Therefore, text mining
and multimedia data mining, integrated with information
retrieval methods, have become increasingly important.
15-Jan-19 Abu Ahmed Ferdaus, CSE, DU. 44
 1.8 Summary
 Data mining is the process of discovering interesting
patterns from massive amounts of data. As a knowledge
discovery process, it typically involves data cleaning, data
integration, data selection, data transformation, pattern
discovery, pattern evaluation, and knowledge presentation.
 A pattern is interesting if it is valid on test data with some
degree of certainty, novel, potentially useful (e.g., can be
acted on or validates a hunch about which the user was
curious), and easily understood by humans. Interesting
patterns represent knowledge. Measures of pattern
interestingness, either objective or subjective, can be
used to guide the discovery process.

15-Jan-19 Abu Ahmed Ferdaus, CSE, DU. 45

 1.8 Summary
A data warehouse is a repository for long-term storage of
data from multiple sources, organized so as to facilitate
management decision making. The data are stored under a
unified schema and are typically summarized. Data
warehouse systems provide multidimensional data analysis
capabilities, collectively referred to as online analytical
processing (OLAP).
 Multidimensional data mining (also called exploratory
multidimensional data mining) integrates core data
mining techniques with OLAP-based multidimensional
analysis. It searches for interesting patterns among
multiple combinations of dimensions (attributes) at varying
levels of abstraction, thereby exploring multidimensional
data space.

15-Jan-19 Abu Ahmed Ferdaus, CSE, DU. 46

 1.8 Summary
 Data mining functionalities are used to specify the kinds
of patterns or knowledge to be found in data mining tasks.
The functionalities include characterization and discrimi-
nation; the mining of frequent patterns, associations, and
correlations; classification and regression; cluster analysis;
and outlier detection. As new types of data, new
applications, and new analysis demands continue to
emerge, there is no doubt we will see more and more novel
data mining tasks in the future (EDM).
 Data mining, as a highly application-driven domain, has
incorporated technologies from many other domains.
These include statistics, machine learning, database and
data warehouse systems, and information retrieval. The
interdisciplinary nature of data mining research and
development contributes significantly to the success of
data mining and its extensive applications.

15-Jan-19 Abu Ahmed Ferdaus, CSE, DU. 47

 Thank You

15-Jan-19 Abu Ahmed Ferdaus, CSE, DU. 48