Chapter 1

Introduction to Data Mining

Syllabus:
What is Data Mining; Kind of patterns to be mined; Technologies used; Major issues in
Data Mining

What is Data Mining:

Data mining, the extraction of hidden predictive information from large
databases, is a powerful new technology with great potential to help companies
focus on the most important information in their data warehouses. Data mining
tools predict future trends and behaviors, allowing businesses to make proactive,
knowledge-driven decisions. Data mining techniques can be implemented
rapidly on existing software and hardware platforms to enhance the value of
existing information resources, and can be integrated with new products and
systems as they are brought on-line.

Fig: 1 Data mining as a process of Knowledge Discovery

Kind of patterns to be mined:
We have observed various types of data and information repositories on which data
mining can be performed. Let us now examine the kinds of patterns that can be mined.
There are a number of data mining functionalities. These include
 Class/ Concept : characterization and discrimination.
 Frequent patterns Mining.
 Associations Mining.
 Correlations Mining.
 Classification and Regression.
 Clustering analysis.
 Outlier analysis.
Data mining functionalities are used to specify the kinds of patterns to be found in
data mining tasks. In general, such tasks can be classified into two categories: descriptive
and predictive. Descriptive mining tasks characterize properties of the data in a target
data set. Predictive mining tasks perform induction on the current data in order to make
predictions.
Data mining functionalities, and the kinds of patterns they can discover, are described
below. What makes a pattern interesting? Interesting patterns represent knowledge.
 Class/ Concept : characterization and discrimination.
Data entries can be associated with classes or concepts. For example, in the
AllElectronics store, classes of items for sale include computers and printers, and
concepts of customers include bigSpenders and budgetSpenders. It can be useful to
describe individual classes and concepts in summarized, concise, and yet precise terms.
Such descriptions of a class or a concept are called class/concept descriptions. These
descriptions can be derived using (1) data characterization, by summarizing the data of
the class under study (often called the target class) in general terms, or (2) data
discrimination, by comparison of the target class with one or a set of comparative
classes (often called the contrasting classes), or (3) both data characterization and
discrimination.
(1) Data characterization is a summarization of the general characteristics or features
of a target class of data. The data corresponding to the user-specified class are typically
collected by a query. For example, to study the characteristics of software products with
sales that increased by 10% in the previous year, the data related to such products can be
collected by executing an SQL query on the sales database. There are several methods for
effective data summarization and characterization.
(2) Data discrimination is a comparison of the general features of the target class data
objects against the general features of objects from one or multiple contrasting classes.
The target and contrasting classes can be specified by a user, and the corresponding data
objects can be retrieved through database queries. For example, a user may want to
compare the general features of software products with sales that increased by 10% last
year against those with sales that decreased by at least 30% during the same period. The
methods used for data discrimination are similar to those used for data characterization.
 Frequent patterns Mining.
Frequent patterns, as the name suggests, are patterns that occur frequently in data.
There are many kinds of frequent patterns, including
 Frequent itemsets.
 Frequent sub-sequences (also known as sequential patterns).
 Frequent substructures.
 Frequent itemset :- A frequent itemset typically refers to a set of items that
often appear together in a transactional data set—for example, milk and
bread, which are frequently bought together in grocery stores by many
customers.
 Frequently sub-sequence :- A frequently occurring subsequence, such as the
pattern that customers, tend to purchase first a laptop, followed by a digital
camera, and then a memory card, is a (frequent) sequential pattern.
 Frequent substructure :- A substructure can refer to different structural forms
(e.g., graphs, trees, or lattices) that may be combined with itemsets or
subsequences. If a substructure occurs frequently, it is called a (frequent)
structured pattern.
 Association Mining
Suppose that, as a marketing manager at AllElectronics, you want to know which
items are frequently purchased together (i.e., within the same transaction). An
example of such a rule, mined from the AllElectronics transactional database, is
buys(X, “computer”) ⇒ buys(X, “software”) [support = 1%,confidence = 50%],

where X is a variable representing a customer. A confidence, or certainty, of 50%

 means that if a customer buys a computer, there is a 50% chance that she/he will buy
software as well. A 1% support means that 1% of all the transactions under analysis
show that computer and software are purchased together.
This association rule involves a single attribute or predicate (i.e., buys) that
repeats. Association rules that contain a single predicate are referred to as
single-dimensional association rules.
Suppose, instead, that we are given the AllElectronics relational database related
to purchases. A data mining system may find association rules like

age(X, “20..29”) ∧ income(X, “40K..49K”) ⇒ buys(X, “laptop”)

[support = 2%, confidence = 60%].

The rule indicates that of the AllElectronics customers under study, 2% are 20 to
29 years old with an income of $40,000 to $49,000 and have purchased a laptop
(computer) at AllElectronics. There is a 60% probability that a customer in this age
and income group will purchase a laptop.
This is an association involving more than one attribute or predicate (i.e., age,
income, and buys). The above rule can be referred to as a multidimensional
association rule.
 Correlation Mining
Additional analysis can be performed to uncover interesting statistical
correlations between associated attribute–value pairs. Correlation analysis will find
the relationship between two items. Strong association rule is not interesting to user,
so there is new technique called Correlation.
Example; Total transaction 10,000 in which computer game is 6,000 and videos
is 7,500. This condition is not satisfied and this rule is may not be interested, So we
choose Correlation analysis technique.
Methods in Correlation;

 Lift method
 X2 method
 Other method
 All confidence
 Cosine
 Classification and Regression for Predictive Analysis
Classification is the process of finding a model (or function) that describes
and distinguishes data classes or concepts. The model are derived based on the
analysis of a set of training data (i.e., data objects for which the class labels are
known). The model is used to predict the class label of objects for which the the
class label is unknown.

What is classification?
Examples of cases where the data analysis task is Classification −

 A bank loan officer wants to analyze the data in order to know which
customer (loan applicant) are risky or which are safe.

 A marketing manager at a company needs to analyze a customer with a

given profile, who will buy a new computer.

In both of the above examples, a model or classifier is constructed to predict

the categorical labels. These labels are risky or safe for loan application data and
yes or no for marketing data

What is prediction?

The examples of cases where the data analysis task is Prediction −

Suppose the marketing manager needs to predict how much a given

customer will spend during a sale at his company. In this example we are bothered to
predict a numeric value. Therefore the data analysis task is an example of numeric
prediction. In this case, a model or a predictor will be constructed that predicts a
continuous-valued-function or ordered value.

Note − Regression analysis is a statistical methodology that is most often used for
numeric prediction.

“How is the derived model presented?” The derived model may be represented in
various forms, such as classification rules i.e., IF-THEN rule, decision trees,
mathematical formula, or neural networks.

A decision tree is a flowchart-like tree structure, where each node denotes a test on
an attribute value, each branch represents an outcome of the test, and tree leaves represent
classes or class distributions. Decision trees can easily be converted to classification
rules.
 Regression analysis is a statistical methodology that is most often used for numeric
prediction, although other methods exist as well. Regression also encompasses the
identification of distribution trends based on the available data.

 Cluster Analysis

Unlike classification and regression, which analyze class-labeled (training) data

sets, clustering analyzes data objects without consulting class labels. In many cases,
class labeled data may simply not exist at the beginning. Clustering can be used to
generate class labels for a group of data. The objects are clustered or grouped based
on the principle of maximizing the intraclass similarity and minimizing the interclass
similarity. That is, clusters of objects are formed so that objects within a cluster have
high similarity in comparison to one another, but are rather dissimilar to objects in
other clusters.
 Each cluster so formed can be viewed as a class of objects, from which rules can
be derived. Clustering can also facilitate taxonomy formation, that is, the
organization of observations into a hierarchy of classes that group similar events
together.

 Outlier Analysis
A data set may contain objects that do not comply with the general behavior or
model of the data. These data objects are outliers. Many data mining methods discard
outliers as noise or exceptions. However, in some applications (e.g., fraud detection)
the rare events can be more interesting than the more regularly occurring ones. The
analysis of outlier data is referred to as outlier analysis or anomaly mining.
Outliers may be detected using statistical tests that assume a distribution or
probability model for the data, or using distance measures where objects that are
remote from any other cluster are considered outliers. Rather than using statistical or
distance measures, density-based methods may identify outliers in a local region,
although they look normal from a global statistical distribution view.
Example; Outlier analysis may uncover fraudulent usage of credit cards by
detecting purchases of unusually large amounts for a given account number in
comparison to regular charges incurred by the same account. Outlier values may also
 be detected with respect to the locations and types of purchase, or the purchase
frequency.

Technology Used in Data Mining:

As a highly application-driven domain, data mining has incorporated many
techniques from other domains such as

 Statistics
 Machine learning,
 Supervised learning
 Unsupervised learning
 Semi-supervised learning
 Active learning
 Database and data warehouse systems
 Information retrieval
 Visualization

 Statistics
Statistics studies the collection, analysis, interpretation or explanation,
and presentation of data. Data mining has an inherent connection with
statistics.
A statistical model is a set of mathematical functions that describe the
behavior of the objects in a target class in terms of random variables and
their associated probability distributions. Statistical models are widely used
to model data and data classes.
For example, in data mining tasks like data characterization and
classification, statistical models of target classes can be built. In other words,
such statistical models can be the outcome of a data mining task.
Statistics research develops tools for prediction and forecasting using
data and statistical models. Statistical methods can be used to summarize or
describe a collection of data.
Statistical methods can also be used to verify data mining results. For
example, after a classification or prediction model is mined, the model
should be verified by statistical hypothesis testing.
 A statistical hypothesis test (sometimes called confirmatory data
analysis) makes statistical decisions using experimental data.
 Machine Learning
Machine learning investigates how computers can learn (or improve
their performance) based on data.
For example, a typical machine learning problem is to program a
computer so that it can automatically recognize handwritten postal codes on
mail after learning from a set of examples.
 Supervised learning
Supervised learning is basically a synonym for classification. The
supervision in the learning comes from the labeled examples in the training
data set. For example, in the postal code recognition problem, a set of
handwritten postal code images and their corresponding machine-readable
translations are used as the training examples, which supervise the learning
of the classification model.
 Unsupervised learning
Unsupervised learning is essentially a synonym for clustering. The
learning process is unsupervised since the input examples are not class
labeled. Typically, we may use clustering to discover classes within the data.
For example, an unsupervised learning method can take, as input, a set
of images of handwritten digits. Suppose that it finds 10 clusters of data.
These clusters may correspond to the 10 distinct digits of 0 to 9, respectively.
However, since the training data are not labeled, the learned model cannot
tell us the semantic meaning of the clusters found.
 Semi-supervised learning
Semi-supervised learning is a class of machine learning techniques that
make use of both labeled and unlabeled examples when learning a model.
For a two-class problem, we can think of the set of examples belonging to
one class as the positive examples and those belonging to the other class as
the negative examples.
 Active learning
Active learning is a machine learning approach that lets users play an
active role in the learning process. An active learning approach can ask a
 user (e.g., a domain expert) to label an example, which may be from a set of
unlabeled examples or synthesized by the learning program.
The goal is to optimize the model quality by actively acquiring
knowledge from human users, given a constraint on how many examples
they can be asked to label.

You can see there are many similarities between data mining and machine
learning. For classification and clustering tasks, machine learning research
often focuses on the accuracy of the model. In addition to accuracy, data
mining research places strong emphasis on the efficiency and scalability of
mining methods on large data sets, as well as on ways to handle complex
types of data and explore new, alternative methods.
 Database Systems and Data Warehouses
Database systems research focuses on the creation, maintenance, and use of
databases for organizations and end-users. Particularly, database systems
researchers have established highly recognized principles in data models, query
languages, query processing and optimization methods, data storage, and
indexing and accessing methods. Database systems are often well known for
their high scalability in processing very large, relatively structured data sets.
Many data mining tasks need to handle large data sets or even real-time, fast
streaming data. Therefore, data mining can make good use of scalable database
technologies to achieve high efficiency and scalability on large data sets.
Moreover, data mining tasks can be used to extend the capability of existing
 database systems to satisfy advanced users’ sophisticated data analysis
requirements. A data warehouse integrates data originating from multiple sources
and various timeframes. It consolidates data in multidimensional space to form
partially materialized data cubes. The data cube model not only facilitates OLAP
in multidimensional databases but also promotes multidimensional data mining.
 Information Retrieval
Information retrieval (IR) is the science of searching for documents or
information in documents. Documents can be text or multimedia, and may reside
on the Web. The differences between traditional information retrieval and
database systems are twofold: Information retrieval assumes that (1) the data
under search are unstructured; and (2) the queries are formed mainly by
keywords, which do not have complex structures.

Major Issues in Data Mining:

The major issues in data mining research, partitioning them into five groups:

i. Mining methodology
ii. User interaction
iii. Efficiency and Scalability
iv. Diversity of data types
v. Data mining and society

Many of these issues have been addressed in recent data mining research and
development to a certain extent and are now considered data mining requirements; others
are still at the research stage. The issues continue to stimulate further investigation and
improvement in data mining.
i. Mining Methodology
This involves the investigation of new kinds of knowledge, mining in
multidimensional space, integrating methods from other disciplines, and the consideration
of semantic ties among data objects.
Mining methodologies should consider issues such as data uncertainty, noise,
and incompleteness. Some mining methods explore how user specified measures can be
used to assess the interestingness of discovered patterns as well as guide the discovery
process.
 Various aspects of mining methodology.
 Mining various and new kinds of knowledge
 Mining knowledge in multidimensional space
 Data mining—an interdisciplinary effort
 Boosting the power of discovery in a networked environment
 Handling uncertainty, noise, or incompleteness of data
 Pattern evaluation and pattern- or constraint-guided mining
 Mining various and new kinds of knowledge:
These tasks may use the same database in different ways and require the
development of numerous data mining techniques. Due to the diversity of applications,
new mining tasks continue to emerge, making data mining a dynamic and fast-growing
field.
For example, for effective knowledge discovery in information networks,
integrated clustering and ranking may lead to the discovery of high-quality clusters and
object ranks in large networks.
 Mining knowledge in multidimensional space:
When searching for knowledge in large data sets, we can explore the data in
multidimensional space. That is, we can search for interesting patterns among
combinations of dimensions (attributes) at varying levels of abstraction. Such mining is
known as (exploratory) multidimensional data mining. In many cases, data can be
aggregated or viewed as a multidimensional data cube. Mining knowledge in cube space
can substantially enhance the power and flexibility of data mining.
 Data mining—an interdisciplinary effort:
The power of data mining can be substantially enhanced by integrating new
methods from multiple disciplines. For example,to mine data with natural language text,
it makes sense to fuse data mining methods with methods of information retrieval and
natural language processing.
 Boosting the power of discovery in a networked environment:
Most data objects reside in a linked or interconnected environment, whether it
be the Web, database relations, files, or documents. Knowledge derived in one set of
objects can be used to boost the discovery of knowledge in a “related” or semantically
linked set of objects.
 Handling uncertainty, noise, or incompleteness of data:
Data often contain noise, errors, exceptions, or uncertainty, or are incomplete.
Errors and noise may confuse the data mining process, leading to the derivation of
erroneous patterns. Data cleaning, data preprocessing, outlier detection and removal, and
uncertainty reasoning are examples of techniques that need to be integrated with the data
mining process.
 Pattern evaluation and pattern- or constraint-guided mining:
All the patterns generated by data mining processes are interesting. What makes
a pattern interesting may vary from user to user. Therefore, techniques are needed to
assess the interestingness of discovered patterns based on subjective measures. These
estimate the value of patterns with respect to a given user class, based on user beliefs or
expectations.
ii. User Interaction
In User Interaction how to interact with a data mining system, how to
incorporate a user’s background knowledge in mining, and how to visualize and
comprehend data mining results. We introduce each of these here.

 Interactive mining
 Incorporation of background knowledge
 Ad hoc data mining and data mining query languages
 Presentation and visualization of data mining results

 Interactive mining:
The data mining process should be highly interactive. It is important to build
flexible user interfaces and an exploratory mining environment, facilitating the user’s
interaction with the system.
A user may like to first sample a set of data, explore general characteristics of
the data, and estimate potential mining results.
Interactive mining should allow users to dynamically change the focus of a
search, to refine mining requests based on returned results, and to drill, dice, and pivot
through the data and knowledge space interactively, dynamically exploring “cube space”
while mining.
 Incorporation of background knowledge:
Background knowledge, constraints, rules, and other information regarding the
domain under study should be incorporated into the knowledge discovery process. Such
knowledge can be used for pattern evaluation as well as to guide the search toward
interesting patterns.
 Ad hoc data mining and data mining query languages:
Query languages (e.g., SQL) have played an important role in flexible
searching because they allow users to pose ad hoc queries.
Similarly, high-level data mining query languages or other high-level flexible
user interfaces will give users the freedom to define ad hoc data mining tasks.
This should facilitate specification of the relevant sets of data for analysis, the
domain knowledge, the kinds of knowledge to be mined, and the conditions and
constraints to be enforced on the discovered patterns.
 Presentation and visualization of data mining results:
How can a data mining system present data mining results, vividly and flexibly,
so that the discovered knowledge can be easily understood and directly usable by humans?
This is especially crucial if the data mining process is interactive. It requires the system to
adopt expressive knowledge representations, user-friendly interfaces, and visualization
techniques.
iii. Efficiency and Scalability
Efficiency and scalability are always considered when comparing data mining
algorithms. As data amounts continue to multiply, these two factors are especially
critical.
 Efficiency and scalability of data mining algorithms
 Parallel, distributed, and incremental mining algorithms
 Efficiency and scalability of data mining algorithms:
Data mining algorithms must be efficient and scalable in order to effectively
extract information from huge amounts of data in many data repositories or in dynamic
data streams. In other words, the running time of a data mining algorithm must be
predictable, short, and acceptable by applications. Efficiency, scalability, performance,
optimization, and the ability to execute in real time are key criteria that drive the
development of many new data mining algorithms.
 Parallel, distributed, and incremental mining algorithms:
The humongous size of many data sets, the wide distribution of data, and the
computational complexity of some data mining methods are factors that motivate the
development of parallel and distributed data-intensive mining algorithms.
The high cost of some data mining processes and the incremental nature of
input promote incremental data mining, which incorporates new data updates without
having to mine the entire data “from scratch.”
iv. Diversity of Database Types
The wide diversity of database types brings about challenges to data mining.
These include,
 Handling complex types of data
 Mining dynamic, networked, and global data repositories
 Handling complex types of data
Diverse applications generate a wide spectrum of new data types, from
structured data such as relational and data warehouse data to semi-structured and
unstructured data; from stable data repositories to dynamic data streams; from simple
data objects to temporal data, biological sequences, sensor data, spatial data,
hypertext data, multimedia data, software program code, Web data, and social
network data.
It is unrealistic to expect one data mining system to mine all kinds of data, given
the diversity of data types and the different goals of data mining. Domain- or
application-dedicated data mining systems are being constructed for in depth mining
of specific kinds of data.
 Mining dynamic, networked, and global data repositories
Multiple sources of data are connected by the Internet and various kinds of
networks, forming gigantic, distributed, and heterogeneous global information
systems and networks.
The discovery of knowledge from different sources of structured,
semi-structured, or unstructured yet interconnected data with diverse data semantics
poses great challenges to data mining.
v. Data Mining and Society
How does data mining impact society? What steps can data mining take to
preserve the privacy of individuals? Do we use data mining in our daily lives without
even knowing that we do? These questions raise the following issues:

 Social impacts of data mining

 Privacy-preserving data mining
 Invisible data mining:

 Social impacts of data mining

The improper disclosure or use of data and the potential violation of individual
privacy and data protection rights are areas of concern that need to be addressed.
 Privacy-preserving data mining
Studies on privacy-preserving data publishing and data mining are ongoing. The
philosophy is to observe data sensitivity and preserve people’s privacy while
performing successful data mining.
 Invisible data mining
Intelligent search engines and Internet-based stores perform such invisible data
mining by incorporating data mining into their components to improve their
functionality and performance.
For example, when purchasing items online, users may be unaware that the store
is likely collecting data on the buying patterns of its customers, which may be used to
recommend other items for purchase in the future.