Data Mining and KDD

Christopher Kalolo ([email protected])

Bernada Sianga ([email protected])
What is Data Mining?
• Data mining refers to the process of discovering meaningful patterns,
trends, and insights from large datasets using various techniques,
including statistical analysis, machine learning, and artificial
intelligence.
• It involves extracting valuable knowledge from vast amounts of data
to support decision-making and solve complex problems.
Patterns in Data Mining - Characterization
• Description:
• Characterization provides a summarization of the general characteristics or features of a
target class of data.
• It involves generating descriptive summaries about the data.
• Example Techniques:
• Data aggregation
• Summarization tools (e.g., SQL aggregation functions)
• Examples:
• Summarizing the characteristics of software products that had a sales increase of 10% in the
previous year.
• Describing the common features of customers who spend more than $5000 a year at a
specific company.
• Objective:
• To provide an overview or summary of data attributes for a specific class or subset of data.
Patterns in Data Mining - Discrimination
• Description:
• Discrimination is the process of comparing the general features of target class data objects
against the general features of objects from one or multiple contrasting classes.
• It aims to identify differences between classes.
• Example Techniques:
• Contrast sets
• Discriminant analysis
• Examples:
• Comparing the characteristics of regular computer product shoppers (e.g., more than twice a
month) with those who shop rarely (e.g., less than three times a year).
• Distinguishing features between high-value customers and low-value customers in a retail
dataset.
• Objective:
• To find attributes or features that distinguish one class from another, often used for decision-
making and classification.
Patterns in Data Mining – Frequent Patterns
• Description:
• Frequent patterns identify items, sequences, or substructures that appear frequently in a
dataset.
• This includes frequent itemsets, subsequences, and subgraphs.
• Example Techniques:
• Apriori algorithm
• FP-Growth algorithm
• Examples:
• Finding itemsets that frequently appear together in transaction data, such as "milk and
bread".
• Identifying common sequences of events in weblogs or customer behavior data.
• Objective:
• To uncover regularities and commonalities in the data, which can be useful for market basket
analysis, sequence mining, and other applications.
Patterns in Data Mining – Association
• Description:
• Association rules mining discovers interesting relationships (associations) among data items.
• It is used to find rules that imply certain item combinations occur together with a certain
probability.
• Example Techniques:
• Association rule learning (e.g., Apriori, Eclat)
• Measures like support, confidence, and lift
• Examples:
• Identifying that customers who buy diapers also frequently buy baby formula.
• Discovering that if a customer purchases a laptop, there is a high probability they will also
buy a laptop bag.
• Objective:
• To identify and quantify relationships between items in large datasets, often used for
recommendation systems and market basket analysis.
Summary to key Patterns
1. Characterization: Summarizes general characteristics of a target class.
• Example: Summarize customers spending over $5000/year.
2. Discrimination: Compares features between target and contrasting
classes.
• Example: Compare frequent vs. infrequent computer product shoppers.
3. Frequent Patterns: Identifies items or events that occur frequently
together.
• Example: Milk and bread bought together frequently.
4. Association: Discovers co-occurrence of items with certain probabilities.
• Example: Diapers and baby formula bought together.
Data Mining Tasks and Techniques
• Data mining involves various tasks and techniques aimed at discovering
patterns, associations, changes, anomalies, and statistically significant
structures and events in data.
• Data Mining Tasks:
• Data mining tasks refer to the specific objectives or problems that data mining aims
to solve.
• They represent the goals or outcomes that the data mining process is intended to
achieve.
• Data Mining Techniques:
• Data mining techniques are the specific methods or algorithms used to perform data
mining tasks.
• These techniques are the tools that implement the strategies necessary to achieve
the objectives set by the data mining tasks.
Data Mining Tasks and Techniques
• Data mining tasks are generally divided into two major categories:
• Predictive tasks (Use some attributes to predict unknown or future values of other
attributes)
• Classification
• Regression
• Anomaly Detection
• Descriptive tasks (Find human-interpretable patterns that describe the data)
• Association Rule Mining
• Clustering
• Summarization
• Text Mining
Data Mining Tasks - Classification
• Task Description:
• Classification is the process of identifying the category or class label of new
observations based on a training dataset containing observations whose category
membership is known.
• Techniques:
• Decision Trees: Splits data into branches to build a model of decisions.
• Naive Bayes: Uses Bayes’ theorem for probabilistic classification.
• Support Vector Machines (SVM): Finds the hyperplane that best separates classes.
• k-Nearest Neighbors (k-NN): Classifies based on the majority class among the k
closest observations.
• Neural Networks: Uses layers of interconnected nodes to model complex patterns.
• Example: Classifying emails as spam or non-spam.
Data Mining Tasks - Regression
• Task Description:
• Regression is used to predict a continuous value.
• It models the relationship between a dependent (target) variable and one or more
independent (predictor) variables.
• Techniques:
• Linear Regression: Models the relationship between two variables by fitting a linear
equation.
• Polynomial Regression: Models the relationship between variables as an nth degree
polynomial.
• Ridge and Lasso Regression: Adds regularization terms to linear regression to prevent
overfitting.
• Support Vector Regression (SVR): Uses SVM principles for regression tasks.
• Neural Networks: Uses backpropagation to fit complex regression models.
• Example: Predicting house prices based on features like area, number of
bedrooms, and age of the house.
Data Mining Tasks - Clustering
• Task Description:
• Clustering involves grouping a set of objects in such a way that objects in the same
group (cluster) are more similar to each other than to those in other groups.
• Techniques:
• K-Means: Partitions data into k clusters by minimizing within-cluster variance.
• Hierarchical Clustering: Builds a tree of clusters by either a bottom-up or top-down
approach.
• DBSCAN (Density-Based Spatial Clustering of Applications with Noise): Finds
clusters based on density and can handle noise.
• Gaussian Mixture Models (GMM): Uses probabilistic models to represent normally
distributed subpopulations.
• Example: Segmenting customers based on purchasing behavior.
Data Mining Tasks - Association Rule Mining
• Task Description:
• Association rule mining identifies interesting relationships (associations)
between variables in large datasets.
• Techniques:
• Apriori Algorithm: Generates frequent itemsets and association rules.
• FP-Growth (Frequent Pattern Growth): Builds a compact data structure to
find frequent itemsets without candidate generation.
• Eclat Algorithm: Uses a depth-first search strategy for mining frequent
itemsets.
• Example: Identifying items that are frequently purchased together in
a supermarket.
Data Mining Tasks – Anomaly Detection
• Task Description:
• Anomaly detection aims to identify rare items, events, or observations that
raise suspicions by differing significantly from the majority of the data.
• Techniques:
• Statistical Methods: Detects anomalies based on statistical deviations.
• Isolation Forest: Isolates anomalies by randomly selecting features and
splitting data.
• One-Class SVM: Models the normal data distribution and identifies
deviations.
• Autoencoders: Uses neural networks to reconstruct input data and identify
deviations based on reconstruction error.
• Example: Detecting fraudulent transactions in credit card data.
Data Mining Tasks – Sequence Mining
• Task Description:
• Sequence mining discovers frequent sequences of events or items over time.
• Techniques:
• GSP (Generalized Sequential Pattern): Finds frequent sequences based on
user-specified constraints.
• PrefixSpan: Uses a prefix-based approach to find sequential patterns.
• Example: Identifying common sequences of web page visits on a
website.
Data Mining Tasks – Summarization
• Task Description:
• Summarization involves creating a compact representation of the dataset,
often providing a high-level overview or summary of the data.
• Techniques:
• Descriptive Statistics: Uses measures like mean, median, and standard
deviation.
• OLAP (Online Analytical Processing): Provides multidimensional data analysis
and summarization.
• Example: Generating a summary report of sales performance by
region and product category.
Data Mining Tasks – Text Mining
• Task Description:
• Text mining involves extracting useful information from unstructured text
data.
• Techniques:
• Natural Language Processing (NLP): Techniques for processing and analyzing
text.
• Topic Modeling (e.g., LDA - Latent Dirichlet Allocation): Identifies topics
present in a collection of documents.
• Sentiment Analysis: Determines the sentiment expressed in text (positive,
negative, neutral).
• Example: Analyzing customer reviews to determine overall sentiment
towards a product.
Knowledge Discovery in Databases
• Knowledge Discovery in Databases (KDD) is the process of identifying
valid, novel, potentially useful, and ultimately understandable
patterns in data.
• It encompasses several steps, from the initial data selection to the
final knowledge representation.
Knowledge Discovery in Databases
KDD Process
1. Data Selection
1. Identify and gather relevant data from various sources.
2. Data Preprocessing (or Data Cleaning and Integration)
1. Data Cleaning: Handle missing values, remove duplicates, and correct errors.
2. Data Integration: Combine data from multiple sources to create a unified dataset.
3. Data Transformation (or Data Preparation)
1. Convert data into suitable formats for mining, including normalization, aggregation, and feature selection.
4. Data Mining
1. Apply algorithms to extract patterns and models from the data.
5. Pattern Evaluation
1. Assess and validate the discovered patterns to ensure they are significant and useful.
6. Knowledge Representation
1. Present the mined knowledge in an understandable and actionable form, such as through visualization or
reports.
KDD Iterative Nature
• It's important to understand that KDD is not a strictly linear process.
• It is iterative and interactive, meaning steps might be revisited based
on insights gained during later stages.
• For example:
• Discovering that additional data cleaning is needed after some initial data
mining.
• Revisiting feature selection based on the patterns found.
Common Variations in KDD Approaches
• Combined Preprocessing Steps:
• Some approaches combine data cleaning, integration, and transformation
into a single preprocessing step.
• This is to emphasize the continuous preparation and refinement of data
before mining.
• Fewer or More Steps:
• Some methodologies break down steps further or combine them to
emphasize certain aspects.
• Five-Step Approach: Selection, preprocessing, transformation, data mining, evaluation.
• Six-Step Approach: Explicitly separates preprocessing into cleaning and integration,
followed by transformation.
Consensus on the KDD Process
• Despite the differences in terminology and step delineation, there is a
consensus on the essential activities:
• Preparing the data:
• Including selection, cleaning, integration, and transformation.
• Mining the data:
• Applying algorithms to extract patterns.
• Evaluating and interpreting the results:
• Ensuring the patterns are meaningful and useful.
• Presenting the findings:
• Making the knowledge understandable and actionable.
Data Mining Architecture
Components of data mining System
• The following are the main components of Data Mining System
Architecture
• Sources of Data
• Database or Data Warehouse Server
• Data Mining Engine
• Modules for Pattern Evaluation
• Graphical User Interface
• Knowledge Base
Data Sources
• Databases:
• Structured collections of data, typically organized in tables.
• Data Warehouses:
• Large, centralized repositories of integrated data from multiple sources,
optimized for querying and analysis.
• Data Lakes:
• Storage systems that hold vast amounts of raw data in its native format,
including structured, semi-structured, and unstructured data.
• External Data Sources:
• Data from external systems, such as web data, social media, and other third-
party sources.
Database or Data warehouse server
• Data Storage:
• Manages the storage and retrieval of data.
• Data Processing:
• Handles operations like data extraction, transformation, and loading (ETL
processes).
Data Mining Engine
• Core Data Mining Algorithms:
• Implements various algorithms for tasks such as classification, regression,
clustering, association rule mining, and anomaly detection.
• Pattern Evaluation Modules:
• Evaluates the discovered patterns to determine their significance and
relevance.
Knowledge Base
• Domain Knowledge:
• Contains domain-specific knowledge that can help guide the data mining
process, such as taxonomies, ontologies, and user preferences.
• Meta-knowledge:
• Stores metadata about the data, algorithms, and patterns discovered.
User interface
• Graphical User Interface (GUI):
• Allows users to interact with the data mining system, providing functionalities
like data selection, preprocessing options, algorithm selection, and
visualization of results.
• Query Interface:
• Enables users to input specific queries to guide the data mining process.
Pattern Evaluation Module
• Evaluation Metrics:
• Uses metrics like accuracy, precision, recall, F1-score, and others to assess the
quality of the discovered patterns.
• Validation Methods:
• Employs techniques such as cross-validation, holdout validation, and
bootstrapping to validate the patterns.
Data Preprocessing Component
• Data Cleaning: Handles missing values, removes duplicates, and
corrects errors.
• Data Integration: Combines data from multiple sources into a
coherent dataset.
• Data Transformation: Normalizes, aggregates, and transforms data
into the appropriate format for mining.
• Data Reduction: Reduces the volume of data through methods like
sampling, dimensionality reduction, and feature selection.
Data Mining Tools
• Software and Libraries:
• Provides tools and libraries for implementing data mining algorithms and
processes.
• Examples include Scikit-learn, Weka, RapidMiner, and KNIME.
Visualization Tools
• Data visualization is the graphical representation of information and data.
• It uses visual elements like charts, graphs, and maps, to provide an accessible
way to see and understand trends, outliers, and patterns in data.
• In the world of Big Data, data visualization tools and technologies are essential to
analyze massive amounts of information and make data-driven decisions.
• Visualization is an increasingly key tool to make sense of the trillions of rows of
data generated every day.
• It helps to tell stories by curating data into a form easier to understand,
highlighting the trends and outliers.
Deployment and Integration Modules
• Model Deployment:
• Manages the deployment of data mining models into production
environments.
• APIs and Connectors:
• Facilitates integration with other systems and applications, allowing the
mined knowledge to be used in decision-making processes.