Scribd
Upload
51 views

Data Mining Lecture 1 - Summary

This document provides an overview of key concepts in data mining and analytics including: - Passive and active approaches to data mining including clustering, classification, and model construction. - Common data types and parameters such as ordered numeric data, time series data, and missing/hidden values. - Statistical concepts like probability distributions, mean, correlation, and outliers. - Similarity and distance measures to quantify relationships between data points including Euclidean, Manhattan, and Mahalanobis distances. - Dimensionality reduction techniques like principal component analysis, factor analysis, and multidimensional scaling.

Uploaded by

Yaksh Shah
Copyright
© © All Rights Reserved
Available Formats
Download as DOC, PDF, TXT or read online on Scribd
51 views

Data Mining Lecture 1 - Summary

This document provides an overview of key concepts in data mining and analytics including: - Passive and active approaches to data mining including clustering, classification, and model construction. - Common data types and parameters such as ordered numeric data, time series data, and missing/hidden values. - Statistical concepts like probability distributions, mean, correlation, and outliers. - Similarity and distance measures to quantify relationships between data points including Euclidean, Manhattan, and Mahalanobis distances. - Dimensionality reduction techniques like principal component analysis, factor analysis, and multidimensional scaling.

Uploaded by

Yaksh Shah
Copyright
© © All Rights Reserved
Available Formats
Download as DOC, PDF, TXT or read online on Scribd
You are on page 1/ 3

Data Mining Lecture 1 – summary

0. Introduction
From data to information
Data mining relates to the area of application
Passive: Observing relations
Active: Finding the underlying model
Passive: Clustering, Classification
Active: Passive + Model construction + parameter estimation

1. Example: PCB production


 Normalization and standardization
 Hidden parameters and missing values
 Input, output, and state parameters

2. Data as Sets of Observations


Parameters and Observations/Measurements
Ordered Data Sets and alphanumeric sets and lexicographic sets
Data is a set of ordered numeric data of n observations of d parameters
Dynamic Sets and Time series
sampling
Data Quality

3. Statistics
Uncertainty and Probability distributions: normal, uniform, Poisson, Bernouli, something
Normal distribution
Kolmogorov-Smirnov test
Multivariate sets
Mean
Mean-centered data
Correlation and correlation coefficient
Covariance matrix
Estimation
Outliers and the linear correlation coefficient
Transforming

4. Similarity and Distance

Measures for Similarity and Dissimilarity


How similar or dissimilar two data points are
sim(p,q) in [0,1]
sim(p,q) = sim(q,p)
sim(p,p) = 1
dissim also in [0,1] : dissim = 1 – sim

Distance measures and Metrics


Distance measured according to certain rule in the data space
dist(p,q) >= 0
if dist(p,q) = 0 then p = q
triangle-inequality: dist(p,q) <= dist(p,a) + dist(a,q)
notice that distance relates to the dissimilarity
a vector space with a distance definition is a metric space

Examples of Distances
Euclidean : distE(p,q)2 = (p1 – q1)2 + (p2 – q2)2 + … + (pn – qn)2
Manhattan: distM(p,q) = |p1 – q1| + |p2 – q2| + … + |pn – qn|
Max-norm: distmax(p,q) = max {|p1 – q1|, |p2 – q2|, …, |pn – qn| }
Notice the graphs of dist@(p,0) in IR2 for @ = Euclidean, Manhattan, max

Generalized p-norm
1/ d
 n d
p d    pi 
 i 1 
notice that for: d =2: ||p - q||d = Euclidean distance
for: d =1: ||p - q||d = Manhattan distance
for: d = ∞: ||p - q||d = Max-norm distance
normally d >= 1

Riemannian Metric
Let g be a definite non-negative matrix on IRd (i.e. all eigen values >= 0)
then g induces a Riemannian metric on the space:
2
p g
 p T gp

Involving peculiarities of the distribution


Let ρ be a probability distribution on a data space. Let m be the mean and C be the
covariance matrix associated to ρ :
C   (x  m)  (x)(x  m)
T
dx
then the Mahalanobis distance is defined as:
dist Mahal (x)  (x  m)C 1 ( x  m ) T
It give a measure for the distance of a data point x to the center of the distribution.
Notice that in the mean-centered space the Mahalanobis-distance is a Riemannian
norm with metric g = C-1.

Similarity and Distance


Notice that we can define the similarity between two data points p and q as some
function f: sim(p,q) = f(dist(p,q)). Examples are:
f(d) = 1/(1 + d/L)
f(d) = exp(-d/L)
f(d) = - d/L if d < L and f(d) = 0 if d >= L
where d = dist(p,q) and L is some characteristic length for the problem.

Correlation

5. Visualizing and Exploring Data


single variable-display:
 Histogram
 Cumulative distribution
* Kernel estimate
Two-variable representation
 Scatter plot
 Contour plot
Multiple-variable representation
 Scatter plot matrix
 Trelis plot
 Chernoff face

6. Dimension Reduction
Principal Components Analysis (PCA)
Find the principal directions in the data, and use them to reduce the number of dimensions of
the set by representing the data in linear combinations of the principal components.
Works best for multivariate data. Finds the m < d eigen-vectors of the covariance matrix with
the largest eigen-values. These eigen-vectors are the principal components. Decompose the
data in these principal components and thus obtain as more concise data set.
Caution1: Depends on the normalization of the data! Ideal for data with equal units.
Caution2: works only in linear relations between the parameters
Caution3: valuable information can be lost in pca

Factor Analysis
Represent data with fewer variables
Not invariant for transformations: multiple equivalent solutions
Widely used esp. in alpha-world and medicine

Multidimensional scaling
Equivalent to PCA, and also in case there are non-linear relations between the parameters.
Input: similarity- or distance-map between the data-points. Output: a 2D- (or even higher D)
map of the data points.

Kohonen SO feature map


(Will be addressed later) Input: distance or similarity-map, Output: a 2D- (or even higher D)
map of the data points.

You might also like