Introduction to Machine
Learning:
Dr. Hikmat Ullah Khan
Assistant Professor
Department of Computer Science
COMSATS Institute of Information Technology,
Wah Cantt, Pakistan
Email:
[email protected]September 16, 2018 Dr. Hikmat Ullah Khan 1
September 16, 2018 Dr. Hikmat Ullah Khan 2
September 16, 2018 Dr. Hikmat Ullah Khan 3
September 16, 2018 Dr. Hikmat Ullah Khan 4
September 16, 2018 Dr. Hikmat Ullah Khan 5
Sources
Book
Introduction to Machine Learning, Second Edition
By Ethem Alpaydin, MIT Press 2004
Reference Books:
Machine Learning, Tom, M., 1st Edition (1997) McGraw
Pattern Recognition, Duda, R.O., Hart, P.E., &Stork, D.
Pattern Recognition and Machine Learning, Bishop, C.,(
Bioinformatics: A Machine Learning Approach, Baldi,
Edition (2001), MIT Press.
Introduction to Semi-Supervised Learning, MIT Press
Introduction to Data Mining by Jawei Han
September 16, 2018 Dr. Hikmat Ullah Khan 6
Evolution of Sciences
Before 1600, empirical science
1600-1950s, theoretical science
Each discipline has grown a theoretical component. Theoretical models
often motivate experiments and generalize our understanding.
1950s-1990s, computational science
Over the last 50 years, most disciplines have grown a third, computational
branch
Computational Science traditionally meant simulation.
1990-now, data science
The ability to store and manage a large volume of data online, Big Data
The Internet makes all these archives universally accessible , IoT
Human can not process huge volume, Machine based tasks - a major new
challenge!
September 16, 2018 Dr. Hikmat Ullah Khan 7
Machine Learning Function: (1)
Generalization
Before ML application, data Preprocessing
Data cleaning,
Data transformation/Normalization
Data integration,
Multidimensional data issues
September 16, 2018 Dr. Hikmat Ullah Khan 8
Machine learning
Association
Supervised Learning
Classification
Regression
Unsupervised Learning
Reinforcement Learning
9
Machine Learning (3)
Supervised Learning
Also known as Classification and label prediction
Construct models based on some training examples
Describe and distinguish classes or concepts for future
prediction
Predict some unknown class labels
Typical methods
Decision trees, naïve Bayesian classification, K-Nearest
Neighbor, support vector machines, neural networks, etc …
Typical applications:
Sentiment Classification,
Spam Classification,
Disease Prediction,
Match result prediction
September 16, 2018 Dr. Hikmat Ullah Khan 10
Classification: Applications
Aka Pattern recognition
Face recognition: Pose, lighting, occlusion
(glasses, beard), make-up, hair style
Character recognition: Different handwriting
styles.
Speech recognition: Temporal dependency.
Use of a dictionary or the syntax of the language.
Sensor fusion: Combine multiple modalities; eg, visual
(lip image) and acoustic for speech
Medical diagnosis: From symptoms to illnesses
...
11
Face Recognition
Training examples of a person
Test images
AT&T Laboratories, Cambridge UK
http://www.uk.research.att.com/facedatabase.html
12
Machine Learning : (4)
Unsupervised Learning
Unsupervised learning
(i.e., Class label is unknown)
Group data to form new categories (i.e., clusters)
Principle:
Maximizing intra-class similarity & minimizing interclass similarity
Many methods and applications
K-means
K-mediod
Applications
Groups or similar people finding
Community detection
September 16, 2018 Dr. Hikmat Ullah Khan 13
Machine Learning (2)
Rule based Learning
Frequent patterns (or frequent itemsets)
What items are frequently purchased together in your
Walmart/SaveMart?
Association, correlation
A typical association rule
Milk Bread [0.5%, 75%] (support, confidence)
Algorithms
Apriori
Our aim is to learn Rules from Association rule
learning
September 16, 2018 Dr. Hikmat Ullah Khan 14
Learning Associations
Basket analysis:
P (Y | X ) probability that somebody who buys X
also buys Y where X and Y are products/services.
Example: P ( chips | juice ) = 0.7
15
Regression
Prediction of
numeric value
Based on Relationship
between two
variables, Prediction y = wx+w0
of one Value of a
variable
Example: Price of a
used car
x : car attributes
y : price
y = g (x | θ )
Lecture Notes for E Alpaydın 2004 Introduction to Machine Learning © The MIT Press 16
(V1.1)
Reinforcement Learning
Learning a policy:
A sequence of outputs
No supervised output but delayed
reward
Game playing
Robot in a maze
17
Semi-Supervised Learning
Semi-Supervised
What is Class Label?
Who provides Label?
What are Issue in Data Labeling.?
Semi-Supervised Learning deals with
LABELLED DATA as well UN-LEBELELD
DATA
September 16, 2018 Dr. Hikmat Ullah Khan 18
Machine Learning : (5)
Outlier Analysis
Also Called ANOMALY DETECION
What is an Outlier:
A data object that does not comply with the general
behavior of the data
Applications:
Finding mish-behavior of Users
Fake Community detection
Fraud detection,
rare events analysis
September 16, 2018 Dr. Hikmat Ullah Khan 19
Resources: Datasets
UCI Repository:
http://www.ics.uci.edu/~mlearn/MLRepository.html
UCI KDD Archive:
http://kdd.ics.uci.edu/summary.data.application.html
Statlib: http://lib.stat.cmu.edu/
Delve: http://www.cs.utoronto.ca/~delve/
20
