INTRODUCTIO

N TO
MACHINE
LEARNING
1
WHAT IS LEARNING?
 Herbert Simon: “Learning is any process by which a
system improves performance from experience.”
 What is the task?
 Classification
 Categorization/clustering
 Problem solving / planning / control
 Prediction
 others
 WHAT IS MACHINE
LEARNING?
“Learning is any process by which a system improves
performance from experience.”
- Herbert Simon

Definition by Tom Mitchell (1998):

Machine Learning is the study of algorithms that
• improve their performance P
• at some task T
• with experience E.
A well-defined learning task is given by <P, T, E>.
3
TRADITIONAL PROGRAMMING

Data
Computer Output

Program
Machine Learning

Data
Computer Progra
m
Output 4
 WHEN DO WE USE MACHINE
LEARNING?
ML is used when:
• Human expertise does not exist (navigating on Mars)
• Humans can’t explain their expertise (speech recognition)
• Models must be customized (personalized medicine)

Learning isn’t always useful:

• There is no need to “learn” to calculate payroll
5
A CLASSIC EXAMPLE OF A TASK THAT
REQUIRES MACHINE LEARNING: IT IS
VERY HARD TO SAY WHAT MAKES A 2

6
SOME MORE EXAMPLES OF TASKS
THAT ARE BEST SOLVED BY
USING A LEARNING ALGORITHM
• Recognizing patterns:
– Facial identities or facial expressions
– Handwritten or spoken words
– Medical images
• Generating patterns:
– Generating images or motion sequences
• Recognizing anomalies:
– Unusual credit card transactions
– Unusual patterns of sensor readings in a nuclear power
plant
• Prediction:
– Future stock prices or currency exchange rates
7
 SAMPLE APPLICATIONS
• Web search
• Computational biology
• Finance
• E-commerce
• Space exploration
• Robotics
• Information extraction
• Social networks
• Debugging software
• [Your favorite area]

8
Slide credit: Pedro Domingos
Some web-based examples of machine learning

1. The web contains a lot of data. Tasks with very big

datasets often use machine learning
• especially if the data is noisy or non-stationary.

2. Spam filtering, fraud detection:

• The enemy adapts so we must adapt too.

3. Recommendation systems:
• Lots of noisy data. Million dollar prize!

4. Information retrieval:
• Find documents or images with similar content.

5. Data Visualization:
• Display a huge database in a revealing way
4. The amount of knowledge available about
certain tasks might be too large for explicit
encoding by humans (e.g., medical
diagnostic).
5. Environments change over time.
6. New knowledge about tasks is constantly
being discovered by humans. It may be
difficult to continuously re-design systems “by
hand”.
AREAS OF INFLUENCE FOR MACHINE
LEARNING
1. Statistics: How best to use samples drawn from unknown
probability distributions to help decide from which distribution
some new sample is drawn?
2. Brain Models: Non-linear elements with weighted inputs
(Artificial Neural Networks) have been suggested as simple
models of biological neurons.
3. Adaptive Control Theory: How to deal with controlling a
process having unknown parameters that must be estimated
during operation?
4. Psychology: How to model human performance on various
learning tasks?
5. Artificial Intelligence: How to write algorithms to
acquire the knowledge humans are able to acquire,
at least, as well as humans?
6.Evolutionary Models: How to model certain
aspects of biological evolution to improve the
performance of computer programs?
 Designing a Learning System
• In designing a learning system, we have to deal
with (at least) the following issues:
1. Training experience
2. Target function
3. Learned function
4. Learning algorithm

Training Experience
Issues concerning the training experience:
1. Direct or indirect evidence (supervised or unsupervised).
2. Controlled or uncontrolled sequence of training examples.
3. Representatively of training data in relation to test data.
 Target Function and Learned Function
• The problem of improving performance can often be
reduced to the problem of learning some particular target
function.
• In many cases we can only hope to acquire some
approximation to the ideal target function.
Learning Algorithm
In order to learn the (approximated) target function we require:
1. A set of training examples (input arguments)
2. A rule for estimating the value corresponding to each
training example (if this is not directly available)
3. An algorithm for choosing the function that best fits the
training data
TYPES OF LEARNING
 Induction vs deduction
 Rote learning (memorization)
 Advice or instructional learning
 Learning by example or practice
 Most popular; many applications

 Learning by analogy; transfer learning

 Discovery learning
 Others?
LEVELS OF LEARNING
TRAINING
Many learning methods involve training
 Training is the acquisition of knowledge, skills,
and competencies as a result of the teaching of
vocational or practical skills and knowledge that
relate to specific useful competencies
(wikipedia).
 Training requires scenarios or examples (data)
 TYPES OF TRAINING
EXPERIENCE
 Direct or indirect
 With a teacher or without a teacher
 An eternal problem:
 Make the training experience representative of the performance
goal
TYPES OF TRAINING
 Supervised learning: uses a series of labelled examples with
direct feedback
 Reinforcement learning: indirect feedback, after many
examples
 Unsupervised/clustering learning: no feedback
 Semisupervised
TYPES OF TESTING
 Evaluate performance by testing on data NOT used for
testing (both should be randomly sampled)
 Cross validation methods for small data sets
 The more (relevant) data the better.
TESTING
 How well the learned system work?
 Generalization
 Performance on unseen or unknown scenarios or data
 Brittle vs robust performance
WHICH OF THESE THINGS IS
NOT LIKE THE OTHERS?
WHICH OF THESE THINGS IS
LIKE THE OTHERS? AND
HOW?
USUAL ML STAGES
 Hypothesis, data
 Training or learning
 Testing or generalization
WHY IS MACHINE LEARNING
NECESSARY?
 learning is a hallmark of intelligence;
many would argue that a system that
cannot learn is not intelligent.
 without learning, everything is new; a
system that cannot learn is not efficient
because it rederives each solution and
repeatedly makes the same mistakes.
Why is learning possible?
Because there are regularities in the
world.
DEFINING THE LEARNING TASK
Improve on task, T, with respect to
performance metric, P, based on experience, E.
T: Playing checkers
P: Percentage of games won against an
opponent arbitrary
E: Playing practice games against itself
T: Recognizing hand-written words
P: Percentage of words correctly classified
E: Database of human-labeled images of handwritten
words
T: Driving on four-lane highways using vision sensors
P: Average distance traveled before a human-judged error
E: A sequence
recorded while of images and steering commands
observing a human driver.
T: Categorize email messages as spam or legitimate.
P: Percentage of email messages correctly classified.
E: Database of emails, some with human-given labels
DESIGNING A LEARNING
SYSTEM
Choose the training experience
 Choose exactly what is too be learned, i.e.
the target function.
 Choose how to represent the target function.
 Choose a learning algorithm to infer the
target function from the experience.

Learner
Environment/
Experience Knowledge
Performanc
e
Element
TRAINING EXPERIENCE
 Direct experience: Given sample input and
output pairs for a useful target function.
 Checker boards labeled with the correct move, e.g.
extracted from record of expert play
 Indirect experience: Given feedback which is
not direct I/O pairs for a useful target function.
 Potentially arbitrary sequences of game moves and
their final game results.
 Credit/Blame Assignment Problem: How to
assign credit blame to individual moves given
only indirect feedback?
TRAINING VS. TEST
DISTRIBUTION
 Generally assume that the training and test examples are
independently drawn from the same overall distribution
of data.
 IID: Independently and identically distributed

 If examples are not independent, requires collective

classification.
 If test distribution is different, requires transfer
learning.
CHOOSING A TARGET
FUNCTION
What function is to be learned and how will it
be used by the performance system?
 For checkers, assume we are given a function
for generating the legal moves for a given
board position and want to decide the best
move.
 Could learn a function:
ChooseMove(board, legal-moves) → best-move
 Or could learn an evaluation function, V(board) →
R, that gives each board position a score for how
favorable it is. V can be used to pick a move by
applying each legal move, scoring the resulting board
position, and choosing the move that results in the
highest scoring board position.
HISTORY OF MACHINE
LEARNING
 1950s
 Samuel’s checker player
 Selfridge’s Pandemonium

 1960s:
 Neural networks: Perceptron
 Pattern recognition
 Learning in the limit theory
 Minsky and Papert prove limitations of Perceptron

 1970s:
 Symbolic concept induction
 Winston’s arch learner
 Expert systems and the knowledge acquisition bottleneck
 Quinlan’s ID3
 Michalski’s AQ and soybean diagnosis
 Scientific discovery with BACON
 Mathematical discovery with AM
HISTORY OF MACHINE
LEARNING
1980s:
 (CONT.)
Advanced decision tree and rule learning
 Explanation-based Learning (EBL)
 Learning and planning and problem solving
 Utility problem
 Analogy
 Cognitive architectures
 Resurgence of neural networks (connectionism,
backpropagation)
 Valiant’s PAC Learning Theory
 Focus on experimental methodology

 1990s
 Data mining
 Adaptive software agents and web applications
 Text learning
 Reinforcement learning (RL)
 Inductive Logic Programming (ILP)
 Ensembles: Bagging, Boosting, and Stacking
 Bayes Net learning
HISTORY OF MACHINE
LEARNING
 2000s
(CONT.)
 Support vector machines
 Kernel methods
 Graphical models
 Statistical relational learning
 Transfer learning
 Sequence labeling
 Collective classification and structured outputs
 Computer Systems Applications
 Compilers
 Debugging
 Graphics
 Security (intrusion, virus, and worm detection)
E mail management
 Personalized assistants that learn
 Learning in robotics and vision
ROBOTICS AND ML
 Areas that robots are used:
 Industrial robots
 Military, government and space robots
 Service robots for home, healthcare, laboratory

 Why are robots used?

 Dangerous tasks or in hazardous environments
 Repetitive tasks
 High precision tasks or those requiring high quality
 Labor savings

 Control technologies:
 Autonomous (self-controlled), tele-operated (remote
control)
 INDUSTRIAL ROBOTS
 Uses for robots in
manufacturing:
 Welding
 Painting
 Cutting
 Dispensing
 Assembly
 Polishing/Finishing
 Material Handling
 Packaging, Palletizing
 Machine loading
 INDUSTRIAL ROBOTS
 Uses for robots in Industry/Manufacturing
 Automotive:
 Video - Welding and handling of fuel tanks from TV show “How
It’s Made” on Discovery Channel. This is a system I worked on in
2003.
 Packaging:
 Video - Robots in food manufacturing.
INDUSTRIAL ROBOTS -
AUTOMOTIVE
 MILITARY/GOVERNMENT
ROBOTS
 iRobot PackBot

Remotec Andros
 MILITARY/GOVERNMENT ROBOTS

Soldiers in Afghanistan being trained how to defuse a

landmine using a PackBot.
MILITARY ROBOTS
 Aerial drones (UAV) Military suit
 SPACE ROBOTS
 Mars Rovers – Spirit and Opportunity
 Autonomous navigation features with human remote control and
oversight
SERVICE ROBOTS
 Many uses…
 Cleaning & Housekeeping
 Humanitarian Demining
 Rehabilitation
 Inspection
 Agriculture & Harvesting
 Lawn Mowers
 Surveillance
 Mining Applications
 Construction
 Automatic Refilling
 Fire Fighters
 Search & Rescue
iRobot Roomba vacuum
cleaner robot
 MEDICAL/HEALTHCARE
APPLICATIONS
DaVinci surgical robot by Intuitive Surgical.
St. Elizabeth Hospital is one of the local hospitals using this robot. You
Japanese health care
can see this robot in person during an open house (website). assistant suit (HAL - Hybrid
Assistive Limb)

Also… Mind-
controlled
wheelchair using
LABORATORY APPLICATIONS
Drug discovery
Test tube sorting
 ALVINN

Drives 70 mph on a public

highway
Predecessor of the Google car
Camera
image

30 outputs
for steering
30x32 weights
4 hidden
into one out of
units
four hidden
30x32 pixels unit
as inputs
In other words the Types of Machine Learning may
be as follows:
• Supervised Learning
– Classification(pattern recognition)
– Regression
• Unsupervised Learning
• Reinforcement Learning
“Machine Learning: Field of study that gives
computers the ability to learn without being
explicitly programmed.” -Arthur Samuel (1959)

SAMUEL’S CHECKERS-PLAYER

9
STATE OF THE ART
APPLICATIONS OF
MACHINE LEARNING

11
 AUTONOMOUS
CARS

• Nevada made it legal for

autonomous cars to drive on
roads in June 2011
• As of 2013, four states (Nevada,
Florida, California, and
Michigan) have legalized
autonomous cars
Penn’s Autonomous
12
Car 
AUTONOMOUS CAR
SENSORS

13
 AUTONOMOUS CAR
TECHNOLOGY
Path

Planning

Laser Terrain Mapping

Learning from Human Drivers

Adaptive Vision

Sebastian

Stanle
y

Images and movies taken from Sebastian Thrun’s multimedia w1e4bsite.

DEEP LEARNING IN THE
HEADLINES

15
 DEEP BELIEF NET ON FACE
IMAGES
object
models

object
parts
(combinati
on of
edges)

edge
s

pixel
Based on materials s 16
by Andrew Ng
 LEARNING OF
OBJECT PARTS

17
Slide credit: Andrew Ng
 TRAINING ON MULTIPLE
OBJECTS

 Trained on 4 classes (cars,

faces, motorbikes,
airplanes).
 Second layer: Shared-
features and object-
specific features.
 Third layer: More
specific features.

18
Slide credit: Andrew Ng
 SCENE LABELING VIA DEEP
LEARNING

[Farabet et al. ICML 2012, PAMI 2013] 19

 INFERENCE FROM DEEP
LEARNED MODELS
Generating posterior samples from faces by “filling in” experiments
(cf. Lee and Mumford, 2003). Combine bottom-up and top-down inference.

Input images

Samples from
feedforward
Inference
(control)

Samples from
Full posterior
inference

20
Slide credit: Andrew Ng
 MACHINE LEARNING
IN AUTOMATIC SPEECH
RECOGNITION
A Typical Speech Recognition System

ML used to predict of phone states from the sound spectrogram

Deep learning has state-of-the-art results

# Hidden Layers 1 2 4 8 10 12

Word Error Rate % 16.0 12.8 11.4 10.9 11.0 11.1

Baseline GMM performance = 15.4%

[Zeiler et al. “On rectified linear units for speech
recognition” ICASSP 2013]
2
1
 IMPACT OF DEEP LEARNING IN SPEECH
TECHNOLOGY

22
Slide credit: Li Deng, MS Research
 ML IN PRACTICE

• Understand domain, prior knowledge, and goals

• Data integration, selection, cleaning, pre-processing, etc.
Loop • Learn models
• Interpret results
• Consolidate and deploy discovered knowledge

48
Based on a slide by Pedro Domingos
ASSIGNMENT
QUESTIONS
 How does ML affect information
science?
 Natural vs artificial learning – which is
better?
 Is ML needed in all problems?
 What are the future directions of ML?