Scribd
Upload
13 views34 pages

Presentation 6

Uploaded by

mobadum666
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
13 views34 pages

Presentation 6

Uploaded by

mobadum666
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
You are on page 1/ 34

Supervised Learning

What is Machine Learning?


● Supervised Learning
○ Classification
○ Regression
What is Machine Learning?
● Supervised Learning
○ Classification
○ Regression
● Unsupervised Learning
What is Machine Learning?
● Supervised Learning
○ Classification
○ Regression
● Unsupervised Learning
● Self-supervised Learning (not in this course)
What is Machine Learning?
● Supervised Learning
○ Classification
○ Regression
● Unsupervised Learning
● Self-supervised Learning (not in this course)
● Reinforcement Learning (not in this course)
Classification example
Question: Based on their properties, is the car type ‘family car’?

Car properties : = (price, engine power)

Hypothesis: if a car is a family car?


Family Car: Training Set
Family Car: True Class
Family Car: Hypothesis class H
Consistent Hypothesis
Consistent Hypothesis
What did we learn from the Family Cars example?
● We must choose some hypothesis to be able to predict anything (unless we
observe all possible data values).
What did we learn from the Family Cars example?
● We must choose some hypothesis to be able to predict anything (unless we
observe all possible data values).
● This causes inductive bias (the choice of hypothesis space affects your
results).
What did we learn from the Family Cars example?
● We must choose some hypothesis to be able to predict anything (unless we
observe all possible data values).
● This causes inductive bias (the choice of hypothesis space affects your
results).
● All consistent hypothesis can be found between the most general and most
specific hypothesis.
What did we learn from the Family Cars example?
● We must choose some hypothesis to be able to predict anything (unless we
observe all possible data values).
● This causes inductive bias (the choice of hypothesis space affects your
results).
● All consistent hypothesis can be found between the most general and most
specific hypothesis.
● In practical applications, there may be no consistent hypothesis due to the
hypothesis space that is too simple (underfitting) or a noisy distribution of data.
Noise and Model Complexity
● Noise is unwanted anomaly of data.
● Because of the noise, we may never reach zero
error.
● Why to use simpler model:
○ Simpler to use
○ Easier to train
○ Easier to explain
○ Generalizes better
Noise and Model Complexity
● Noise may be caused by:
○ Errors in measurements of input attributes or class labels.
○ Unknown or ignored (hidden or latent) attributes.
(Example: plane ticket price may increase when there is
an event in the region.)
○ Model is wrong or inaccurate (figure)
Regression
● Classification is the prediction of a class label, given attributes.
● Regression is the prediction of a real number, given attributes (usually with
noise).
Regression
● Classification is the prediction of a class label, given attributes.
● Regression is the prediction of a real number, given attributes (usually with
noise).
● The training set is given by , where .
Regression
● Classification is the prediction of a class label, given attributes.
● Regression is the prediction of a real number, given attributes (usually with
noise).
● The training set is given by , where .
● Each hypothesis is a function . We would like to find for
all items in the training set.
Regression
● Classification is the prediction of a class label, given attributes.
● Regression is the prediction of a real number, given attributes (usually with
noise).
● The training set is given by , where .
● Each hypothesis is a function . We would like to find for
all items in the training set.
● Usually, we want to minimize a quadratic error function:
Regression
● The simplest case is linear regressor:
Regression
● The simplest case is linear regressor:
● Optimization task: find w0 and w1 such that the error

is minimized.
Linear Regression: Toy data
Polynomial Regressors
Polynomial Regressors
Polynomial Regressors
Polynomial Regressors
Polynomial Regressors
Polynomial Regressors
Polynomial Regressors
Polynomial Regressors
● Etrain is the error in the training data. It decreases as model complexity
increases.
● Etest is the error on the remaining 93 data items, or “test set”. It has minimum at
k = 3.
Polynomial Regressors
Big Data!

is d-by-d.
requires N x d2 multiplications and additions.
Inverting is O(d3) (Gauss-Jordan)
Afterwards, requires N x d2 + N x d multiplications.

You might also like