Scribd
Upload
22 views

Week-14 Lecture 28

This document discusses different machine learning algorithms for regression and classification problems. It begins by defining learning, machine learning, and different types of regression problems like linear, multiple linear, polynomial and support vector regression. Then it explains classification problems and algorithms like logistic regression, k-nearest neighbors, decision trees, random forests and neural networks. Examples of each algorithm are provided along with code implementations and results for logistic regression.

Uploaded by

Syed Hamza Ibrar Shah
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
22 views

Week-14 Lecture 28

This document discusses different machine learning algorithms for regression and classification problems. It begins by defining learning, machine learning, and different types of regression problems like linear, multiple linear, polynomial and support vector regression. Then it explains classification problems and algorithms like logistic regression, k-nearest neighbors, decision trees, random forests and neural networks. Examples of each algorithm are provided along with code implementations and results for logistic regression.

Uploaded by

Syed Hamza Ibrar Shah
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 34

Artificial Intelligence

LEARNING: REGRESSION VS CLASSIFICATION

Week-14 Lecture-28
Agenda 2

❑ Concept of Learning

❑ Regression

► Multiple Linear Regression.

► Polynomial Regression.

► Support Vector Regression.

❑ Classification

► Logistic Regression.

► k-nearest neighbors.
Learning
4
Concept of Learning

► Self learning: Ability to recognize patterns, learn from data, and become more intelligent
over time (can be AI or programmatically based).
► Machine learning: It is an application of artificial intelligence (AI) that provides
systems the ability to automatically learn and improve from experience without being
explicitly programmed. Machine learning focuses on the development of computer
programs that can access data and use it to learn for themselves.
Regression
6
Recap: Linear Regression

Dependent Variable Independent Variable

�
�
�� Constant Coefficient
Dependent
Variable

�
Independent Variable
�
��
7
Example: Linear Regression

Salary Data
Example: Linear Regression (Ordinary Least 8
Squares)
9
Multiple features (variables)

Size (feet2) Price ($1000)

2104 460
1416 232
1534 315
852 178
… …
10
Multiple features (variables)

Size (feet2) Number of Number of Age of home Price ($1000)


bedrooms floors (years)

2104 5 1 45 460
1416 3 2 40 232
1534 3 2 30 315
852 2 1 36 178
… … … … …
11
Multiple features (variables)

Size (feet2) Number of Number of Age of home Price ($1000)


bedrooms floors (years)

2104 5 1 45 460
1416 3 2 40 232
1534 3 2 30 315
852 2 1 36 178
… … … … …
Notation:
= number of features
= input (features) of training example.
= value of feature in training example.
12
Hypothesis:

Previously:
(No More)

Now:
13

For convenience of notation, define .

Multivariate linear regression.


14
Housing prices prediction
15
Polynomial regression

Price
(y)

Size
(x)
16
Other Type of Regression

Support Vector Regression

Decision Tree Regression

Random Forest Regression

Artificial neural network regression

Deep learning based regression


17
Example: Linear Regression (Implementation)

Importing Libraries

Data Import

Train and Test Split


18
Example: Simple Linear Regression (Implementation)

Fitting a linear regressor


to the Salary Data

Plotting the Results


19
Example: Simple Linear Regression (Results)
Classification
21
Classification

❑ A classification problem is when the output variable is a category


“red” or “blue” or “disease” and “no disease”.
❑ A classification model attempts to draw some conclusion from
observed values.
❑ Given one or more inputs a classification model will try to predict
the value of one or more outcomes.

Which of the following is/are classification problem(s)?

▪ Predicting the gender of a person by his/her handwriting style


▪ Predicting house price based on area
▪ Predicting whether monsoon will be normal next year
▪ Predict the number of copies a music album will be sold next
Image source: Kaggle
month
Logistic Regression
23
Logistic Regression: Overview

❑ Defined: A model for predicting one variable from


other variable(s).

❑ Variables: IV(s) is continuous/categorical,


DV is dichotomous (contrast b/w things)

❑ Relationship: Prediction of group membership

❑ Example: Can we predict students passage from GPA, etc.

❑ Assumptions: Multicollinearity (not linearity or normality)


24
Comparison to Linear Regression

❑ Since dichotomous outcome, can’t use linear


regression because not linear

❑ Since dichotomous outcome, we are now talking


about “probabilities” (of 0 or 1)

❑ So logistic is about predicting the probability of


the outcome occurring.
25
Logistic Regression basics

• Logistic is based upon “odds ratio”


• which is the probability of an event divided by probability of non-event.
• For example, if Exp(b) =2, then a one unit change would make the event
twice as likely (.67/.33) to occur.
26
Logistic Regression basics

❑ Single predictor

❑ Multiple predictor

❑ Notice the linear regression equation


❑ e is the base of the natural logarithm (about 2.718)
k-nearest neighbor: Intuition
27
❑ K nearest neighbors is a simple algorithm that stores all available
cases and classifies new cases based on a similarity measure (e.g.,
distance functions).

❑ KNN has been used in statistical estimation and pattern recognition


already in the beginning of 1970’s as a non-parametric technique.

Hasan, M.J.; Kim, J.-M. Fault Detection of a Spherical Tank Using a Genetic Algorithm-Based Hybrid Feature Pool and k-Nearest Neighbor Algorithm. Energies 2019, 12, 991.
Tree Based
Methods 28

Decision Tree Complexity

* Slides from Seo Hui(LG Electronics), “Gradient Boosting


Model”
29
Implementation: Logistic Regression
30
Implementation: Logistic Regression Cont.
31
Implementation: Logistic Regression Cont.
32
Logistic Regression: Training and Testing Results
33
Another Classification Algorithms

Support vector machine


Random Forest
Naïve Bayes

Neural Network (MLP)


…
34
References

► Y. Bengio, I. Goodfellow, and A. Courville, Deep learning, vol.1. Citeseer,


2017.ai.berkeley.edu

► SuperDataScience

You might also like