Chapter One

Introduction to Machine
Learning

Birhanu A.
E-mail: [email protected] or [email protected]

03/27/2025 Machine Learning 1

 The Way Forward
Topics Areas covered
Introduction to Machine Definition of Machine Learning, Fundamentals of Machine Learning, Application of
Learning (ML) Machine Learning
Concept of Supervised Learning, Logistic Regression, Decision Trees, Naïve Bayesian
Supervised Learning Classification, the k-Nearest Neighbours Classifiers, Ensemble, Linear Discriminant
Algorithms Analysis, Support Vector Machine, Time-Series Forecasting, Sequential Pattern
Analysis
Evaluation in Machine Metrics, cross-validation, statistics, Addressing the multiple comparisons problem.
Learning
Unsupervised Learning Concept of unsupervised learning, k-Means Clustering, Hierarchical Clustering,
Algorithms Gaussian Mixture Model, Hidden Markov Model, Principal Component Analysis
Introduction to Reinforcement Learning, Markov Decision Process, Monte Carlo
Reinforcement Learning
Methods for Prediction & Control
Concept of deep learning, Regularization , convolutional neural networks, recurrent
Deep learning neural networks, utoencoders
Introduction to Expert Systems, the representation and manipulation of knowledge in
Expert System a computer, Expert System Architectures, Building Expert Systems

03/27/2025 Machine Learning 2

Reference
1 Ethem ALPAYDIN. Introduction to Machine Learning, Third Edition.
The MIT Press, August 2014
2 Mohssen Mohammed, Muhammad Badruddin Khan, Eihab Bashier
Mohammed Bashier Machine Learning: Algorithms and Applications,
CRC press, 2017

3 Mohssen Mohammed, Muhammad Badruddin Khan, Eihab Bashier

Mohammed Bashier Machine Learning: Algorithms and Applications,
CRC press, 2017
Manohar Swamynathan. Mastering Machine Learning with Python in
Six Steps, Appress,2017

4 Ian Goodfellow Yoshua Bengio Aaron Courville, Deep Learning, The

MIT press, 2016

03/27/2025 3
Reference
1. Ethem ALPAYDIN. Introduction to Machine Learning, Third Edition, 2014
2. Mohssen Mohammed, Muhammad Badruddin Khan, Eihab Bashier,
Mohammed Bashier Machine Learning: Algorithms and Applications,2017
3. Manohar Swamynathan. Mastering Machine Learning with Python in Six
Steps,2017
4. Ian Goodfellow Yoshua Bengio Aaron Courville, Deep Learning, 2016

03/27/2025 4
Tools and Dataset
 Tools:
R
Python
MatLab
 Dataset:
UCI Machine Learning Repository: http://archive.ics.uci.edu/ml/

03/27/2025 5
Chapter One: Introduction to Machine Learning
 Definition of Machine Learning(ML):
It is a subset of AI. It focus mainly on the designing of systems, thereby
allowing them to learn and make predictions based on some experiences
which is data in of machines.

It is the field of study that gives computers the ability to learn without being
explicitly programmed (Arthur Samuel, 1959).

A computer program is said to learn from experience E with respect to some

class of tasks T and performance measure P, if its performance at tasks in T, as
measured by P, improves with experience E (Tom Mitchell, 1999).
 Task T: classifying handwritten digits from images
 Performance measure P : percentage of digits classified correctly
 Training experience E: dataset of digits given classifications
03/27/2025 6
Chapter One: Introduction to Machine Learning
 AL vs ML vs DL

A technique which enables a

Artificial machine to mimic human
Intelligence(AI) behaviors
Machine
Learning(ML) A subset of AI technique which uses
statistical methods to enable machine
to improve with experiences
Deep Learning

A subset of ML which makes the

computation of multilayers neural
network feasible

03/27/2025 7
Chapter One…
 Definition of Machine Learning(ML)…

Basic Paradigm of ML:

• Observe set of examples: Training
data

• Infer something about process

that generated that data

• Use inference to make predictions

about previously unseen data:
test data

03/27/2025 8
Chapter One…
 How does Machine Learning works?

03/27/2025 9
Chapter One…
 Machine Learning types:

03/27/2025 10
Chapter One…
 Supervised Learning:
Supervised learning is the subcategory of machine learning that focuses on learning a
classification or regression model
 learning from labeled training data (i.e., inputs that also contain the desired outputs or targets;
basically, examples " of what we want to predict”).

Illustration of a binary classification problem (plus, minus) and two feature variable (x1 and x2).

03/27/2025 11
Chapter One…
 Supervised Learning Example:

03/27/2025 12
Chapter One…
 Supervised Learning …
Can be
 Classification
 Numeric Prediction/regression

 Classification
predicts categorical class labels (discrete or nominal)
classifies data (constructs a model) based on the training set and the values
(class labels) in a classifying attribute and uses it in classifying new data
Example: classify e-mail as Spam or Non-Spam
 Numeric Prediction
models continuous-valued functions, i.e., predicts unknown or missing values
Example: predict electric power consumption of a city in Kilo-watt

03/27/2025 13
Chapter One…
 Supervised Learning…
How do we predict housing prices
What do we start with?
 Training set (this is your data set)
 m = number of training examples
 x's = input variables / features
 y's = output variable "target" variables

 With our training set defined - how do we used it?

Take training set
Pass into a learning algorithm
Algorithm outputs a function i.e. y=f(x)
 This function takes an input (e.g. size of new house)
 Tries to output the estimated value of Y

03/27/2025 14
Chapter One…
 Supervised Learning…

 Given this data, a friend has a house 750 square feet - how much can they be expected to get?
 What approaches can we use to solve this?
 Straight line through data
• Maybe $150 000
 Second order polynomial
• Maybe $200 000
 Each of these approaches represent a way of doing supervised learning
03/27/2025 15
Chapter One…
 Supervised Learning Workflow :

03/27/2025 16
Chapter One…
 Unsupervised Learning:
 The class labels of training data is unknown
 Given a set of measurements, observations, etc. with the aim of establishing the
existence of classes or clusters in the data
 Example
x x x
 Given a set of points, with a notion of distance x x x x x x x
between points, group the points into some x xx x x x x
number of clusters, so that members of a cluster x x x x x x
are in some sense as close to each other as x x x xx x
possible.
 While data points in the same cluster are x x
similar, those in separate clusters are dissimilar x x x x
to one another x x x
x
03/27/2025 17
Chapter One…
 Unsupervised Example
The example below demonstrates the clustering of padlocks of same kind.
There are a total of 10 padlocks which various in color, size, shape, etc.

How many possible clusters of padlocks can be identified?

 There are three different kind of padlocks; which can be grouped into three different clusters.
 The padlocks of same kind are clustered into a group as shown below

03/27/2025 18
Chapter One…
 Unsupervised Learning Example- customer Segmentation
A company with a data of past customers
 the demographic information

 the past transactions with the company

The company may want to see the distribution of the profile of its customers, to see
what type of customers frequently occur.
Clustering allocates similar customer in the same group

Once such groups are found, the company may decide strategies, for example, services
and products, specific to different groups.
03/27/2025 19
Chapter One…
 Reinforcement learning :
Reinforcement is the process of learning from rewards while performing a series of
actions.
In reinforcement learning, we do not tell the learner or agent, for example, a (ro)bot,
which action to take but merely assign a reward to each action and/or the overall
outcome
Instead of having /correct/false" label for each step, the learner must discover or learn
a behavior that maximizes the reward for a series of actions

03/27/2025 20
Chapter One…
 Reinforcement learning …
the machine learning program should be able to assess the goodness of policies and
learn from past good action sequences to be able to generate a policy i.e. RL
 Example: Game playing
A good example is game playing where a single move by itself is not that important;
it is the sequence of right moves that is good.
A move is good if it is part of a good game playing policy

03/27/2025 21
Chapter One…
 Application of ML:
 Learning Associations
Basket analysis- In the case of retail-finding associations between products bought by
customers: If people who buy X typically also buy Y
going over our data, we calculate that P(chips/beer) = 0.7.
 we can define the rule: 70 percent of customers who buy beer also buy chips.
 Classification:
Credit scoring -the bank calculates the risk given the amount of credit and the
information about the customer.
 Customer Information:-income, savings, collaterals, profession, age, past financial history, and so
forth
 record of past loan- the loan was paid back or not
From this data of particular applications, the aim is to infer a general rule coding the
association between a customer’s attributes and his risk
03/27/2025 22
Chapter One…
 Application of ML…
The machine learning system fits a model to the past data to be able to calculate the
risk for a new application and then decides to accept or refuse it accordingly.
After training with the past data, a classification rule learned may be of the form
Discriminant rule: IF income > θ1 AND savings > θ2
THEN low-risk
ELSE high-risk

03/27/2025 23
Chapter One…
 Application of ML…
Pattern recognition- recognizing character codes from their images
 Handwritten—for example, to read zip codes on envelopes or amounts on checks.
 People have different handwriting styles; characters may be written small or large, slanted, with a
pen or pencil, and there are many possible images corresponding to the same character.
Face recognition
 The input is an image, the classes are people to be recognized, and the learning program should
learn to associate the face images to identities.
 Difficulty in lighting, glasses may hide the eyes and eyebrows, and a beard may hide the chin.
In medical diagnosis
 The inputs are the relevant information we have about the patient and the classes are the illnesses.
 The inputs contain the patient’s age, gender, past medical history, and current symptoms

Machine Learning
03/27/2025 24
Chapter One…
 Application of ML…
In speech recognition-
 the input is acoustic and the classes are words that can be uttered
 This time the association to be learned is from an acoustic signal to a word of some language
 Different people, because of differences in age, gender, or accent, pronounce the same word
differently, which makes this task rather difficult.
Biometrics is recognition
 Recognition or authentication of people using their physiological and/or behavioral characteristics
 Examples of physiological characteristics are images of the face, fingerprint, iris, and palm; examples
of behavioral characteristics are dynamics of signature, voice, gait, and key stroke
Outlier detection
 Which is finding outlier detection the instances that do not obey the rule and are exceptions
 For example – fraud

03/27/2025 25
Chapter One…
 Application of ML…
Learning a rule from data also allows knowledge extraction.

The rule is a simple model that explains the data, and looking at this model we have an
explanation about the process underlying the data.
For example, once we learn the discriminant separating low-risk and high-risk
customers, we have the knowledge of the properties of low-risk customers.
We can then use this information to target potential low-risk customers more
efficiently, for example, through advertising.

03/27/2025 26
Chapter One…
 Application of Machine learning-Regression
Predict the price of a used car
 Inputs are the car attributes—brand, year, engine capacity, mileage, and other information—that we
believe affect a car’s worth.
 The output is the price of the car. Such problems where the output is a number are regression
problems
A machine that roasts coffee
 The machine has many inputs that affect the quality: various settings of temperatures, times, coffee
bean type, and so forth.
 We measure the quality of the coffee, for example, as consumer satisfaction.
 To find the optimal setting, we fit a regression model linking these inputs to coffee quality and
choose new points to sample near the optimum of the current model to look for a better
configuration.

03/27/2025 27
Chapter One…
 Application of Machine learning- Unsupervised Learning
Document clustering
 the aim is to group similar documents
 For example, news reports can be subdivided as those related to politics, sports, fashion, arts, and so
on
 Documents are then grouped depending on the number of shared words.

03/27/2025 28
Chapter One…
 5 Steps for Approaching an ML:

1. Define the problem to be solved.

2. Collect data.
3. Choose an algorithm class.
4. Choose an optimization metric for learning the model.
5. Choose a metric for evaluating the model.

03/27/2025 29