UNIT 1: Introduction to

Machine Learning
BTCS 618‐18
Dr. Vandana Mohindru
Topics to be discussed
• Why Machine Learning?
• What is Machine Learning
• How does Machine Learning Works
• Well‐Posed learning problems
• Applications of Machine Learning
• Designing a learning system
• Issues in machine learning
• Types of machine learning:
• Learning associations,
• Supervised learning,
• Unsupervised learning and
• Reinforcement learning.
Why Machine Learning?
BIG DATA
• Widespread use of personal computers and wireless communication
leads to “big data”
• We are both producers and consumers of data
• Data is not random, it has structure, e.g., customer behavior
• We need “big theory” to extract that structure from data for
(a) Understanding the process
(b) Making predictions for the future
Why Machine Learning?
• Recent progress in algorithms and theory
• Growing flood of online data
• Computational power is available
• Budding industry
Three niches for machine learning:
• Data mining : using historical data to improve decisions
medical records  medical knowledge
• Software applications we can't program by hand
autonomous driving
speech recognition
• Self customizing programs
Newsreader that learns user interests
What is Machine Learning
• In the real world, we are
surrounded by humans who can
learn everything from their
experiences with their learning
capability, and we have
computers or machines which
work on our instructions.
• But can a machine also learn
from experiences or past data
like a human does? So here
comes the role of Machine
Learning.
What is Machine Learning
• Machine Learning is said as a subset of artificial intelligence that is
mainly concerned with the development of algorithms which allow a
computer to learn from the data and past experiences on their own.
The term machine learning was first introduced by Arthur Samuel in
1959. We can define it in a summarized way as:

• Machine learning enables a machine to automatically learn from

data, improve performance from experiences, and predict things
without being explicitly programmed.
What is Machine Learning
• With the help of sample historical data, which is known as training
data, machine learning algorithms build a mathematical model that
helps in making predictions or decisions without being explicitly
programmed. Machine learning brings computer science and
statistics together for creating predictive models. Machine learning
constructs or uses the algorithms that learn from historical data. The
more we will provide the information, the higher will be the
performance.
• A machine has the ability to learn if it can improve its performance
by gaining more data.
How does Machine Learning work
• A Machine Learning system learns from historical data, builds the prediction
models, and whenever it receives new data, predicts the output for it. The
accuracy of predicted output depends upon the amount of data, as the huge
amount of data helps to build a better model which predicts the output more
accurately.
• Suppose we have a complex problem, where we need to perform some predictions,
so instead of writing a code for it, we just need to feed the data to generic
algorithms, and with the help of these algorithms, machine builds the logic as per
the data and predict the output. Machine learning has changed our way of thinking
about the problem. The below block diagram explains the working of Machine
Learning algorithm:
Features of Machine Learning
• Machine learning uses data to detect various patterns in a
given dataset.
• It can learn from past data and improve automatically.
• It is a data‐driven technology.
• Machine learning is much similar to data mining as it also
deals with the huge amount of the data.
Need for Machine Learning
• Rapid increment in the production of data
• Solving complex problems, which are difficult for a
human
• Decision making in various sector including finance
• Finding hidden patterns and extracting useful
information from data.
Well‐Posed learning problems
Definition: A computer program is said to learn from experience E with
respect to some class of tasks T and performance measure P, if its
performance at tasks in T, as measured by P, improves with experience
E.

For example, a computer program that learns to play checkers might

improve its performance as measured by its ability to win at the class
of tasks involving playing checkers games, through experience
obtained by playing games against itself.
Well‐Posed learning problems
In general, to have a well‐defined learning problem, we must identity these
three features: the class of tasks, the measure of performance to be
improved, and the source of experience.

A checkers learning problem:

• Task T: playing checkers
• Performance measure P: percent of games won against opponents
• Training experience E: playing practice games against itself

We can specify many learning problems in this fashion, such as learning to

recognize handwritten words, or learning to drive a robotic automobile
autonomously.
Well‐Posed learning problems
A handwriting recognition learning problem:
• Task T: recognizing and classifying handwritten words within images
• Performance measure P: percent of words correctly classified
• Training experience E: a database of handwritten words with given classifications

A robot driving learning problem:

• Task T : driving on public four‐lane highways using vision sensors
• Performance measure P: average distance traveled before an error (as judged by
human overseer)
• Training experience E: a sequence of images and steering commands recorded
while observing a human driver
Well‐Posed learning problems
Our definition of learning is broad enough to include most tasks that we
would conventionally call "learning" tasks, as we use the word in everyday
language.

It is also broad enough to encompass computer programs that improve

from experience in quite straightforward ways. For example, a database
system that allows users to update data entries would fit our definition of
a learning system: it improves its performance at answering database
queries, based on the experience gained from database updates.
Some successful applications of Machine Learning
1. Learning to recognize spoken words:
All of the most successful speech recognition systems employ machine
learning in some form. For example, the SPHINXsy stem (e.g., Lee 1989)
learns speaker‐specific strategies for recognizing the primitive sounds
(phonemes) and words from the observed speech signal. Neural network
learning methods (e.g., Waibel et al. 1989) and methods for learning
hidden Markov models (e.g., Lee 1989) are effective for automatically
customizing to, individual speakers, vocabularies, microphone
characteristics, background noise, etc. Similar techniques have potential
applications in many signal‐interpretation problems.
Some successful applications of Machine Learning
2. Learning to drive an autonomous vehicle:
Machine learning methods have been used to train computer‐
controlled vehicles to steer correctly when driving on a variety of road
types. For example, the ALVINN system (Pomerleau 1989) has used its
learned strategies to drive unassisted at 70 miles per hour for 90 miles
on public highways among other cars. Similar techniques have possible
applications in many sensor‐based control problems.
Some successful applications of Machine Learning
3. Learning to classify new astronomical structures:
Machine learning methods have been applied to a variety of large
databases to learn general regularities implicit in the data. For example,
decision tree learning algorithms have been used by NASA to learn how
to classify celestial objects from the second Palomar Observatory Sky
Survey (Fayyad et al. 1995). This system is now used to automatically
classify all objects in the Sky Survey, which consists of three terrabytes
of image data.
Some successful applications of Machine Learning
4. Learning to play world‐class backgammon:
The most successful computer programs for playing games such as
backgammon are based on machine learning algorithms. For example,
the world's top computer program for backgammon, TD‐
GAMMON(Tesauro 1992, 1995). learned its strategy by playing over
one million practice games against itself. It now plays at a level
competitive with the human world champion. Similar techniques have
applications in many practical problems where very large search spaces
must be examined efficiently.
Some disciplines and examples of their influence
on Machine Learning
• Artificial intelligence
• Bayesian methods
• Computational complexity theory
• Control theory
• Information theory
• Philosophy
• Psychology and neurobiology
• Statistics
DESIGNING A LEARNING SYSTEM
1. Choosing the Training Experiences
2. Choosing from Target Function
3. Choosing a Representation from Target Function
4. Choosing a Function Approximation Algorithm
5. The Final Design
Choosing the Training Experience
• The first design choice we face is to choose the type of training
experience from which our system will learn.
• The type of training experience available can have a significant impact
on success or failure of the learner.
• One key attribute is whether the training experience provides direct
or indirect feedback regarding the choices made by the performance
system.
• For example, in learning to play checkers, the system might learn from
direct training examples consisting of individual checkers board states
and the correct move for each. Alternatively, it might have available
only indirect information consisting of the move sequences and final
outcomes of various games played.
Choosing the Training Experience
• In this later case, information about the correctness of specific moves
early in the game must be inferred indirectly from the fact that the
game was eventually won or lost.
• Here the learner faces an additional problem of credit assignment, or
determining the degree to which each move in the sequence
deserves credit or blame for the final outcome.
• Credit assignment can be a particularly difficult problem because the
game can be lost even when early moves are optimal, if these are
followed later by poor moves.
• Hence, learning from direct training feedback is typically easier than
learning from indirect feedback.
Choosing the Training Experience
• A second important attribute of the training experience is the degree
to which the learner controls the sequence of training examples.
• For example, the learner might rely on the teacher to select
informative board states and to provide the correct move for each.
• Alternatively, the learner might itself propose board states that it
finds particularly confusing and ask the teacher for the correct move.
• Or the learner may have complete control over both the board states
and (indirect) training classifications, as it does when it learns by
playing against itself with no teacher present.
Choosing the Training Experience
• A third important attribute of the training experience is how well it
represents the distribution of examples over which the final system
performance P must be measured.
• In general, learning is most reliable when the training examples follow
a distribution similar to that of future test examples.
• In our checkers learning scenario, the performance metric P is the
percent of games the system wins in the world tournament.
• If its training experience E consists only of games played against itself,
there is an obvious danger that this training experience might not be
fully representative of the distribution of situations over which it will
later be tested.
Choosing the Training Experience
A checkers learning problem:
• Task T: playing checkers
• Performance measure P: percent of games won in the world tournament
• Training experience E: games played against itself

In order to complete the design of the learning system, we must now

choose
1. the exact type of knowledge to be learned
2. a representation for this target knowledge
3. a learning mechanism
Choosing from Target Function
• The next design choice is to determine exactly what type of knowledge
will be learned and how this will be used by the performance program.
• Let us begin with a checkers‐playing program that can generate the legal
moves from any board state.
• The program needs only to learn how to choose the best move from
among these legal moves.
• This learning task is representative of a large class of tasks for which the
legal moves that define some large search space are known a priori, but
for which the best search strategy is not known.
• Many optimization problems fall into this class, such as the problems of
scheduling and controlling manufacturing processes where the available
manufacturing steps are well understood, but the best strategy for
sequencing them is not.
Choosing from Target Function
Let us therefore define the target value V(b) for an arbitrary board state b
in B, as follows:
1. if b is a final board state that is won, then V(b) = 100
2. if b is a final board state that is lost, then V(b) = ‐100
3. if b is a final board state that is drawn, then V(b) = 0
4. if b is a not a final state in the game, then V(b) = V(bl), where b' is the
best final board state that can be achieved starting from b and playing
optimally until the end of the game (assuming the opponent plays
optimally, as well).
Choosing from Target Function
• While this recursive definition specifies a value of V(b) for every board
state b, this definition is not usable by our checkers player because it is
not efficiently computable.
• Except for the trivial cases (cases 1‐3) in which the game has already
ended, determining the value of V(b) for a particular board state requires
(case 4) searching ahead for the optimal line of play, all the way to the
end of the game! Because this definition is not efficiently computable by
our checkers playing program, we say that it is a nonoperational
definition.
• The goal of learning in this case is to discover an operational description
of V ; that is, a description that can be used by the checkers‐playing
program to evaluate states and select moves within realistic time
bounds.
Choosing from Target Function
• Thus, we have reduced the learning task in this case to the problem of
discovering an operational description of the ideal target function V. It
may be very difficult in general to learn such an operational form of V
perfectly.
• In fact, we often expect learning algorithms to acquire only some
approximation to the target function, and for this reason the process of
learning the target function is often called function approximation.
• In the current discussion we will use the symbol Ṽ to refer to the
function that is actually learned by our program, to distinguish it from
the ideal target function V.
Choosing a Representation for the Target Function
• Now that we have specified the ideal target function V, we must choose
a representation that the learning program will use to describe the
function Ṽ that it will learn.
• As with earlier design choices, we again have many options. We could,
for example, allow the program to represent Ṽ using a large table with a
distinct entry specifying the value for each distinct board state.
• Or we could allow it to represent Ṽ using a collection of rules that match
against features of the board state, or a quadratic polynomial function of
predefined board features, or an artificial neural network.
• In general, this choice of representation involves a crucial tradeoff.
Choosing a Representation for the Target Function
• On one hand, we wish to pick a very expressive representation to allow
representing as close an approximation as possible to the ideal target
function V.
• On the other hand, the more expressive the representation, the more
training data the program will require in order to choose among the
alternative hypotheses it can represent.
• To keep the discussion brief, let us choose a simple representation for
any given board state, the function Ṽ will be calculated as a linear
combination of the following board features:
Choosing a Representation for the Target Function
• xl: the number of black pieces on the board
• x2: the number of red pieces on the board
• x3: the number of black kings on the board
• x4: the number of red kings on the board
• x5: the number of black pieces threatened by red (i.e., which can be
captured on red's next turn)
• X6: the number of red pieces threatened by black
Thus, our learning program will represent Ṽ (b) as a linear function of the
form
Ṽ(b) = w0 + w1x1 + w2x2 + w3x3 + w4x4 + w5x5 + w6x6
Choosing a Representation for the Target Function
• where w0 through w6 are numerical coefficients, or weights, to be
chosen by the learning algorithm. Learned values for the weights w1
through w6 will determine the relative importance of the various board
features in determining the value of the board, whereas the weight w0
will provide an additive constant to the board value.
Partial design of a checkers learning program:
• Task T: playing checkers
• Performance measure P: percent of games won in the world tournament
• Training experience E: games played against itself
• Target function: V:Board ‐> R
• Target function representation
Ṽ(b) = w0 + w1x1 + w2x2 + w3x3 + w4x4 + w5x5 + w6x6
Choosing a Representation for the Target Function
• The first three items above correspond to the specification of the
learning task, whereas the final two items constitute design choices for
the implementation of the learning program. Notice the net effect of this
set of design choices is to reduce the problem of learning a checkers
strategy to the problem of learning values for the coefficients w0 through
w6 in the target function representation.
Choosing a Function Approximation Algorithm
• In order to learn the target function Ṽ we require a set of training
examples, each describing a specific board state b and the training value
Vtrain(b) for b. In other words, each training example is an ordered pair of
the form (b, Vtrain(b)).
• For instance, the following training example describes a board state b in
which black has won the game (note x2 = 0 indicates that red has no
remaining pieces) and for which the target function value Vtrain(b)
therefore +100.
((x1 = 3, x2 = 0, x3 = 1, x4 = 0, x5 = 0, x6 = 0), + 100)
• Below we describe a procedure that first derives such training examples
from the indirect training experience available to the learner, then
adjusts the weights wi to best fit these training examples.
Choosing a Function Approximation Algorithm
1. ESTIMATING TRAINING VALUES: To assign the training value of Vtrain(b)
for any intermediate board state b to be Ṽ(successor (b)), where Ṽ is
the learner's current approximation to V and where Successor(b)
denotes the next board state following b for which it is again the
program's turn to move (i.e., the board state following the program's
move and the opponent's response). This rule for estimating training
values can be summarized as:
Rule for estimating training values
Vtrain(b)  Ṽ(successor (b)
While it may seem strange to use the current version of Ṽ to estimate
training values that will be used to refine this very same function,
notice that we are using estimates of the value of the Successor(b) to
estimate the value of board state b.
Choosing a Function Approximation Algorithm
2. ADJUSTING THE WEIGHTS: All that remains is to specify the learning
algorithm for choosing the weights wi to best fit the set of training
examples { (b, Vtrain(b))}. As a first step we must define what we mean by
the best fit to the training data.
One common approach is to define the best hypothesis, or set of weights,
as that which minimizes the squared error E between the training values
and the values predicted by the hypothesis Ṽ.

• Thus, we seek the weights, or equivalently the Ṽ , that minimize E for the
observed training examples.
Choosing a Function Approximation Algorithm
• Several algorithms are known for finding weights of a linear function that
minimize E defined in this way.
• In our case, we require an algorithm that will incrementally refine the
weights as new training examples become available and that will be
robust to errors in these estimated training values.
• One such algorithm is called the least mean squares, or LMS training
rule. For each observed training example it adjusts the weights a small
amount in the direction that reduces the error on this training example.
Choosing a Function Approximation Algorithm
• The LMS algorithm is defined as follows:

• Here ɳ is a small constant (e.g., 0.1) that moderates the size of the
weight update.
The Final Design
• The final design of our checkers learning system can be naturally
described by four distinct program modules that represent the central
components in many learning systems.
• These four modules, summarized in Figure, are as follows:
a) The Performance System is the module that must solve the given
performance task, in this case playing checkers, by using the learned
target function(s). It takes an instance of a new problem (new game) as
input and produces a trace of its solution (game history) as output. In
our case, the strategy used by the Performance System to select its
next move at each step is determined by the learned Ṽ evaluation
function. Therefore, we expect its performance to improve as this
evaluation function becomes increasingly accurate.
The Final Design
• The final design of our checkers learning system can be naturally
The Final Design
b) The Critic takes as input the history or trace of the game and
produces as output a set of training examples of the target function. As
shown in the diagram, each training example in this case corresponds
to some game state in the trace, along with an estimate Vtrain of the
target function value for this example. In our example, the Critic
corresponds to the training rule given by Equation (estimating training
values).

c) The Generalizer takes as input the training examples and produces

an output hypothesis that is its estimate of the target function. It
generalizes from the specific training examples, hypothesizing a general
function that covers these examples and other cases beyond the
training examples. In our example, the Generalizer corresponds to the
LMS algorithm, and the output hypothesis is the function Ṽ described
by the learned weights wo, . . . , W6.
The Final Design
d) The Experiment Generator takes as input the current hypothesis
(currently learned function) and outputs a new problem (i.e., initial
board state) for the Performance System to explore. Its role is to pick
new practice problems that will maximize the learning rate of the
overall system. In our example, the Experiment Generator follows a
very simple strategy: It always proposes the same initial game board to
begin a new game. More sophisticated strategies could involve creating
board positions designed to explore particular regions of the state
space.
• Together, the design choices we made for our checkers program produce
specific instantiations for the performance system, critic; generalizer, and
experiment generator. Many machine learning systems can‐be usefully
characterized in terms of these four generic modules
The Final Design
• The sequence of design choices made for the checkers program is
summarized in Figure
 ISSUES IN MACHINE LEARNING
Our checkers example raises a number of generic questions about machine
learning. The field of machine learning, and much of this book, is concerned
with answering questions such as the following:
• What algorithms exist for learning general target functions from specific
training examples? In what settings will particular algorithms converge to
the desired function, given sufficient training data? Which algorithms
perform best for which types of problems and representations?
• How much training data is sufficient? What general bounds can be found to
relate the confidence in learned hypotheses to the amount of training
experience and the character of the learner's hypothesis space?
• When and how can prior knowledge held by the learner guide the process
of generalizing from examples? Can prior knowledge be helpful even when
it is only approximately correct?
 ISSUES IN MACHINE LEARNING
• What is the best strategy for choosing a useful next training experience,
and how does the choice of this strategy alter the complexity of the
learning problem?
• What is the best way to reduce the learning task to one or more function
approximation problems? Put another way, what specific functions should
the system attempt to learn? Can this process itself be automated?
• How can the learner automatically alter its representation to improve its
ability to represent and learn the target function?
Artificial Intelligence Vs Machine Learning
Artificial Intelligence Machine learning
Artificial intelligence is a technology which enables a machine to Machine learning is a subset of AI which allows a machine to
simulate human behavior. automatically learn from past data without programming explicitly.
The goal of AI is to make a smart computer system like humans The goal of ML is to allow machines to learn from data so that
to solve complex problems. they can give accurate output.
In AI, we make intelligent systems to perform any task like a In ML, we teach machines with data to perform a particular task
human. and give an accurate result.
Machine learning and deep learning are the two main subsets of Deep learning is a main subset of machine learning.
AI.
AI has a very wide range of scope. Machine learning has a limited scope.
AI is working to create an intelligent system which can perform Machine learning is working to create machines that can perform
various complex tasks. only those specific tasks for which they are trained.
AI system is concerned about maximizing the chances of success. Machine learning is mainly concerned about accuracy and
patterns.
The main applications of AI are Siri, customer support using The main applications of machine learning are Online
catboats, Expert System, Online game playing, intelligent recommender system, Google search algorithms, Facebook
humanoid robot, etc. auto friend tagging suggestions, etc.
On the basis of capabilities, AI can be divided into three types, Machine learning can also be divided into mainly three types that
which are, Weak AI, General AI, and Strong AI. are Supervised learning, Unsupervised learning,
and Reinforcement learning.
It includes learning, reasoning, and self-correction. It includes learning and self-correction when introduced with new
data.
AI completely deals with Structured, semi-structured, and Machine learning deals with Structured and semi-structured data.
unstructured data.
TYPES OF MACHINE LEARNING
1. Supervised Learning
2. Unsupervised Learning
3. Reinforcement Learning
Learning Associations
What is Association Learning?
• Association learning is a rule based machine learning and
data mining technique that finds important relations
between variables or features in a data set. Unlike
conventional association algorithms measuring degrees of
similarity, association rule learning identifies hidden
correlations in databases by applying some measure of
interestingness to generate an association rule for new
searches.
Learning Associations
How Does Association Learning Work?
• Association rule algorithms count the frequency of
complimentary occurrences, or associations, across a large
collection of items or actions. The goal is to find
associations that take place together far more often than
you would find in a random sampling of possibilities. This
rule‐based approach is a fast and powerful tool for mining
categorized, non‐numeric databases.
Learning Associations
How Does Association Learning Work?
• In the case of retail—for example, a supermarket chain, which is
finding associations between products bought by customers: If
people who buy X typically also buy Y, and if there is a customer
who buys X and does not buy Y, he or she is a potential Y customer.
Once we find such customers, we can target them for cross‐selling.
• In finding an association rule, we are interested in learning a
conditional probability of the form P(Y|X) where Y is the product we
would like to condition on X, which is the product or the set of
products which we know that the customer has already purchased.
• Let us say, going over our data, we calculate that P(chips|beer) =
0.7. Then, we can define the rule:
70 percent of customers who buy beer also buy chips.
Learning Associations
How Does Association Learning Work?
• We may want to make a distinction among customers and toward
this, estimate P(Y|X,D) where D is the set of customer attributes,
for example, gender, age, marital status, and so on, assuming that
we have access to this information.
• If this is a bookseller instead of a supermarket, products can be
books or authors. In the case of a web portal, items correspond to
links to web pages, and we can estimate the links a user is likely
to click and use this information to download such pages in
advance for faster access.
Learning Associations
Practical Uses of Association Learning
• Basket data analysis – Whether planning product placement in a
storefront, running a marketing campaign or designing a business
catalog, association mining is a useful tool to take the guesswork out
of what your customers are looking for.
• Web usage mining and intrusion detection – Finding these hidden
correlations is a powerful predictive tool to discover brand new
security threats and network performance issues that haven’t been
analyzed first by a human.
• Bioinformatics – From biology to engineering and everything in
between, association mining is one of the go‐to foundational tools for
spotting overlooked and potentially useful techniques.
Supervised Learning
• Supervised learning is the types of machine learning in
which machines are trained using well "labelled" training
data, and on basis of that data, machines predict the
output.
• The labelled data means some input data is already tagged
with the correct output.
• In supervised learning, the training data provided to the
machines work as the supervisor that teaches the
machines to predict the output correctly. It applies the
same concept as a student learns in the supervision of the
teacher.
Supervised Learning
• Supervised learning is a process of providing input data as
well as correct output data to the machine learning
model.
• The aim of a supervised learning algorithm is to find a
mapping function to map the input variable(x) with the
output variable(y).
• In the real‐world, supervised learning can be used for Risk
Assessment, Image classification, Fraud Detection, spam
filtering, etc.
Supervised Learning
How Supervised Learning Works?
• In supervised learning, models are trained using labelled
dataset, where the model learns about each type of data.
Once the training process is completed, the model is
tested on the basis of test data (a subset of the training
set), and then it predicts the output.
• The working of Supervised learning can be easily
understood by the below example and diagram:
Supervised Learning
Supervised Learning
• Suppose we have a dataset of different types of shapes
which includes square, rectangle, triangle, and Polygon.
Now the first step is that we need to train the model for
each shape.
• If the given shape has four sides, and all the sides are equal,
then it will be labelled as a Square.
• If the given shape has three sides, then it will be labelled as
a triangle.
• If the given shape has six equal sides then it will be labelled
as hexagon.
Supervised Learning
• Now, after training, we test our model using the test set,
and the task of the model is to identify the shape.
• The machine is already trained on all types of shapes, and
when it finds a new shape, it classifies the shape on the
bases of a number of sides, and predicts the output.
Supervised Learning
Steps Involved in Supervised Learning:

• First Determine the type of training dataset

• Collect/Gather the labelled training data.
• Split the training dataset into training dataset, test dataset, and validation
dataset.
• Determine the input features of the training dataset, which should have
enough knowledge so that the model can accurately predict the output.
• Determine the suitable algorithm for the model, such as support vector
machine, decision tree, etc.
• Execute the algorithm on the training dataset. Sometimes we need
validation sets as the control parameters, which are the subset of training
datasets.
• Evaluate the accuracy of the model by providing the test set. If the model
predicts the correct output, which means our model is accurate.
Supervised Learning
Types of supervised Machine learning Algorithms:
• Supervised learning can be further divided into two types
of problems:
Supervised Learning
1. Regression
• Regression algorithms are used if there is a relationship
between the input variable and the output variable. It is used
for the prediction of continuous variables, such as Weather
forecasting, Market Trends, etc. Below are some popular
Regression algorithms which come under supervised learning:
• Linear Regression
• Regression Trees
• Non‐Linear Regression
• Bayesian Linear Regression
• Polynomial Regression
Supervised Learning
2. Classification
• Classification algorithms are used when the output
variable is categorical, which means there are two classes
such as Yes‐No, Male‐Female, True‐false, etc.
• Spam Filtering,
• Random Forest
• Decision Trees
• Logistic Regression
• Support vector Machines
Supervised Learning
• Advantages of Supervised learning:
• With the help of supervised learning, the model can predict the
output on the basis of prior experiences.
• In supervised learning, we can have an exact idea about the classes
of objects.
• Supervised learning model helps us to solve various real‐world
problems such as fraud detection, spam filtering, etc.
• Disadvantages of supervised learning:
• Supervised learning models are not suitable for handling the
complex tasks.
• Supervised learning cannot predict the correct output if the test
data is different from the training dataset.
• Training required lots of computation times.
• In supervised learning, we need enough knowledge about the
classes of object.
Unsupervised Learning
• As the name suggests, unsupervised learning is a machine
learning technique in which models are not supervised
using training dataset.
• Instead, models itself find the hidden patterns and insights
from the given data. It can be compared to learning which
takes place in the human brain while learning new things.
It can be defined as:
• Unsupervised learning is a type of machine learning in
which models are trained using unlabeled dataset and
are allowed to act on that data without any supervision.
Unsupervised Learning
• Unsupervised learning cannot be directly applied to a
regression or classification problem because unlike
supervised learning, we have the input data but no
corresponding output data.
• The goal of unsupervised learning is to find the underlying
structure of dataset, group that data according to
similarities, and represent that dataset in a compressed
format.
 Unsupervised Learning
• Example: Suppose the unsupervised
learning algorithm is given an input
dataset containing images of different
types of cats and dogs. The algorithm is
never trained upon the given dataset,
which means it does not have any idea
about the features of the dataset.
• The task of the unsupervised learning
algorithm is to identify the image
features on their own. Unsupervised
learning algorithm will perform this task
by clustering the image dataset into the
groups according to similarities
between images.
Unsupervised Learning
Why use Unsupervised Learning?
• Unsupervised learning is helpful for finding useful insights from
the data.
• Unsupervised learning is much similar as a human learns to
think by their own experiences, which makes it closer to the
real AI.
• Unsupervised learning works on unlabeled and uncategorized
data which make unsupervised learning more important.
• In real‐world, we do not always have input data with the
corresponding output so to solve such cases, we need
unsupervised learning.
Unsupervised Learning
Working of Unsupervised Learning
Unsupervised Learning
• Here, we have taken an unlabeled input data, which
means it is not categorized and corresponding outputs are
also not given. Now, this unlabeled input data is fed to the
machine learning model in order to train it.
• Firstly, it will interpret the raw data to find the hidden
patterns from the data and then will apply suitable
algorithms such as k‐means clustering, Decision tree, etc.
• Once it applies the suitable algorithm, the algorithm
divides the data objects into groups according to the
similarities and difference between the objects.
Unsupervised Learning
Types of Unsupervised Learning Algorithm:
Unsupervised Learning
1. Clustering: Clustering is a method of grouping the objects into
clusters such that objects with most similarities remains into a group
and has less or no similarities with the objects of another group.
Cluster analysis finds the commonalities between the data objects
and categorizes them as per the presence and absence of those
commonalities.

2. Association: An association rule is an unsupervised learning

method which is used for finding the relationships between variables
in the large database. It determines the set of items that occurs
together in the dataset. Association rule makes marketing strategy
more effective. Such as people who buy X item (suppose a bread) are
also tend to purchase Y (Butter/Jam) item. A typical example of
Association rule is Market Basket Analysis.
Unsupervised Learning
Unsupervised Learning algorithms:
• K‐means clustering
• KNN (k‐nearest neighbors)
• Hierarchal clustering
• Anomaly detection
• Neural Networks
• Principle Component Analysis
• Independent Component Analysis
• Apriori algorithm
• Singular value decomposition
Unsupervised Learning
Advantages of Unsupervised Learning
• Unsupervised learning is used for more complex tasks as compared
to supervised learning because, in unsupervised learning, we don't
have labeled input data.
• Unsupervised learning is preferable as it is easy to get unlabeled
data in comparison to labeled data.

Disadvantages of Unsupervised Learning

• Unsupervised learning is intrinsically more difficult than supervised
learning as it does not have corresponding output.
• The result of the unsupervised learning algorithm might be less
accurate as input data is not labeled, and algorithms do not know
the exact output in advance.
Difference between Supervised and Unsupervised Learning
Supervised Learning Unsupervised Learning
Supervised learning algorithms are trained using labeled data. Unsupervised learning algorithms are trained using unlabeled data.

Supervised learning model takes direct feedback to check if it is predicting Unsupervised learning model does not take any feedback.
correct output or not.
Supervised learning model predicts the output. Unsupervised learning model finds the hidden patterns in data.

In supervised learning, input data is provided to the model along with the In unsupervised learning, only input data is provided to the model.
output.
The goal of supervised learning is to train the model so that it can predict The goal of unsupervised learning is to find the hidden patterns and useful
the output when it is given new data. insights from the unknown dataset.

Supervised learning needs supervision to train the model. Unsupervised learning does not need any supervision to train the model.

Supervised learning can be categorized Unsupervised Learning can be classified

in Classification and Regression problems. in Clustering and Associations problems.

Supervised learning can be used for those cases where we know the input Unsupervised learning can be used for those cases where we have only
as well as corresponding outputs. input data and no corresponding output data.

Supervised learning model produces an accurate result. Unsupervised learning model may give less accurate result as compared to
supervised learning.
Supervised learning is not close to true Artificial intelligence as in this, we Unsupervised learning is more close to the true Artificial Intelligence as it
first train the model for each data, and then only it can predict the correct learns similarly as a child learns daily routine things by his experiences.
output.
It includes various algorithms such as Linear Regression, Logistic It includes various algorithms such as Clustering, KNN, and Apriori
Regression, Support Vector Machine, Multi-class Classification, Decision tree, algorithm.
Bayesian Logic, etc.
 Reinforcement Learning
What is Reinforcement Learning?
• Reinforcement Learning is a feedback‐based Machine learning technique in
which an agent learns to behave in an environment by performing the
actions and seeing the results of actions. For each good action, the agent
gets positive feedback, and for each bad action, the agent gets negative
feedback or penalty.
• In Reinforcement Learning, the agent learns automatically using feedbacks
without any labeled data, unlike supervised learning.
• Since there is no labeled data, so the agent is bound to learn by its
experience only.
• RL solves a specific type of problem where decision making is sequential, and
the goal is long‐term, such as game‐playing, robotics, etc.
• The agent interacts with the environment and explores it by itself. The
primary goal of an agent in reinforcement learning is to improve the
performance by getting the maximum positive rewards.
 Reinforcement Learning
What is Reinforcement Learning?
• The agent learns with the process of hit and trial, and based on the experience, it
learns to perform the task in a better way. Hence, we can say that "Reinforcement
learning is a type of machine learning method where an intelligent agent (computer
program) interacts with the environment and learns to act within that." How a
Robotic dog learns the movement of his arms is an example of Reinforcement
learning.
• It is a core part of Artificial intelligence, and all AI agent works on the concept of
reinforcement learning. Here we do not need to pre‐program the agent, as it learns
from its own experience without any human intervention.
• Example: Suppose there is an AI agent present within a maze environment, and his
goal is to find the diamond. The agent interacts with the environment by performing
some actions, and based on those actions, the state of the agent gets changed, and it
also receives a reward or penalty as feedback.
• The agent continues doing these three things (take action, change state/remain in
the same state, and get feedback), and by doing these actions, he learns and explores
the environment.
 Reinforcement Learning
What is Reinforcement Learning?
• The agent learns that what actions lead to positive feedback or
rewards and what actions lead to negative feedback penalty. As a
positive reward, the agent gets a positive point, and as a penalty, it
gets a negative point.
Reinforcement Learning
Key Features of Reinforcement Learning
• In RL, the agent is not instructed about the environment
and what actions need to be taken.
• It is based on the hit and trial process.
• The agent takes the next action and changes states
according to the feedback of the previous action.
• The agent may get a delayed reward.
• The environment is stochastic, and the agent needs to
explore it to reach to get the maximum positive rewards.
Reinforcement Learning
Types of Reinforcement learning
There are mainly two types of reinforcement learning, which are:
1. Positive Reinforcement:
• The positive reinforcement learning means adding something to increase the
tendency that expected behavior would occur again. It impacts positively on the
behavior of the agent and increases the strength of the behavior.
• This type of reinforcement can sustain the changes for a long time, but too much
positive reinforcement may lead to an overload of states that can reduce the
consequences.
2. Negative Reinforcement:
• The negative reinforcement learning is opposite to the positive reinforcement as it
increases the tendency that the specific behavior will occur again by avoiding the
negative condition.
• It can be more effective than the positive reinforcement depending on situation and
behavior, but it provides reinforcement only to meet minimum behavior.
Difference between Reinforcement Learning & Supervised
Learning

Reinforcement Learning Supervised Learning

RL works by interacting with the Supervised learning works on the
environment. existing dataset.
The RL algorithm works like the human Supervised Learning works as when a
brain works when making some human learns things in the supervision of
decisions. a guide.
There is no labeled dataset is present The labeled dataset is present.

No previous training is provided to the Training is provided to the algorithm so

learning agent. that it can predict the output.

RL helps to take decisions sequentially. In Supervised learning, decisions are

made when input is given.
Reinforcement Learning Applications
• Robotics:
• RL is used in Robot navigation, Robo‐soccer, walking, juggling, etc.
• Control:
• RL can be used for adaptive control such as Factory processes, admission control in
telecommunication, and Helicopter pilot is an example of reinforcement learning.
• Game Playing:
• RL can be used in Game playing such as tic‐tac‐toe, chess, etc.
• Chemistry:
• RL can be used for optimizing the chemical reactions.
• Business:
• RL is now used for business strategy planning.
• Manufacturing:
• In various automobile manufacturing companies, the robots use deep reinforcement learning
to pick goods and put them in some containers.
• Finance Sector:
• The RL is currently used in the finance sector for evaluating trading strategies.