Machine Learning

R18 B.Tech. CSE (AIML) III Year – I Semester

By
Mohammed Afzal
Assistant Professor
Computer Science & Engineering Department
Sphoorthy Engineering College
UNIT - I
Chapter-I: Introduction - Well-posed learning problems, designing a
learning system, Perspectives and issues in machine learning
INTRODUCTION:
 Computers can be made to learn.
 If we could understand how to program them to learn and to improve them
automatically with experience, the impact would be dramatic.
Examples:
o Computers learning from medical records and giving us which treatments
are most effective for new diseases.
o Houses learning from experience to optimize energy costs based on the
particular usage patterns of their occupants.
o Personal Software Assistants (PSAs) learning the evolving interests of their
users in order to highlight relevant stories from the online morning
newspaper.
 A successful understanding of how to make computers learn is required to
open up many new uses of computers and new levels of competence and
customization.
 A detailed understanding of information processing algorithms for machine
learning might lead to a better understanding of human learning abilities
(and disabilities) as well.
 Till now we don’t know, how to make computers learn nearly as well as
people learn. However, algorithms have been invented that are effective
for certain types of learning tasks.
Examples:
o For problems such as speech recognition, algorithms based on
machine learning outperform all other approaches that have been
attempted to date.
o In data mining, machine learning algorithms are being used routinely
to discover valuable knowledge from large commercial databases
containing equipment maintenance records, loan applications,
financial transactions, medical records, and so on …etc.,
 As our understanding of computers continues to mature, it seems inevitable
that machine learning will play an increasingly central role in computer
science and computer technology.
 The following table summarizes Several recent applications of machine
learning

Learning to recognize spoken words.

Learning to drive an autonomous vehicle.

Learning to classify new astronomical structures.

Learning to play world-class backgammon.

 Machine learning is a multidisciplinary field. Derived from artificial
intelligence, probability and statistics, computational complexity
theory, control theory, information theory, philosophy, psychology,
neurobiology, and other fields.
WELL-POSED LEARNING PROBLEMS

 Learning can be broadly defined as, to include any computer program that
improves its performance at some tasks through experience.

 Put more precisely,

Definition: A computer program is said to learn from experience E with respect to

some class of tasks T and performance measure P, if its performance at tasks in T,
as measured by P, improves with experience E.

 For example, a computer program that learns to play checkers might improve its
performance as measured by its ability to win at the class of tasks involving playing
checkers games, through experience obtained by playing games against itself.
 In general, to have a well-defined learning problem, we must identity
these three features: the class of tasks T, the measure of performance to
be improved P, and the source of experience E.
A checkers learning problem:

 Task T: playing checkers

 Performance measure P: percent of games won against opponents
 Training experience E: playing practice games against itself
A handwriting recognition learning problem:

 Task T: recognizing and classifying handwritten words within images

 Performance measure P: percent of words correctly classified
 Training experience E: a database of handwritten words with given
classifications
A robot driving learning problem:

 Task T: driving on public four-lane highways using vision sensors

 Performance measure P: average distance travelled before an error (as
judged by human overseer)
 Training experience E: a sequence of images and steering commands
recorded while observing a human driver
Fruit Prediction Problem

 Task – forecasting different fruits for recognition

 Performance Measure – able to predict maximum variety of fruits

 Experience – training machine with the largest datasets of fruits images

Face Recognition Problem

 Task – predicting different types of faces

 Performance Measure – able to predict maximum types of faces

 Experience – training machine with maximum amount of datasets of

different face images
Automatic Translation of documents

 Task – translating one type of language used in a document to other language

 Performance Measure – able to convert one language to other efficiently

 Experience – training machine with a large dataset of different types of

languages
DESIGNING A LEARNING SYSTEM

 To illustrate some of the basic design issues and approaches to machine

learning, consider designing a program to learn to play checkers game,
with the goal of entering it in the world checkers tournament.
 To design a learning system that follows the learning process, consider the
following design choices.

1. Type of training experience

2. Choosing the Target Function

3. Choosing a representation for the Target Function

4. Choosing an approximation algorithm for the Target Function

5. The final Design

 Take the example of checkers game learning problem and apply the above
design choices.
 For a checkers learning problem, the three elements will be:
 Task T: To play checkers
 Performance measure P: Total percentage of the games won in the
tournament.
 Training experience E: A set of games played against itself.
1. Type of training experience:
 When designing the checker's learning system, the type of training experience
available for a learning system will have a significant effect on the success or
failure of the learning.

A. Direct or Indirect training experience:

 In the case of direct training experience, an individual board states and
correct move for each board state are given.
 In case of indirect training experience, the move sequences for a game and
the final result (win, lose or draw) are given for a number of games.
 How to assign credit or blame to individual moves is the credit assignment
problem.
B. Teacher or Not:
Supervised:
The training experience will be labelled, which means, all the board states
will be labelled with the correct move. So the learning takes place in the
presence of a supervisor or a teacher.
Un-Supervised:
The training experience will be unlabelled, which means, all the board states
will not have the moves. So the learner generates random games and plays
against itself with no supervision or teacher involvement.
Semi-supervised:
Learner generates game states and asks the teacher for help in finding the
correct move if the board state is confusing.
C. Is the training experience good:
 Do the training examples represent the distribution of examples over which
the final system performance will be measured?
 Performance is best when training examples and test examples are from the
same/a similar distribution.
 The checker player learns by playing against oneself. Its experience is
indirect. It may not encounter moves that are common in human expert
play. Once the proper training experience is available, the next design step
will be choosing the Target Function.
2. Choosing the Target Function:
 When you are playing the checkers game, at any moment of time, you make a
decision on choosing the best move from different possibilities.
 You think and apply the learning that you have gained from the experience.
Here the learning is, for a specific board, you move a checker such that your
board state tends towards the winning situation. Now the same learning has to
be defined in terms of the target function.
 Here there are 2 considerations — direct and indirect experience.
During the direct experience:
 The checkers learning system needs only to learn how to choose the best
move among some large search space. We need to find a target function that
will help us choosing the best move among alternatives.
 Let us call this function Choose Move and use the notation
Choose Move: B→M to indicate that this function accepts as input any board
from the set of legal board states B and produces as output some move from
the set of legal moves M.
 During the indirect experience:

 When there is indirect experience, it becomes

difficult to learn such function. How about
assigning a real score to the board state.
 So the function be V : B →R indicating that this
accepts as input any board from the set of legal
board states B and produces an output a real
score.
 This function assigns the higher scores to better
board states.
 Let us therefore define the target value V(b) for an arbitrary board state b in
B, as follows:
1. if b is a final board state that is won, then V(b) = 100
2. if b is a final board state that is lost, then V(b) = -100
3. if b is a final board state that is drawn, then V(b) = 0
4. if b is a not a final state in the game, then V (b) = V (b’)
Where b’ is the best final board state that can be achieved starting from b
and playing optimally until the end of the game.

 The (4) is a recursive definition and to determine the value of V(b) for a particular
board state, it performs the search ahead for the optimal line of play, all the way to
the end of the game. So this definition is not efficiently computable by our checkers
playing program, we say that it is a non-operational definition.
3. Choosing a representation for the Target Function
 Now that we have specified the ideal target function V, we must choose a
representation that the learning program will use to describe the function ^V
that it will learn.
 To keep the discussion brief, let us choose a simple representation: for any
given board state, the function ^V will be calculated as a linear combination
of the following board features:

 x1(b) — number of black pieces on board b

 x2(b) — number of red pieces on b
 x3(b) — number of black kings on b
 x4(b) — number of red kings on b
 x5(b) — number of red pieces threatened by black
 x6(b) — number of black pieces threatened by red

^V = w0 + w1 · x1(b) + w2 · x2(b) + w3 · x3(b) + w4 · x4(b) +w5 · x5(b)

+ w6 · x6(b)
^V = w0 + w1 · x1(b) + w2 · x2(b) + w3 · x3(b) + w4 · x4(b) +w5 · x5(b)
+ w6 · x6(b)

 Where w0 through w6 are numerical coefficients or weights to be obtained by

a learning algorithm. Weights w1 to w6 will determine the relative importance
of different board features.
Specification of the Machine Learning Problem at this time:

 Till now we worked on choosing the type of training experience, choosing the target
function and its representation.

 The checkers learning task can be summarized as below.

Task T: Play Checkers

Performance Measure: % of games won in world tournament

Training Experience E: opportunity to play against itself

Target Function: V: Board → R

 Target Function Representation:

^V = w0 + w1 · x1(b) + w2 · x2(b) + w3 ·x3(b) + w4 · x4(b) +w5 · x5(b) + w6 ·

x6(b)

The first three items above correspond to the specification of the learning task, whereas the
final two items constitute design choices for the implementation of the learning program.
4. Choosing an approximation algorithm for the Target Function:

 Generating training data — To train our learning program, we need a set of

training data, each describing a specific board state b and the training value
V_train (b) for b. Each training example is an ordered pair <b,v_train(b)>.

 Temporal difference (TD) learning is a concept central to reinforcement learning,

in which learning happens through the iterative correction of your estimated
returns towards a more accurate target return.

†V_train(b) ← ^V(Successor(b))
5. Final Design for Checkers Learning system:

The final design of our checkers learning system can be naturally described by
four distinct program modules that represent the central components in many
learning systems.

1. The performance System: Takes a new board as input and outputs a trace of the
game it played against itself.

2. The Critic: Takes the trace of a game as an input and outputs a set of training
examples of the target function.
4. The Generalizer: Takes training examples as input and outputs a hypothesis
that estimates the target function. Good generalization to new cases is crucial.

5. The Experiment Generator: Takes the current hypothesis (currently learned

function) as input and outputs a new problem (an initial board state) for the
performance system to explore.
Final Design of Checkers Learning Program