Scribd
Upload
17 views11 pages

Concept Learning

Uploaded by

royalaclashg6k8
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
17 views11 pages

Concept Learning

Uploaded by

royalaclashg6k8
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
You are on page 1/ 11

Concept Learning

• Inducing general functions from specific training examples is a


main issue of machine learning.
• Concept Learning: Acquiring the definition of a general category
from given sample positive and negative training examples of the
category.
• Concept Learning can seen as a problem of searching through a
predefined space of potential hypotheses for the hypothesis
that best fits the training examples.
• The hypothesis space has a general-to-specific ordering of
hypotheses, and the search can be efficiently organized by taking
advantage of a naturally occurring structure over the hypothesis 03/27/2024
space.
Concept Learning
• A Formal Definition for Concept Learning:

Inferring a boolean-valued function from training examples


of its input and output.

• An example for concept-learning is the learning of bird-concept from


the given examples of birds (positive examples) and non-birds
(negative examples).
• We are trying to learn the definition of a concept from given examples.
03/27/2024
A Concept Learning Task – Enjoy Sport Training Examples

Example Sky AirTemp Humidity Wind Water Forecast EnjoySport

1 Sunny Warm Normal Strong Warm Same YES

2 Sunny Warm High Strong Warm Same YES

3 Rainy Cold High Strong Warm Change NO

4 Sunny Warm High Strong Warm Change YES

ATTRIBUTES

CONCEPT
• A set of example days, and each is described by six attributes. 03/27/2024
• The task is to learn to predict the value of EnjoySport for arbitrary day,
EnjoySport – Hypothesis Representation

• Each hypothesis consists of a conjuction of constraints on the


instance attributes.
• Each hypothesis will be a vector of six constraints, specifying the
values of the six attributes
– (Sky, AirTemp, Humidity, Wind, Water, and Forecast).
• Each attribute will be:
? - indicating any value is acceptable for the attribute (don’t care)
single value – specifying a single required value (ex. Warm) (specific)
0 - indicating no value is acceptable for the attribute (no value)

03/27/2024
Hypothesis Representation
• A hypothesis:
Sky AirTemp Humidity Water Forecast
Wind
< Sunny, ? , ? , Strong , ? , Same >
• The most general hypothesis – that every day is a positive example
<?, ?, ?, ?, ?, ?>
• The most specific hypothesis – that no day is a positive example
<0, 0, 0, 0, 0, 0>
• EnjoySport concept learning task requires learning the sets of days
for which EnjoySport=yes, describing this set by a conjunction of03/27/2024

constraints over the instance attributes.


EnjoySport Concept Learning Task
Given
– Instances X : set of all possible days, each described by the attributes
• Sky – (values: Sunny, Cloudy, Rainy)
• AirTemp – (values: Warm, Cold)
• Humidity – (values: Normal, High)
• Wind – (values: Strong, Weak)
• Water – (values: Warm, Cold)
• Forecast – (values: Same, Change)
– Target Concept (Function) c : EnjoySport : X  {0,1}
– Hypotheses H : Each hypothesis is described by a conjunction of constraints
on the attributes.
– Training Examples D : positive and negative examples of the target function
03/27/2024
Determine
– A hypothesis h in H such that h(x) = c(x) for all x in D.
The Inductive Learning Hypothesis

• Although the learning task is to determine a hypothesis h identical to


the target concept cover the entire set of instances X, the only
information available about c is its value over the training examples.
– Inductive learning algorithms can at best guarantee that the output hypothesis fits the
target concept over the training data.
– Lacking any further information, our assumption is that the best hypothesis regarding
unseen instances is the hypothesis that best fits the observed training data. This is
the fundamental assumption of inductive learning.

• The Inductive Learning Hypothesis - Any hypothesis found to


approximate the target function well over a sufficiently large set of
training examples will also approximate the target function well
03/27/2024
over other unobserved examples.
Concept Learning As Search

• Concept learning can be viewed as the task of searching through a large


space of hypotheses implicitly defined by the hypothesis
representation.
• The goal of this search is to find the hypothesis that best fits the
training examples.
• By selecting a hypothesis representation, the designer of the learning
algorithm implicitly defines the space of all hypotheses that the
program can ever represent and therefore can ever learn.

03/27/2024
Enjoy Sport - Hypothesis Space
• Sky has 3 possible values, and other 5 attributes have 2 possible values.
• There are 96 (= 3.2.2.2.2.2) distinct instances in X.
• There are 5120 (=5.4.4.4.4.4) syntactically distinct hypotheses in H.
– Two more values for attributes: ? and 0
• Every hypothesis containing one or more 0 symbols represents the
empty set of instances; that is, it classifies every instance as
negative.
• There are 973 (= 1 + 4.3.3.3.3.3) semantically distinct hypotheses in
H.
– Only one more value for attributes: ?, and one hypothesis representing empty set of
instances. 03/27/2024

• Although EnjoySport has small, finite hypothesis space, most


General-to-Specific Ordering of Hypotheses
• Many algorithms for concept learning organize the search through the hypothesis
space by relying on a general-to-specific ordering of hypotheses.
• By taking advantage of this naturally occurring structure over the hypothesis space, we
can design learning algorithms that exhaustively search even infinite hypothesis
spaces without explicitly enumerating every hypothesis.

• Consider two hypotheses


h1 = (Sunny, ?, ?, Strong, ?, ?)
h2 = (Sunny, ?, ?, ?, ?, ?)

• Now consider the sets of instances that are classified positive by hl and by h2.
– Because h2 imposes fewer constraints on the instance, it classifies more instancesas
positive.
03/27/2024
– In fact, any instance classified positive by hl will also be classified positive by h2.
– Therefore, we say that h2 is more general than hl.
More-General-Than Relation
• For any instance x in X and hypothesis h in H, we say that x satisfies
h if and only if h(x) = 1.

• More-General-Than-Or-Equal Relation:
Let h1 and h2 be two boolean-valued functions defined over X.
Then h1 is more-general-than-or-equal-to h2 (written h1
≥ h2) if and only if
any instance that satisfies h2 also satisfies h1.

• h1 is more-general-than h2 ( h1 > h2) if and only if h1≥h2 is true 03/27/2024

and h2≥h1 is false. We also say h2 is more-specific-than h1.

You might also like