Support Vector Machines: (Vapnik, 1979)
Support Vector Machines: (Vapnik, 1979)
(Vapnik, 1979)
S = {(x1, y1), (x2, y2), ..., (xn, yn) | (xi, yi)∈ ℜn ×{+1, -1}
w⋅ x + b = 0
Equation of line:
f2 + w⋅ x + b = w1 f1 + w 2 f 2 + b = 0
- + +
€ -
- +
- €
f1
w⋅ x + b = 0
f2 +
- + +
€ -
- +
f1
f2 +
- + +
-
- d+ +
- -
f1
• The margin of the negative examples, d- with respect to
that hyperplane, is the shortest distance from a negative
example to the hyperplane:
f2 +
- + +
d-
-
- d+ +
- -
f1
• The margin of the training set S with respect to the
hyperplane is d+ + d- .
f2 +
- + +
d-
-
- d+ +
- -
f1
• The margin of the training set S with respect to the
hyperplane is d+ + d- .
f2 +
+ +
Vapnik showed that the hyperplane- d-
-
d+ +
maximizing the margin of S will have -
minimal VC dimension in the set of- -
all consistent hyperplanes, and will
thus be optimal. f1
• The margin of the training set S with respect to the
hyperplane is d+ + d- .
f2 +
+ +
Vapnik showed that the hyperplane- d-
-
d+ +
maximizing the margin of S will have -
minimal VC dimension in the set of- -
all consistent hyperplanes, and will
thus be optimal. f1
This is an optimization
problem!
• Note that the hyperplane is defined as
w⋅ x + b = 0
Classified Classified
Positive Negative
Positive
Examples True positives False negatives
(TP) (FN)
Negative
Examples False positives True negatives
(FP) (TN)
Some performance measures
• Accuracy: proportion of classifications, over all the N
examples, that were correct:
Accuracy =
Recall =
Precision =
Interpretation of precision and recall
• Precision and recall are often plotted against one another,
especially in “detection” applications (such as spam
detection), when positive examples are sparse in the
observed data.