UNIT- 2

CHAPTER- 3
SUPPORT VECTOR
MACHINE
SUPPORT VECTOR MACHINES
● Support Vector Machine or SVM is one of the most popular Supervised Learning
algorithms, which is used for Classification as well as Regression problems. However,
primarily, it is used for Classification problems in Machine Learning.
● The goal of the SVM algorithm is to create the best line or decision boundary that can
segregate n-dimensional space into classes so that we can easily put the new data point in
the correct category in the future. This best decision boundary is called a hyperplane.
● SVM chooses the extreme points/vectors that help in creating the hyperplane. These
extreme cases are called as support vectors, and hence algorithm is termed as Support
Vector Machine.
● SVM algorithm can be used for Face detection, image classification, text categorization,
etc.
SUPPORT VECTOR MACHINES
Consider the below diagram in which there are two different categories
that are classified using a decision boundary or hyperplane:
TYPES OF SVM
SVM can be of two types:

Linear SVM: Linear SVM is used for linearly separable data, which means if a dataset can be classified into two
classes by using a single straight line, then such data is termed as linearly separable data, and classifier is used
called as Linear SVM classifier.

Non-linear SVM: Non-Linear SVM is used for non-linearly separated data, which means if a dataset cannot be
classified by using a straight line, then such data is termed as non-linear data and classifier used is called as Non-
linear SVM classifier.
CONFIDENCE INTERVALS
Hyperplane:

There can be multiple lines/decision boundaries to segregate the classes in n-dimensional space, but we need to find out the
best decision boundary that helps to classify the data points. This best boundary is known as the hyperplane of SVM.

The dimensions of the hyperplane depend on the features present in the dataset, which means if there are 2 features (as shown
in image), then hyperplane will be a straight line. And if there are 3 features, then hyperplane will be a 2-dimension plane.

We always create a hyperplane that has a maximum margin, which means the maximum distance between the data points.

Support Vectors:

The data points or vectors that are the closest to the hyperplane and which affect the position of the hyperplane are termed as
Support Vector. Since these vectors support the hyperplane, hence called a Support vector.
HOW DOES SVM WORKS?
1. Linear SVM:

The working of the SVM algorithm can be understood by using an example. Suppose we have a dataset that has
two tags (green and blue), and the dataset has two features x1 and x2. We want a classifier that can classify the
pair(x1, x2) of coordinates in either green or blue.

Consider the below image:

If data is linearly arranged, then we can separate it by using a straight line, but for non-
linear data, we cannot draw a single straight line. Consider the below image:
So to separate these data points, we need to add one more dimension. For linear data, we
have used two dimensions x and y, so for non-linear data, we will add a third dimension z.
It can be calculated as:

Z=χ^2+γ^2

By adding the third dimension, the sample space will become as below image:
So now, SVM will divide the datasets into classes in the following way. Consider the
below image:
SVM KERNEL FUNCTIONS
SVM algorithms use a set of mathematical functions that are defined as the kernel.
The function of kernel is to take data as input and transform it into the required form.
Different SVM algorithms use different types of kernel functions. These functions can be
different types. For example linear, nonlinear, polynomial, radial basis function (RBF), and
sigmoid.
The most used type of kernel function is RBF. Because it has localized and finite response
along the entire x-axis.
The kernel functions return the inner product between two points in a suitable feature space.
Thus by defining a notion of similarity, with little computational cost even in very high-
dimensional spaces.
 TYPES OF KERNELS
1. Polynomial kernel
It is popular in image processing.
Equation is:

where d is the degree of the polynomial.

2. Gaussian kernel
It is a general-purpose kernel; used when there is no prior knowledge about the data.
Equation is:
3. Gaussian radial basis function (RBF)

It is a general-purpose kernel; used when there is no prior knowledge about the data.
Equation is:

4. Hyperbolic tangent kernel

We can use it in neural networks.

for some (not every) k>0 and c<0.

We can use it as the proxy for neural networks.

Equation is:
PROPERTIES OF SVM
1. SVM works relatively well when there is a clear margin of separation
between classes.
2. SVM is more effective in high dimensional spaces.
3. SVM is effective in cases where the number of dimensions is greater than
the number of samples.
4. SVM is relatively memory efficient
ISSUES IN SVM
1. SVM algorithm is not suitable for large data sets.
2. SVM does not perform very well when the data set has more noise i.e. target classes
are overlapping.
3. In cases where the number of features for each data point exceeds the number of
training data samples, the SVM will underperform.
4. As the support vector classifier works by putting data points, above and below the
classifying hyperplane there is no probabilistic explanation for the classification.