SUPPORT
VECTOR
MACHINE
(SVM)
IN-DEPTH ANALYSIS WITH APPLICATIONS, KERNELS,
AND EXAMPLES
WHAT IS SUPPORT VECTOR
MACHINE (SVM)?
• Support Vector Machine (SVM) is a supervised
machine learning algorithm used primarily for
classification, and to some extent, regression
tasks.
• It finds an optimal hyperplane that separates
data points of different classes with the
maximum margin. The data points that lie
closest to the decision boundary are called
support vectors.
HYPERPLANE AND MARGIN
• A hyperplane is a decision boundary that
separates different classes. SVM aims to
maximize the margin between the classes —
the distance between the hyperplane and the
closest data points (support vectors).
• Maximizing this margin helps in better
generalization.
TYPES OF SVM
• Linear SVM – when data is linearly separable
• Non-linear SVM – when data is not linearly
separable, requiring kernel transformation
APPLICATIONS OF SVM
• Text classification (e.g., spam detection)
• Image recognition and classification
• Bioinformatics (e.g., cancer detection)
• Handwriting recognition
• Face detection
• Stock market prediction
ADVANTAGES OF SVM
• Effective in high-dimensional spaces
• Works well with clear margin of separation
• Efficient in memory usage (only support vectors
are stored)
• Versatile with different kernel functions
DISADVANTAGES OF SVM
• Not suitable for large datasets (training is slow)
• Difficult to choose the right kernel function
• Not effective when classes overlap significantly
• Hard to interpret model outputs compared to
decision trees
CONCEPT OF KERNELS IN SVM
• Kernels are mathematical functions that allow
the SVM to operate in a high-dimensional space
without explicitly transforming the data.
• This trick is called the 'kernel trick' and is useful
when data is not linearly separable.
TYPES OF KERNEL FUNCTIONS
• Linear Kernel: K(x, y) = x · y
• Polynomial Kernel: K(x, y) = (x · y + c)^d
• RBF/Gaussian Kernel: K(x, y) = exp(-γ ||x - y||²)
• Sigmoid Kernel: K(x, y) = tanh(α x · y + c)
CODE EXAMPLE - LINEAR SVM
(PYTHON)
• from sklearn import datasets
• from sklearn.svm import SVC
• from sklearn.model_selection import train_test_split
• iris = datasets.load_iris()
• X, y = iris.data, iris.target
• X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3)
• model = SVC(kernel='linear')
• model.fit(X_train, y_train)
• print(model.score(X_test, y_test))
CODE REDUNDANCY IN SVM
• Using libraries like scikit-learn reduces code
redundancy significantly by abstracting away
mathematical computations.
• Different kernels can be applied by just
changing a single line of code (e.g.,
kernel='rbf', kernel='poly').
SOLVING NON-LINEARLY
SEPARABLE PROBLEMS
• To solve problems where classes are not linearly
separable, use kernel functions like RBF or
Polynomial kernels.
• These functions transform the input space to a
higher-dimensional space where the data
becomes linearly separable.
EXAMPLE – NON-LINEAR
KERNEL (RBF)
• from sklearn.svm import SVC
• model = SVC(kernel='rbf', gamma=0.5)
• model.fit(X_train, y_train)
• print(model.score(X_test, y_test))
