UNIT – IV “Support Vector Machine”

What is Support Vector Machine (SVM)?

SVM is a supervised machine learning algorithm used for:

 Classification (main use)

 Regression (less common)

Its main goal is to find the best boundary (called a hyperplane) that separates data into different
classes.

Real-life Example:

Imagine you're a teacher and you want to separate students into two groups:

 ]L□ Students who passed

 ]
L
. Students who failed
You have data like:
 Number of hours studied
 Number of assignments completed
SVM will help you draw a line (or a curve) that best separates these two groups, based on this data.

1
 How does it work?
 It finds the best boundary (hyperplane) between classes line.
 It chooses the line that is farthest from both classes (so it's more confident).
 The closest points to this line are called Support Vectors.

Important Terms

Hyperplane The decision boundary that

separates the two classes

The closest data points to the separating

Support Vectors
line

Space between the

separating line and the
Margin nearest points from both
classes

What if data is not linearly separable?

Let’s say:
 The pass/fail pattern is in circles or curves, not straight lines.
Then SVM uses a kernel trick:-
 It transforms data into a higher dimension to make it separable.

2
 Types of SVM:

Type Description
Linear SVM Used when data is linearly separable
Non-Linear SVM Uses kernel trick for curved boundaries

✓ Advantages of SVM
⬛
 Works well for clear margin of separation between classes.
 Effective in high-dimensional spaces.
 Memory efficient – uses only support vectors.
 Can handle non-linear data using kernel tricks.
 Good for both classification and regression tasks.

+ Disadvantages of SVM
 Slow training on large datasets.
 Hard to choose the right kernel sometimes.
 Performance drops if there’s too much noise in the data.
 Not suitable for very large or real-time problems.
 Less effective when classes overlap a lot.

What is a Maximum Margin Classifier?

It is the basic idea behind SVM.
It tries to draw a line (or hyperplane) between two classes with the largest possible distance
(margin) from the nearest data points of both classes.

‘ Super Simple Real-Life Example:

¡_˙
Classroom Bench Example
You're a teacher in a classroom.
You have:
 ˜ Boys sitting on the left side
;
"
7
 ;i˙ Girls sitting on the right side
You want to place a long stick (chalk line on the floor) between them so that:
 No one crosses over
 The line is as far away as possible from both sides (so it’s safe and fair)
 The closest boy and girl to the line are your support vectors
That chalk line is your “Maximum Margin Classifier”

Why this chalk line is important?

 It gives you the best separation between boys and girls
 It avoids arguments (fair distance from both sides)
 It helps you decide who belongs to which group if a new student enters and sits somewhere

3
 Why Maximize the Margin?
 Better Generalization: A large margin means the model will perform better on new/unseen data.
 More Robust: Less sensitive to small changes or noise in the data

How Does it Work? (Step-by-step logic):

1. You have two classes of data (like Pass vs Fail).
2. You try to draw many possible lines to separate them.
3. SVM (or maximum margin classifier) chooses the line with the widest empty space (margin)
between the two groups.
4. Only the closest points to the line are used – these are called support vectors.
5. The final line is drawn exactly in the middle of these closest points, with maximum margin.

Limitations of Maximum Margin Classifier:

 It only works well when data is perfectly separable.
 It fails when there is overlapping or noisy data.
 That’s why we use Soft Margin SVM or Kernel SVM in practical cases.

Summary in One Line:

Maximum Margin Classifier finds the decision boundary that separates two classes with the
widest possible margin, ensuring better generalization and confidence.

What is Classification using a Separating Hyperplane?

It simply means:
yS Using a line or surface to divide data into different categories or classes.
)
When we have two classes, the goal is to:

 Draw a line (in 2D), a plane (in 3D), or a hyperplane (in higher dimensions)
 So that each class lies on different sides of this dividing boundary

Real-Life Example:
M Apple vs *
`
'
• . Banana Sorting
Imagine you're working in a fruit sorting machine:
 It takes features like weight and color of the fruit.
 You want to classify:
o M̀• Apple
'
o . * Banana
You collect data, and plot each fruit on a graph:
 X-axis: color score
 Y-axis: weight
Now, you can draw a line that separates:
 M Apples on one side
•
'
`
 .* Bananas on the other side
✓ That line is your separating hyperplane (in 2D it's just a line).
⬛

4
 Term Meaning
Classification Assigning data into different categories (like fruit type)
A fancy word for the "dividing boundary" – line in 2D, plane in
Hyperplane
3D, etc.
Separating
A line or surface that splits the classes perfectly
Hyperplane

⬛ Summary
✓
 Classification ka matlab hai cheezon ko alag-alag group mein baantna.
 Separating hyperplane ek line ya surface hoti hai jo do classes ko alag karti hai.
 2D data mein yeh line hoti hai, 3D mein plane, aur zyada features mein hyperplane.
 Iska use hota hai new data ko predict karne ke liye kis class mein aayega.
 Algorithms jaise SVM isi concept par kaam karte hain.

_
ˆ
J Conclusion
 Separating hyperplane ek powerful tool hai classification ke liye.
 Agar data clearly alag ho, toh yeh accurate prediction karta hai.
 Real life examples jaise fruit sorting ya email spam filtering mein yeh bahut useful hai.

What is Maximal Margin Classifier?

Ye ek algorithm hai jo 2 classes ke beech me ek line (ya hyperplane) draw karta hai with the
maximum margin (yaani dono classes se jitna zyada possible ho utna door).

Real Life Example (Simple & Relatable)

† School Bus Line Example

.z<
˙
 Socho tum monitor ho, aur tumhe ladkon aur ladkiyon ki school bus ke line mein alag jagah
banani hai.

5
  Tum dono ke groups ke beech ek rassi kheechte ho (rope), jahan se dono sides ke bachche door-
door rahen, par line ke bilkul paas sirf ek-ek ladka aur ladki khade hain.
● Wahi rassi hai Maximal Margin Line
’◉
;i";˜7 Jo bachche line ke sabse paas hain, woh hain “Support Vectors”
˙

Is baar ladke aur ladkiyon ki line thodi ghus ghus ke khade ho gaye hai.
 Tum perfect rassi nahi kheench sakte.
 Ab tum thoda flexible hokar rassi kheenchte ho, thoda error allow karke.
y That flexible line is what “Support Vector Classifier finds”.
S)

Important Points
Ye sirf tab kaam karta hai jab data perfectly alag ho (no overlap).
 Ye base concept hai SVM (Support Vector Machine) ka.
 Iska goal hota hai — maximum safety zone banana dono classes ke beech.

Advantages:

Simple, fast, and very accurate for clean data
 Best boundary provide karta hai for generalization
Disadvantages:
 Real-world data perfect nahi hota, isliye yeh classifier fail ho sakta hai jab data overlapping ho.
 Tab use karte hain: Soft Margin SVM or Kernel SVM

Maximal Margin Classifier ek aisi line draw karta hai jo dono classes ke beech sabse zyada doori
banaye rakhe, bina galti kiye. Yeh tabhi kaam karta hai jab classes clearly alag ho. SVM isi concept par
based hai.

What is Support Vector Classifier (SVC)?

Support Vector Classifier is an improved version of the Maximal Margin Classifier.
)y Jab data thoda sa overlapping ho, ya perfect separation na ho — tab SVC use hota hai.
S
It tries to:
 Find the best boundary (line/hyperplane)
 While allowing some mistakes (misclassifications) to handle real-world messy data

How it works (Easy Words):

1. Tries to separate the two classes as much as possible
2. Allows a few points to be on the wrong side (called slack)
3. Still maintains max margin for the rest of the points
4. Only the support vectors (nearest points) matter for the boundary

◆ Point ¸Ç Meaning
•
?
Soft Margin Thoda galti allow karta hai
Support Vectors Boundary ke sabse paas ke points
Hyperplane Boundary line ya plane that separates classes

6
 Advantages:
 Works well even if data is not perfectly separable
 Focuses only on important data points (support vectors)
 Robust and accurate for practical use
Disadvantages:
 Thoda complex hota hai samajhna
 Slow for large datasets
 Needs parameter tuning (like C, the error penalty)

Conclusion
Support Vector Classifier ek smart algorithm hai jo real-world data mein flexible boundary
banata hai, jisme thoda error allow hota hai. Ye Maximal Margin Classifier ka upgraded
version hai, aur real problems ke liye zyada useful hai.

Support Vector Machines (SVM) – Simple Explanation

c
’
) SVM Kya Hai?
SVM ek supervised machine learning algorithm hai jo mainly classification ke kaam aata hai,
lekin regression me bhi use ho sakta hai.

Iska main goal hota hai ki data points ko alag-alag categories/classes me sabse best tareeke se divide
karna.

Concept Samjho – Imagine Karo

Maan lo tumhare paas ek aisi problem hai:
 Tumhare paas students ke marks aur attendance ka data hai.
 Tumhe decide karna hai ki student pass hoga ya fail.
SVM kya karega?
 Ye tumhare data points (students) ko plot karega 2D/3D space mein.
 Phir ye ek line (ya hyperplane) banayega jo “Pass” aur “Fail” students ko sabse ache tareeke se
separate kare.

✓ Margin
⬛
Ye distance hota hai hyperplane se sabse kareeb data points (support vectors) tak ka.
SVM ka goal hota hai margin ko maximize karna – jitna zyada margin, utna accurate classification.

SVM Real-Life Examples

1. □
a Email Spam Detection
V
Y
'
 Input: Email ka content (words, frequency, etc.)
 Output: Spam ya Not Spam
 SVM use karta hai email ke features ko and banata hai boundary between spam and non-spam.

7
 2. =
* Disease Prediction
●
Y
 Input: Symptoms ka data
 Output: Disease hai ya nahi
 SVM symptoms ke basis pe classify karta hai ki patient infected hai ya healthy.
3. „˙ Customer Segmentation
 Input: Customer ke purchase behavior
 Output: High spender ya low spender
 SVM algorithm in customers ko alag groups me divide karta hai.

Visual Example – Imagine This:

Maan lo tumhare paas aise dots hain:
● = Fail
● = Pass
Agar tum ye dots 2D graph pe plot karo (x-axis = attendance, y-axis = marks), toh SVM ek aisi line draw
karega jo maximum difference ke sath red aur blue points ko alag kare.

8
 Why Use SVM?
Advantage Description
⬛ Accurate
✓ Margin maximize karta hai, isliye zyada sahi predictions deta hai.
⬛ Works in High-Dimension Agar features zyada ho (like 100s), tab bhi achha perform karta hai.
✓
⬛ Effective for Classification
✓ Especially jab classes clearly separated ho.

WHERE YOU HAVE TO NOT USE ?

 Jab data bahut noisy ho ya overlapping ho (classes alag se clear na ho).
 Jab training data bahut bada ho – SVM thoda slow ho sakta hai.

Kernel Trick – Thoda Advanced but Important

Agar data line se divide nahi ho raha (non-linear data), toh SVM kernel trick use karta hai.
Example:
Socho ek circular shape me Pass students hain aur unke around Fail students hain – toh normal line se
divide nahi hoga.
Kernel Trick 3D space me le jaake easily separate kar deta hai.

Support Vector Machine ek smart classifier hai jo best boundary dhoondta hai to separate different
classes by maximizing the margin between them.
Real-world me ye bahut use hota hai jahan accurate aur fast classification chahiye – jaise face
recognition, fraud detection, email filtering, etc.

Classification with non-linear decision boundaries

◆ Linear SVM kya karta hai?

Ek straight line (ya plane) se data ko alag karta hai.

◆ Non-linear SVM kya karta hai?

Aise cases me jab data line se alag nahi ho sakta (jaise circle ke andar aur bahar ke points), tab SVM
kernel trick ka use karta hai jaise "rbf" (Radial Basis Function) ya "poly" (Polynomial) jo data ko
higher dimension me map karta hai aur fir wahan line se separate karta hai. To 2D to 3D

Output me kya milega?

 2 circles jaisa data (andar wale = class 0, bahar wale = class 1)
 Aur ek curve (non-linear) jo unko separate karega

Jab data linearly separate nahi hota (e.g. round/circular shape), tab hum kernel='rbf' ya kernel='poly'
use karte hain.

9
 SVM kernel trick se data ko higher dimension me le jaata hai jaha wo linearly separate ho jata
hai.

10
 ★ One-Versus-One (OvO) Classification

Concept:
 Agar classes zyada hain (e.g. Class A, B, C),
to har do classes ke beech ek SVM classifier train hota hai.
 Har classifier sirf 2 classes ko differentiate karta hai.

Example:
 Suppose tumhare paas 3 classes hain:
M
`
'
• Apple, * . Banana, ? 7` Grapes
Banenge:
 Classifier 1: Apple vs Banana
 Classifier 2: Apple vs Grapes
 Classifier 3: Banana vs Grapes

Sy)Har classifier predict karega apna result.

Sy Voting ke through final prediction liya jaayega (jis class ko zyada votes milte hain wo
)
jeet jaati hai).

2. One-Versus-Rest (OvR) / One-Versus-Many Classification

Concept:
 Yahan har class ke against baaki sab classes ke liye ek SVM train hota hai.

Example:
 3 classes again: M`
• Apple, *
' . Banana, ?
7
` Grapes
Banenge:
 Classifier 1: Apple vs (Banana + Grapes)
 Classifier 2: Banana vs (Apple + Grapes)
 Classifier 3: Grapes vs (Apple + Banana)
)y Jo classifier sabse confident (highest score) hoga, wo final prediction deta hai.
S

Point One-Versus-One (OvO) One-Versus-Rest (OvR)

A separate model for every One model for each class vs
How it works?
pair of classes. all other classes.
How many
n(n-1)/2 models n models
models?
How prediction Voting decides the final The most confident model
happens? winner. gives the final answer.
When classes are few and When there are many
When to use?
accuracy is important. classes and speed is needed.

11
  OvO = "Each pair fights, voting picks the winner."
 OvR = "Each class fights all others, most confident one wins."

Extra Important Points

Scikit-Learn (Python library) automatically sambhal leta hai OvO aur OvR jab aap SVC()
lagate ho.

Aap manually bhi control kar sakte ho using:

Quick Code Exampl

from sklearn import datasets
from sklearn.model_selection import train_test_split
from sklearn.svm import SVC
from sklearn.multiclass import OneVsOneClassifier, OneVsRestClassifier

# Dataset: Iris (3 classes - Setosa, Versicolor, Virginica)

iris = datasets.load_iris()
X = iris.data
y = iris.target

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3)

# One-Versus-One (OvO)
ovo_clf = OneVsOneClassifier(SVC(kernel='linear'))
ovo_clf.fit(X_train, y_train)
print("OvO Score:", ovo_clf.score(X_test, y_test))

# One-Versus-Rest (OvR)
ovr_clf = OneVsRestClassifier(SVC(kernel='linear'))
ovr_clf.fit(X_train, y_train)
print("OvR Score:", ovr_clf.score(X_test, y_test))

OUTPUT :-
OvO Score: 0.9555555555555556
OvR Score: 0.9333333333333333

12
13