Scribd
Upload
42 views

06 MultiClass Classification

This document discusses multi-class classification techniques for categorizing data into multiple class labels. It describes two common techniques: One-vs-Rest (OvR) and One-vs-One (OvO). OvR builds one classifier per class, treating it as positive and all others as negative. OvO builds classifiers for each unique pair of classes. Both techniques break the problem down into multiple binary classification problems to handle multiple classes. OvR requires fewer models but they are larger, while OvO requires more smaller models.

Uploaded by

S Bhupendra
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd
42 views

06 MultiClass Classification

This document discusses multi-class classification techniques for categorizing data into multiple class labels. It describes two common techniques: One-vs-Rest (OvR) and One-vs-One (OvO). OvR builds one classifier per class, treating it as positive and all others as negative. OvO builds classifiers for each unique pair of classes. Both techniques break the problem down into multiple binary classification problems to handle multiple classes. OvR requires fewer models but they are larger, while OvO requires more smaller models.

Uploaded by

S Bhupendra
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd
You are on page 1/ 16

https://machinelearningmastery.

com/one-vs-rest-and-one-vs-one-for-multi-class-

classification/#:~:text=look%20at%20each.-,One%2DVs%2DRest%20for%20Multi%2DClass

%20Classification,into%20multiple%20binary%20classification%20problems.

1.
2. What is Multi-class Classification?
3. https://towardsdatascience.com/multi-
class-classification-one-vs-all-one-vs-
one-94daed32a87b

When we solve a classification problem having only two class


labels, then it becomes easy for us to filter the data, apply any
classification algorithm, train the model with filtered data, and
predict the outcomes. But when we have more than two class
instances in input train data, then it might get complex to analyze
the data, train the model, and predict relatively accurate results.
To handle these multiple class instances, we use multi-class
classification.

Multi-class classification is the classification technique that allows


us to categorize the test data into multiple class labels present in
trained data as a model prediction.

There are mainly two types of multi-class classification


techniques:-

 One vs. All (one-vs-rest)

 One vs. One

2. Binary classification vs. Multi-class


classification
Binary Classification

 Only two class instances are present in the dataset.

 It requires only one classifier model.

 Confusion Matrix is easy to derive and understand.

 Example:- Check email is spam or not, predicting gender


based on height and weight.

Multi-class Classification

 Multiple class labels are present in the dataset.

 The number of classifier models depends on the classification


technique we are applying to.

 One vs. All:- N-class instances then N binary classifier


models
 One vs. One:- N-class instances then N* (N-1)/2 binary
classifier models

 The Confusion matrix is easy to derive but complex to


understand.

 Example:- Check whether the fruit is apple, banana, or


orange.

3. One vs. All (One-vs-Rest)


In one-vs-All classification, for the N-class instances dataset, we
have to generate the N-binary classifier models. The number of
class labels present in the dataset and the number of generated
binary classifiers must be the same.
As shown in the above image, consider we have three classes, for
example, type 1 for Green, type 2 for Blue, and type 3 for Red.

Now, as I told you earlier that we have to generate the same


number of classifiers as the class labels are present in the dataset,
So we have to create three classifiers here for three respective
classes.

 Classifier 1:- [Green] vs [Red, Blue]

 Classifier 2:- [Blue] vs [Green, Red]

 Classifier 3:- [Red] vs [Blue, Green]


Now to train these three classifiers, we need to create three
training datasets. So let’s consider our primary dataset is as
follows,
Figure 5: Primary Dataset

You can see that there are three class


labels Green, Blue, and Red present in the dataset. Now we
have to create a training dataset for each class.

Here, we created the training datasets by putting +1 in the class


column for that feature value, which is aligned to that particular
class only. For the costs of the remaining features, we put -1 in the
class column.
Let’s understand it by an example,

 Consider the primary dataset, in the first row; we have x1, x2,
x3 feature values, and the corresponding class value is G, which
means these feature values belong to G class. So we put +1
value in the class column for the correspondence of green type.
Then we applied the same for the x10, x11, x12 input train data.

 For the rest of the values of the features which are not in
correspondence with the Green class, we put -1 in their class
column.
I hope that you understood the creation of training datasets.

Now, after creating a training dataset for each classifier, we


provide it to our classifier model and train the model by applying
an algorithm.
Figure 8: Photo via researchgate.net

After the training model, when we pass input test data to the
model, then that data is considered as input for all generated
classifiers. If there is any possibility that our input test data
belongs to a particular class, then the classifier created for that
class gives a positive response in the form of +1, and all other
classifier models provide an adverse reaction in the way of -1.
Similarly, binary classifier models predict the probability of
correspondence with concerning classes.

By analyzing the probability scores, we predict the result as the


class index having a maximum probability score.
Figure 9: Photo via SlidePlayer.com

 Let’s understand with one example by taking three test


features values as y1, y2, and y3, respectively.

 We passed test data to the classifier models. We got the


outcome in the form of a positive rating derived from
the Green class classifier with a probability score of (0.9).

 Again We got a positive rating from the Blue class with a


probability score of (0.4) along with a negative classification
score from the remaining Red classifier.
 Hence, based on the positive responses and decisive
probability score, we can say that our test input belongs to
the Green class.

Look at the below example of fitting multi-class Logistic


Regression model using a built-in one vs. rest
(OvR) technique.
#Import LogisticRegression() model from scikit_learn
from sklearn.datasets import make_classification
from sklearn.linear_model import LogisticRegression#define dataset
X_train, y_train = make_classification(n_samples=500,
n_features=8, n_informative=5, n_redundant=5, n_classes=4,
random_state=1)#define classification model
Multiclass_model = LogisticRegression(multi_class='ovr')#fit model
Multiclass_model.fit(X_train, y_train)#make final predictions
y_pred = model.predict(X_train)

4. One vs. One (OvO)


Figure 10: Photo via ScienceDirect.com

In One-vs-One classification, for the N-class instances dataset,


we have to generate the N* (N-1)/2 binary classifier models.
Using this classification approach, we split the primary dataset
into one dataset for each class opposite to every other class.

Taking the above example, we have a classification problem having


three types: Green, Blue, and Red (N=3).
We divide this problem into N* (N-1)/2 = 3 binary classifier
problems:

 Classifier 1: Green vs. Blue

 Classifier 2: Green vs. Red

 Classifier 3: Blue vs. Red

Each binary classifier predicts one class label. When we input the
test data to the classifier, then the model with the majority counts
is concluded as a result.

5. Conclusion
 As you got the idea behind working of One vs. All multi-
class classification, it is challenging to deal with large datasets
having many numbers of class instances.

 Because we generate that much classifiers model and to train


to those models, we create that much input training datasets
from the primary dataset.

 In One vs. One multi-class classification, we split the


primary dataset into one binary classification dataset for each
pair of classes.

That’s all folk !!

You might also like