Scribd
Upload
5 views

Naive_Bayes_Classifier

Naïve Bayes is a probabilistic classifier based on Bayes' Theorem, assuming feature independence, and is effective in applications like spam filtering and sentiment analysis. It includes variants such as Gaussian, Multinomial, and Bernoulli Naïve Bayes tailored for different data types. While it is simple and performs well on small and high-dimensional datasets, it has limitations due to its independence assumption and sensitivity to imbalanced data.

Uploaded by

oishis2004
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd
5 views

Naive_Bayes_Classifier

Naïve Bayes is a probabilistic classifier based on Bayes' Theorem, assuming feature independence, and is effective in applications like spam filtering and sentiment analysis. It includes variants such as Gaussian, Multinomial, and Bernoulli Naïve Bayes tailored for different data types. While it is simple and performs well on small and high-dimensional datasets, it has limitations due to its independence assumption and sensitivity to imbalanced data.

Uploaded by

oishis2004
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd
You are on page 1/ 3

Naïve Bayes Classifier

Introduction
Naïve Bayes is a probabilistic classifier based on Bayes' Theorem. It assumes that the
features are conditionally independent of each other, which simplifies computation. Despite
its simplicity, Naïve Bayes performs well in many real-world applications, such as spam
filtering, sentiment analysis, and document classification.

1. Bayes’ Theorem
Bayes' Theorem forms the foundation of the Naïve Bayes classifier. It is expressed as:

P(A|B) = [P(B|A) * P(A)] / P(B)

Where:
- P(A|B): Posterior probability of A given B.
- P(B|A): Likelihood of B given A.
- P(A): Prior probability of A.
- P(B): Probability of B.

2. Types of Naïve Bayes Classifiers


Naïve Bayes has different variants tailored to specific data types:

 - **Gaussian Naïve Bayes**: Assumes that the data follows a Gaussian (normal)
distribution.
 - **Multinomial Naïve Bayes**: Suitable for discrete data, commonly used for text
classification.
 - **Bernoulli Naïve Bayes**: Designed for binary/boolean data.

3. Example of Naïve Bayes Classifier


Consider a spam email classification problem. The task is to classify an email as spam or not
spam based on the presence of certain words.

Dataset:
| Email | Word1 (offer) | Word2 (win) | Word3 (lottery) | Spam |
|-----------|---------------|-------------|-----------------|------|
| Email1 | 1 |1 |0 | Yes |
| Email2 | 0 |1 |1 | Yes |
| Email3 | 1 |0 |0 | No |
| Email4 | 0 |0 |1 | No |

Step-by-step classification for a new email: {offer=1, win=1, lottery=0}:

1. Calculate prior probabilities:


 - P(Spam) = 2/4 = 0.5
 - P(Not Spam) = 2/4 = 0.5

2. Calculate likelihoods:

 - P(offer=1 | Spam) = 1/2 = 0.5


 - P(win=1 | Spam) = 2/2 = 1.0
 - P(lottery=0 | Spam) = 1/2 = 0.5
 - P(offer=1 | Not Spam) = 1/2 = 0.5
 - P(win=1 | Not Spam) = 0/2 = 0.0
 - P(lottery=0 | Not Spam) = 1/2 = 0.5

3. Apply Bayes' Theorem to calculate posterior probabilities:

- P(Spam | Data) = P(Data | Spam) * P(Spam)

- P(Not Spam | Data) = P(Data | Not Spam) * P(Not Spam)

4. Compare posterior probabilities and classify:

If P(Spam | Data) > P(Not Spam | Data), classify as Spam; otherwise, classify as Not Spam.

4. Advantages and Disadvantages


**Advantages:**

 - Simple and easy to implement.


 - Performs well on small datasets.
 - Works well for high-dimensional data.

**Disadvantages:**

 - Assumes independence between features, which is not always true.


 - Sensitive to imbalanced data.

5. Applications
 - Spam email detection.
 - Sentiment analysis.
 - Medical diagnosis.
 - Document classification.

6. Conclusion
Naïve Bayes is a powerful yet simple classifier widely used in various domains. Its
effectiveness lies in its probabilistic foundation and ability to handle high-dimensional data.
Although its independence assumption may not always hold, it remains a popular choice for
many real-world applications.
References
1. https://scikit-learn.org/stable/modules/naive_bayes.html
2. https://www.geeksforgeeks.org/naive-bayes-classifiers/
3. https://en.wikipedia.org/wiki/Naive_Bayes_classifier

You might also like