Scribd
Upload
12 views21 pages

Lecture 12 Dr. Lamiaa

Bayesian classification utilizes Bayes' Theorem to make decisions under uncertainty, with the Naive Bayes classifier being a popular implementation that assumes feature independence. Variants include Gaussian, Multinomial, and Bernoulli Naive Bayes, each suited for different types of data. While it offers advantages like simplicity and effectiveness in text classification, it also has limitations such as the assumption of feature independence and potential poor performance with correlated features.

Uploaded by

amrkamoun2012
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
12 views21 pages

Lecture 12 Dr. Lamiaa

Bayesian classification utilizes Bayes' Theorem to make decisions under uncertainty, with the Naive Bayes classifier being a popular implementation that assumes feature independence. Variants include Gaussian, Multinomial, and Bernoulli Naive Bayes, each suited for different types of data. While it offers advantages like simplicity and effectiveness in text classification, it also has limitations such as the assumption of feature independence and potential poor performance with correlated features.

Uploaded by

amrkamoun2012
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 21

Arab Academy for Science, Technology & Maritime Transport

College of Engineering & Technology


Computer Engineering Department
Bayesian classification
Bayesian classification:

Bayesian classification is a probabilistic approach based on Bayes' Theorem, which provides a principled
way of making decisions under uncertainty.
Understanding Bayes' Theorem:

Bayes' Theorem. Named after Reverend Thomas Bayes, this theorem describes the probability of an
event, based on prior knowledge of conditions that might be related to the event. Mathematically,
it's expressed as:

• P(A∣B) is the posterior probability: the probability of hypothesis A given data B.


• P(B|A)P(B∣A) is the likelihood: the probability of data B given hypothesis A.
• P(A)P(A) is the prior probability: the initial probability of hypothesis A.
• P(B)P(B) is the marginal likelihood: the total probability of data B.
Naive Bayes Classifier:

•A popular implementation of Bayesian classification is the Naive Bayes classifier.


•It is called "naive" because it makes a simplifying assumption: features are conditionally independent
given the class.
•Naive Bayes often performs surprisingly well in practice, especially for text classification problems
like
• spam detection
• sentiment analysis which is the process of analyzing digital text to determine if the emotional tone
of the message is positive, negative, or neutral.
The most popular types differ based on the distributions of the feature values. Some of
these include:

•Gaussian Naïve Bayes (GaussianNB):


This is a variant of the Naïve Bayes classifier, which is used with Gaussian distributions—
i.e. normal distributions—and continuous variables. This model is fitted by finding the mean
and standard deviation of each class.

•Multinomial Naïve Bayes (MultinomialNB):


This type of Naïve Bayes classifier assumes that the features are from multinomial
distributions. This variant is useful when using discrete data, such as frequency counts, and
it is typically applied within natural language processing use cases, like spam
classification.

•Bernoulli Naïve Bayes (BernoulliNB):


This is another variant of the Naïve Bayes classifier, which is used with Boolean
variables—that is, variables with two values, such as True and False or 1 and 0.
Advantages of Naive Bayes
• Simple and fast to train
• Effective with high-dimensional data
• Good performance in text classification

Disadvantages of Naive Bayes


• Assumption of feature independence
• Poor performance with highly correlated features
• Zero probability issue
Assumption of Naive Bayes

The fundamental Naive Bayes assumption is that each feature makes an:

•Feature independence: The features of the data are conditionally independent of each
other, given the class label.

•Continuous features are normally distributed: If a feature is continuous, then it is


assumed to be normally distributed within each class.

•Discrete features have multinomial distributions: If a feature is discrete, then it is


assumed to have a multinomial distribution within each class.

•Features are equally important: All features are assumed to contribute equally to the
prediction of the class label.

•No missing data: The data should not contain any missing values.
Example 1
Example 1: Text Classification with Naive Bayes
Consider a spam detection problem where we classify emails as 'spam' or 'not spam'. Our
features are words in the email, and the classes are 'spam' (S) and 'not spam' (¬S).
Example 2
X = (Rainy, Hot, High, False)
y = No
today = (Sunny, Hot, Normal, False)
Evaluating your Naïve Bayes classifier

•Accuracy: Proportion of correctly classified emails.


•Precision: Proportion of true positives among predicted positives.
•Recall: Proportion of true positives among actual positives.
•F1-score: Harmonic mean of precision and recall.

You might also like