Unit-4

Bayesian Networks
Overview
 Recalling of Naïve Bayes
 Bayesian networks
 Making predictions
 Learning Bayesian networks
 Specific algorithms(K2, Tree Augmented Naïve Bayes (TAN),
Bayesian Multinet)
 Data structures for fast learning (an All-Dimensions (AD) tree )
Recalling of Naïve Bayes

 The fundamental Naive Bayes assumption is Fig1:-

Diabetes
that each feature makes an Independent
contribution to the outcome.
 Example: Naïve Bays work best when arrows
follow the direction of causality:-

Glucose Mass Preg Age Insulin

Recalling of Naïve Bayes cont..

 Disadvantage:- In a Naïve Bayes network, Fig1:-

Diabetes
arrows are often not in the causal direction :-
1. Diabetes does not cause pregnancies.
2. Diabetes does not cause age.

Glucose Mass Preg Age Insulin

Bayesian networks

 A Bayesian network is a graphical model for probabilistic relationships among a set of

variables.
 They are not naïve, because the network architecture is a representation of the dependence
among attributes.
 Bayesian network consists of directed acyclic graph(no cycles) (DAG) with a conditional
probability table for each node.
 It also contains nodes and arcs.
 Graph’s nodes hold attributes.
 Arcs between nodes represent their probabilistic dependencies.
 Bayesian network can be used for a wide range of tasks including prediction, anomaly
detection, diagnostics, automated insight, reasoning, time series prediction and decision
making under uncertainty
Bayesian networks cont..

 In many Bayesian networks, each node represents a Variable such as someone's height,
age or gender. A variable might be discrete, such as Gender = {Female, Male} or might
be continuous such as someone's age.
 Links are added between nodes to indicate that
1. one node directly influences the other.
2. When a link does not exist between two nodes, this does not mean that they are
completely independent, as they may be connected via other nodes. They may however
become dependent or independent depending on the evidence that is set on other nodes.
Directed Acyclic Graph

A node X is a parent of
Fig 2:- A another node Y if there is an
arrow from node X to node Y
Each node in the graph is a e.g. A is a parent of B
random variable B

c D

Informally, an arrow from

node X to node Y means X
has a direct influence on Y
Directed Acyclic Graph for Diabetes

 First, it is certain that diabetes can be  Fig 3:- Preg Age

caused by Preg (number of times
pregnant), Age(older),Mass(overweight).
Mass
 Assumptions of Bayesian Networks:-
1. All attributes are nominal (Nominal data
is categorical data that assigns numerical Diabetes
values as an attribute to an object,
animal, person or any other non-number ).
2. There are no missing values. Glucose Insulin

3. Assumed to be conditionally independent.

Important terms

 Conditional Probability  Joint probability

In probability theory, conditional probability is a measure A joint probability refers to the probability of more than one
of the probability of an event (some particular situation variable occurring together, such as the probability of A and B,
occurring) given that (by assumption, presumption, assertion denoted P(A,B).
or evidence) another event has occurred.
Ex:-
We use P(A | B) to denote the conditional probability of A
given B. Joint probability distribution for variables Raining and Windy is
shown below. For example, the probability of it being windy and
Ex:- not raining is 0.16 (or 16%).
The probability that any given person has a cough on any given
day may be only 5%. But if we know or assume that the person
has a cold, then they are much more likely to be coughing. The
conditional probability of coughing given that you have a cold
might be a much higher 75%.
Making Predictions

Rainy
day
 A simple Bayesian network.  Ex:-
 Assume all these nodes corresponds to variable which
are Boolean. Late Accident
 7 Nodes are:- wakeup
1. Late wakeup
2. Accident
3. Rainy day
Traffic
4. Traffic jam jam
Late for
5. Late for work work
6. Late for meeting
7. Meeting postponed Meeting
postponed
Late for
meeting
Making Predictions

Rainy
day
 7 Nodes or Variable.  Fig:-
 So there are (2^7)-1 combination and for each Late Accident
combination we can obtain probability table. wakeup

Traffic
jam
Late for
work

Meeting
postponed
Late for
meeting
Making Predictions

Rainy
day
 From these networks we can read different  Fig:-
conditional independence relationship.
Late Accident
wakeup

 Inference:-You know the value of some

variable you want to find out probability
distribution of some other variable which are
Traffic
of your interest. jam
Late for
work

Meeting
postponed
Late for
meeting
General Product Rule
 Sample of General Product Rule

X1

X2 X3

X5

X4 X6

p(x1, x2, x3, x4, x5, x6) = p(x6 | x5) p(x5 | x3, x2) p(x4 | x2, x1) p(x3 | x1) p(x2 | x1) p(x1)

14
 Arc Reversal - Bayes Rule
X1 X2 X1 X2

X3 X3

p(x1, x2, x3) = p(x3 | x1) p(x2 | x1) p(x1) p(x1, x2, x3) = p(x3 | x2, x1) p(x2) p( x1)

is equivalent to is equivalent to

X1 X2 X1 X2

X3 X3

p(x1, x2, x3) = p(x3, x2 | x1) p( x1)

p(x1, x2, x3) = p(x3 | x1) p(x2 , x1)
= p(x2 | x3, x1) p(x3 | x1) p( x1)
= p(x3 | x1) p(x1 | x2) p( x2)
15
Application of Bayesian networks
Specific algorithms

 Now we move on to actual algorithms for learning Bayesian networks

 All the below three are based on predictions.
1. K2
2. TAN(Tree Augmented Naïve Bayes)
3. Bayesian Multinet
K2

 It is used to avoid Overfitting (Overfitting happens when a model learns the detail and
noise in the training data to the extent that it negatively impacts the performance of the
model on new data.).