Knowledge Representation

11/15/2021 1
 In order to solve complex AI problems, a large amount of knowledge and some
mechanisms for manipulating that knowledge both are required

 There are 2 different kind of entities:

a) facts – truths in some relevant world
b) Representation of facts in some chosen format

forward representation

mapping Reasoning Programs

Internal
facts
Representations
backward representation
mapping

English
Representations

11/15/2021 2
 Fact represented by English sentence: Tommy is a dog

 Logical representation: dog(Tommy)

 Logical representation of Another fact: all dogs have tails

∀x: dog(x) → hastail(x)

 From these above two facts the following can be inferred: hastail(Tommy)

 Applying appropriate backward mapping equivalent English sentence can be

obtained as follows: Tommy has a tail

11/15/2021 3
 The meaning of knowledge is closely related to the meaning of intelligence .
 Characteristic of intelligent people is that they possess much knowledge.
 Knowledge may be declarative or procedural.
 Heuristic, that is another special type of knowledge, used by humans to solve complex
problems.
 knowledge is defined as a set of rules & facts
 A fact is always unconditionally true.
 rule is true if some condition is satisfied. Rules have:
a) a condition part (right hand side)
b) a conclusion part (left hand side)
c) : - means if
d) a rule is read as if right hand side is true then left hand side is true, or
left hand side is true if right hand side is true.
 Example pam tom

 fact: parent (tom, bob).

 bob rule: offspring( X, Y):- parent (Y, X).

 ann
11/15/2021 4
A good knowledge representation system should possess the following properties:

 Representational Adequacy: ability to represent all kinds of knowledge

 Inferential Adequacy: ability to manipulate the representational structures to

derive new knowledge inferred from old one

 Inferential Efficiency: ability to incorporate into the knowledge structure additional

information that makes the inference most promising

 Acquisitional Efficiency: ability to acquire new information easily

11/15/2021 5
 One of the most useful forms of inference is property inference, where elements of
specific classes inherit attributes & values from more general classes in which they
are included

 In-order to achieve property inheritance, objects must be organized into classes

and classes must be arranged in a generalization hierarchy

 This type of structure is called as semantic network or slot-and-filler structure or

a collection of frames

11/15/2021 6
 handed
Person Right

Isa

Adult-
5-10
Male height

Isa

height 6-1
Baseball
-Player Bat avg.

Isa Isa
.252
Bat Avg. Bat Avg.
.106 Pitcher Fielder .262

instance
instance
team team
Team 1 A B Team 2

11/15/2021 7
 Apart from property inheritance, all the power of traditional logic is necessary for
inference

 Inference procedures implement the standard logic rules of inference

 There are many procedures, like –

i. Forward reasoning
ii. Backward reasoning
iii. Resolution

 ∀x: instance(x, Roman) → loyalto(x, Caesar) V hate(x, Caesar)

11/15/2021 8
 All the knowledge so far discussed have concentrated on relatively static facts

 Procedural knowledge can be represented in programs in many ways

 The most common way is simply as code (in PROLOG/ LISP) for doing something

11/15/2021 9
 One particular way of representing knowledge – principles of logic

 A logic involves
▪ a language
▪ Inference rules for manipulating sentences
▪ Semantics for associating elements of the language with elements of some subject matter

 Propositional calculus and First Order Predicate Calculus are said to be played an
important role for representing knowledge

11/15/2021 10
 Propositions are elementary atomic sentences that may be either true or false, but
may take on no other values

 Sentences in propositional logic are determined according to the rules of

propositional syntax

 This syntax governs the combination of the basic building blocks of propositional
calculus, such as propositions and logical connectives

11/15/2021 11
 Elements of the Propositional calculus are as follows:
❑ Atoms: countably infinite set of those strings of characters that begin with a
capital letter, e.g. P, Q, R, ….., P1 and so on, including two special atoms T & F
❑ Connectives: ∨, ∧, ¬, →, ↔

 Syntax of well-formed formulas (wffs), also called sentences

❑ Any atom is a wff, e.g. P, R, Q

❑ If w1 & w2 are wffs, then so are

w1 ∨ w2, w1 ∧ w2, ¬w1, w1 → w2, w1 ↔ w2

❑ All formulas are generated from a finite number of the above mentioned
operations
❑ There are no other wffs

 Atoms in positive or negative forms are called literals

11/15/2021 12
 Meaning of a sentence is just the value true/ false, i.e. it’s an assignment of a
truth value to the sentence

 An interpretation for a sentence is an assignment of truth value to each

propositional symbol

 Example: for sentence like (P & ¬Q), one interpretation (I1) assigns T to P and F to
Q, whereas another interpretation (I2) assigns T to P and T to Q.

 Once an interpretation has been given to a sentence, its truth value can be
determined

11/15/2021 13
Rule no. True Sentences False Sentences
1. T F
2. ¬f ¬t
3. t ∧ t’ f∧a
4. t∨a a∧f
5. a→t t→f
6. f→a t↔f
7. t↔t’ f↔t

Here, t and t’ denote any two true sentences, f and f’ denote any two false
sentences, a is any sentence

11/15/2021 14
 Find the meaning of the sentence ((P∧¬Q) →R) ∨Q for some instance I

 If the interpretation I assigns T to P, F to Q & R, then the meaning of the sentence

is F

11/15/2021 15
 Satisfiable: if there is some interpretation for which the sentence is true
e.g. P is satisfiable

 Valid: if is true for every interpretation. Valid sentences are also called tautologies
e.g. P ∨ ¬P is a valid sentence

 Contradiction: if there is no interpretation for which it is true

e.g. P ∧ ¬P

 Equivalence: two sentences are equivalent if they have the same truth value under
every interpretation
e.g. P → Q and ¬P ∨ Q are equivalent with each other

 Logical Consequence: a sentence is a logical consequence of another if it is

satisfied by all the interpretations that satisfy the first
e.g. P is a logical consequence of P ∧ Q.

11/15/2021 16
 Inference rules provide the means for logical proofs or deductions

 The problem is to prove the truth of S ( the conclusion) from a given set of
sentences S’ = {s1, s2, …., sn} ( Premises).

 Modus Ponens: From P & P → Q, infer Q

 Modus Tollens: From ¬Q and P → Q, infer ¬P.
 Chain Rule: From P → Q , Q → R, infer P → R

11/15/2021 17
 A literal is an atom in either positive form or negative form

 A clause is a set of literals

 For example, P ∨ Q ∨ ¬R is a clause, as well as a wff

 The empty clause {} is equivalent to F

 Resolution rule for Propositional calculus is stated as follows:

▪ from {λ} ∪ Σ1 and {¬ λ} ∪ Σ2, Σ1 ∪ Σ2 can be inferred, which is called the
resolvent of the given two clauses

 The atom λ is the atom resolved upon and the said process is called resolution

11/15/2021 18
 Resolving R ∨ P and ¬P ∨ Q yields R ∨ Q . This can also be proved by rule of
chaining. So it can be said that chaining is a special case of resolution

 Resolving R and ¬R ∨ P yields P. This can also be proved by modus ponens. So

it can be said that modus ponens is also a special case of resolution

11/15/2021 19
 An arbitrary wff ω can be inferred from a set of clauses ∆ by resolution as
follows:
❑ Convert the wffs in ∆ to CNF

❑ Convert the negation of the wff to be proved, ω, to CNF

❑ Combine all the CNF clauses into a single set Γ

❑ Iteratively apply resolution to the clauses in Γand add the results to Γ either
until there are no more resolvents that can be added or until the empty clause
is produced

11/15/2021 20
 A wff written as a conjunction of disjunction of literals is said to be in CNF
 Let’s convert a wff ¬(P→Q) ∨ (R → P) to CNF as follows:

❑ eliminate implication using equivalent form using ∨

¬ (¬P ∨Q) ∨(¬R ∨P)

❑ Reduce the scope of ¬ by using DeMorgan’s law

(P ∧ ¬Q) ∨(¬R ∨P)

❑ Convert to CNF by using the associative & distributive laws

(P ∨¬R ∨P) ∧ (¬Q ∨¬R ∨P) ⇒ (P ∨¬R) ∧ (¬Q ∨¬R ∨P)

❑ A conjunction of clauses is usually expressed as a set of clauses (with

conjunction of the clauses implied), thus we have the following
{(P ∨¬R), (¬Q ∨¬R ∨P)}

11/15/2021 21
 Consider a robot that is able to lift a block, if the block is liftable, i.e. not too
heavy, and if the robot’s battery power source is adequate. If both of these
conditions are satisfied, then when the robot tries to lift a block it is holding, its
arm moves.
 Given set of wffs are as follows:
1. BAT_OK, 2. ¬MOVES, 3. (BAT_OK ∧LIFTABLE)→MOVES

➢ Wff to be proved: ¬ LIFTABLE

➢ From 3 we have, 4. ¬BAT_OK ∨ ¬ LIFTABLE ∨ MOVES

➢ Negation of the wff to be proved is, 5. LIFTABLE

➢ Resolution works as follows:
➢ Resolving 5 with 4 we have, 6. ¬BAT_OK ∨ MOVES
➢ Resolving 2 with 6 we have, 7. ¬BAT_OK
➢ Now finally resolving 1 with 7 we have, 8. NIL
➢ Since contradiction occurs so we can say the wff to be proved is inferred.

11/15/2021 22
 It does not permit to make generalized statements about classes of similar objects

 It lacks the structure to express relations that exist among 2 or more entities

 e.g. In propositions about toy blocks, ON_A_B and ON_B_C are completely different
with absolutely nothing in common

 A new language is required that has names for the objects about which we want to
state propositions and names for the propositions we want to state

11/15/2021 23
 The predicate calculus has symbols like object constants, relation constants and
function constants

 Object Constants: infinite set and will be strings of alphanumeric characters

begin with either a capital letter or a numeral, e.g. Aa, 13B, Q

 Function Constants: infinite set of all arities. Strings of alphanumeric characters

beginning always with a lowercase letter and superscripted
by their arity. E.g. fatherof1, distancebetween2

 Relation Constants: infinite set of all arities. Strings of alphanumeric characters

beginning always with a capital letter and superscripted
by their arity. It is also called a predicate. E.g. Parent2

11/15/2021 24
 An object constant is a term, also called as individuals

 A function constant of arity n, followed by n terms in parentheses and

separated by comma is also a term

 A function is on individuals. Functions map n tuples of individuals into

individuals

 Object constants can be treated as a function constant of arity 0

 Example: fatherof(John, Bill), quotient (10,2,5), times(4, plus(3,6)), John

11/15/2021 25
 A relation constant of arity n, followed by n terms in parentheses and separated
by commas is an atom.

 An atom is a wff

 If P & Q are two wffs, then ¬P, P ∧ Q, P V Q, P→Q, ∀x P(x), ∃x P(x) are also wffs

 Wffs are formed by applying the above rules a finite no of times

 Example: Greaterthan(7, 2), P(A, B, C)

11/15/2021 26
 Consider the blocks world where there exist objects A, B, C, and Floor
 Let’s consider two relations ON & CLEAR among these objects
 ON is a binary relation, whereas CLEAR is unary
 Let’s assume that the following configuration of blocks hold:

B
A
C

 In predicate calculus, the above configuration is represented as,

ON(B, A), ON(A, C), ON(C, Floor), CLEAR(B)

11/15/2021 27
 When an assignment of values is given to each term and to each predicate symbol
in a wff, then it is called an interpretation

 The set of objects to which assignments are made is called the domain of the
interpretation

 If the truth values for 2 different wffs are same under every interpretation, then
they are said to be equivalent

 Predicate without any argument is called a ground atom

11/15/2021 28
 Let’s suppose it is needed to mention that every object in the domain has a
certain property/ participated in a certain relation

 Let’s again suppose that we want to mention that at least one object in the
domain has a certain property

 For finite domains, it is possible to write sentences like CLEAR(B1) ∧ CLEAR(B2) ∧

CLEAR(B3) for the first case or CLEAR (B1) ∨ CLEAR (B2) ∨ CLEAR (B3) for the 2nd
case

 New syntactic entities for this purpose is required, like variable symbols and
quantifier symbols

11/15/2021 29
 Variable symbols: infinite set consisting of strings beginning with lowercase
letters. Variable symbol is also a term

 Quantifier symbols: Universal quantifier (∀) and existential quantifier (∃)

 If ω is a wff and ξ is a variable symbol, then both (∀ξ)ω and (∃ξ)ω are also wffs

 Universal quantifier: (∀ξ)ω(ξ) has the value true just in case ω(ξ) has the value
true for all assignments of the variable symbol ξ to objects in the domain

 Existential quantifier: (∃ξ)ω(ξ) has the value true just in case ω(ξ) has the value
true for at least one assignment of the variable symbol ξ to objects in the domain

 In the version of the predicate calculus, which is called FOPL, use quantification
over variable symbols

 2nd or higher order predicate calculus allow quantification over relation and
function symbols

11/15/2021 30
 Evaluate the truth value of the following expression:

E: ∀x((A(a, x) ∨ B(f(x))) ∧ C(x)) → D(x)

11/15/2021 31
 Represent the following English sentences in FOPL form:

 E1: all employees earning $1400/ more per year pay taxes

 E2: some employees are sick today

 E3: no employee earns more than the President

11/15/2021 32
 Let’s define the following relations & functions:

 E(x) for x is an employee, P(x) for x is President

 i(x) – calculates the income of x
 GE(u, v) – u>=v, S(x) - x is sick today, T(x) - x pays tax

 E1’: ∀x((E(x) & GE( i(x), $1400))→T(x))

 E2’: ∃x(E(x) ∧ S(x))
 E3’: ∀xy(( E(x) & P(y)) → ~GE(i(x), i(y)))

11/15/2021 33
 Marcus was a man.
 Marcus was a Pompeian.
 All Pompeian were Roman.
 Caesar was a ruler.
 All Romans were either loyal to Caesar or hated him.

 Everyone is loyal to someone.

 People only try to assassinate rulers they are not loyal to.
 Marcus tried to assassinate Caesar.

11/15/2021 34
 Man (Marcus).
 Pompeian (Marcus).
 ∀x: Pompeian (x) → Roman (x).
 Ruler (Caesar).
 ∀x:Roman (x) → loyalto (x, Caesar) ∨ hated (x, Caesar).
 ∀x:∃y:loyalto (x, y).
 ∀x: ∀y: Person (x) ∧ Ruler (y) ∧ tryassassinate (x, y) → ¬loyalto (x, y)
 tryassassinate (Marcus, Caesar).

11/15/2021 35
 Eliminate implication signs like propositional calculus

 Reduce scopes of negation signs as in propositional calculus

 Standardize variables so that each quantifier binds a unique variable

e.g. ∀x[¬P(x) ∨ (∃x)Q(x)] ⇒ ∀x[¬P(x) ∨ (∃y)Q(y)]

 Eliminate existential quantifiers

❑ A formula that contains an existentially quantified variable asserts that there is
a value that can be substituted for the variable to make it true

❑ If existential quantifiers occur within the scope of universal quantifier, then the
value that satisfies the predicate may depend on the values of universally
quantified variable

❑ e.g. (∀x)[(∃y)Height (x, y)] means” every person, x, has a height y”. It can be
written as by eliminating existential var, (∀x) Height (x, h(x)), where h(x) is a
skolem function that maps each value of x into the y that exists.

11/15/2021 36
 Convert to prenex form

❑ Since there are no remaining existential quantifiers, and each universal

quantifier has its own variable symbol, we can now move all the universal
quantifiers to the front of the wff

❑ The resulting wff is said to be in prenex form, where a wff consists of a string
of quantifiers called a prefix, followed by quantifier free formula, called a
matrix.
❑ e.g. (∀x)(∀y){¬P(x) ∨{[¬P(y) ∨P(f(x, y))] ∧[Q(x, h(x)) ∧¬P(h(x))]}}

 Put the matrix in CNF by repeatedly using one of the distributive rules, namely, by
replacing expressions of the form ω1 ∨ (ω2 ∧ ω5) by (ω1 ∨ ω2) ∧(ω1 ∨ ω5)

❑ When the matrix of the preceding example put in CNF, we have the following:
(∀x)(∀y){[¬P(x) ∨¬P(y) ∨P(f(x, y))] ∧[¬P(x) ∨ Q(x, h(x))] ∧[¬P(x) ∨¬P(h(x))]}

11/15/2021 37
 Eliminate universal quantifiers.

 Eliminate ∧ symbols.
❑ It is now possible to eliminate the explicit occurrence of ∧ symbols by replacing
expressions of the form (ω1 ∧ ω2) with the set of wffs {ω1, ω2}.

 Rename variable. Variable symbols may be renamed so that no variable symbol

appears in more than one clause
❑ So the clauses for the preceding example are now as follows:

¬P(x1) ∨¬P(y) ∨P(f(x1, y))

¬P(x2) ∨ Q(x2, h(x2))
¬P(x3) ∨¬P(h(x3))

11/15/2021 38
 In propositional logic, it’s easy to determine that two literals can’t be true both at
the same time

 But in FOPL, this matching process is more complicated since the args of the
predicates must be considered
 e.g. man (John) & ¬man (John) is a contradiction, whereas man (John) & ¬man (Spot) is not

 So, in order to determine contradiction, we need a matching procedure that

compares 2 literals & discovers whether there exist a set of substitution that
makes them identical

 The appropriate substitution is computed by a process called unification

 If the predicate symbols match, then the args are checked one pair at a time

 Different predicates can’t match

 A variable can match other variable, any constant, or a predicate expression,

provided the predicate expression must not contain any instances of the variable
being matched
11/15/2021 39
 Let’s suppose that wffs P(x, x) & P(y, z) to be unified. The process of substitution
works as follows:

1. 2 instances of P match

2. Now x & y are compared, required substitution is y/x, i.e. substitute y for x
and the result is P(y, y), P(y, z)

3. Now y & z are compared, and the substitution required is z/y and the result is
P(z, z), P(z, z)

4. Therefore, the unification process has succeeded with the composition of the
following 2 substitutions: (z/y) (y/x)

11/15/2021 40
 Resolve the following clauses:
1. Man (Marcus)
2. ¬Man (x1) ∨Mortal (x1)

 Solution goes like the following:

Man (Marcus) and Man (x1) can be unified by the substitution Marcus/ x1
so the resolvent is Mortal (Marcus)

11/15/2021 41
 Let’s consider the following wffs in FOPL:

 Man (Marcus).
 Pompeian (Marcus).
 ∀x: Pompeian (x) → Roman (x).
 Ruler (Caesar).
 ∀x:Roman (x) → loyalto (x, Caesar) ∨ hated (x, Caesar).
 ∀x:∃y:loyalto (x, y).
 ∀x: ∀y: Man (x) ∧ Ruler (y) ∧ tryassassinate (x, y) → ¬loyalto (x, y)
 tryassassinate (Marcus, Caesar).

 With this set of wffs as knowledge base prove the sentence “Did Marcus hate
Caesar?”
 So the goal can be written in FOPL as hate(Marcus, Caesar)

11/15/2021 42
 Converting above wffs in CNF we have the following clauses:

 1.Man (Marcus).
 2. Pompeian (Marcus).
 3. ¬Pompeian(x1) ∨ Roman(x1)
 4. Ruler (Caesar).
 5. ¬Roman(x2) ∨Loyalto(x2, Caesar) ∨Hate(x2, Caesar)
 6. Loyalto (x3, f(x3))
 7. ¬Man(x4) ∨ ¬Ruler(y1) ∨ ¬Tryassassinate(x4, y1) ∨ ¬Loyalto (x4, y1)
 8. Tryassassinate(Marcus, Caesar)

 Since the goal is also needed to be converted in FOPL and as well as in negation
form, so the revised formation of the goal will be as follows:
 9. ¬ hate(Marcus, Caesar)

11/15/2021 43
 Clause 9 Clause 5

Marcus/x2
Clause 3 ¬Roman(Marcus) ∨Loyalto(Marcus, Caesar)

Marcus/x1
¬Pompeian(Marcus) ∨Loyalto(Marcus, Caesar) Clause 2

Clause 7 Loyalto(Marcus, Caesar)

Marcus/x1, Caesar/ y1
¬Man(Marcus) ∨ ¬Ruler(Caesar) ∨ ¬Tryassassinate(Marcus, Caesar) Clause 1

Clause 4 ¬Ruler(Caesar) ∨ ¬Tryassassinate(Marcus, Caesar)

Clause 8 ¬Tryassassinate(Marcus, Caesar)

Nil
11/15/2021 44
 Let’s consider the following axioms in FOPL:

1. in_room (bananas)
2. in_room (chair)
3. in_room (monkey)
4. tall (chair)
5. ¬close (bananas, floor)
6. can_move(monkey, chair, bananas)
7. can_climb (monkey, chair)
8. ∀xy close(x, y)→can_reach(x, y)
9. ∀xy ((get_on(x, y)∧under(y, bananas) ∧tall(y))→ close(x, bananas))
10. ∀xy ((in_room(x) ∧ in_room (y) ∧in_room(z) ∧ can_move (x, y, z)) →
close(z, floor) ∨ under(y, z))
11. ∀xy can_climb(x, y) → get_on (x, y)
12. Goal: can_reach(monkey, bananas)

11/15/2021 45
 CNF representation of FOPL sentences:

1. in_room (bananas)
2. in_room (chair)
3. in_room (monkey)
4. tall (chair)
5. ¬close (bananas, floor)
6. can_move(monkey, chair, bananas)
7. can_climb (monkey, chair)
8. ¬close (x1, y1) ∨can_reach(x1, y1)
9. ¬ get_on(x2, y2) ∨ ¬ under(y2, bananas) ∨ ¬ tall(y2) ∨close(x2, bananas)
10. ¬ in_room(x3) ∨ ¬ in_room (y3) ∨ ¬ in_room(z1) ∨ ¬ can_move (x3, y3, z1) ∨
close(z1, floor) ∨under(y3, z1)
11. ¬ can_climb(x4, y4) ∨ get_on (x4, y4)

12. ¬ can_reach(monkey, bananas)

11/15/2021 46
 Clause 1, 2, 3 Clause 10
monkey/x3, chair/y3, bananas/z1
Clause 7 ¬ can_move (monkey, chair, bananas) ∨ close(bananas, floor) ∨under(chair, bananas)

close(bananas, floor) ∨under(chair, bananas) clause 6

Clause 9 under(chair, bananas)

Chair/y2
clause 5 ¬ get_on(x2, chair) ∨ ¬ tall(chair) ∨close(x2, bananas)

Clause 11 ¬ get_on(x2, chair) ∨ close(x2, bananas)

Chair/y4, x4/x2
clause 7 ¬ can_climb(x4, chair) ∨ close(x4, bananas)

Monkey/ x4
close(monkey, bananas) clause 8
monkey/x1, bananas/y1
clause 12 can_reach (monkey, bananas)

NIL

11/15/2021 47
 Given the following text “Everyone who enters in a theatre has bought a ticket.
Person who does not have money can’t buy ticket. Vinod enters a theatre.” Prove by
resolution that “Vinod buys a ticket”

11/15/2021 48
 FOPL representation of the given sentences is as follows:
1. ∀x: Enters(x, theatre)→Buyticket(x)
2. ∀x: Person(x) ∧¬Havemoney(x) → ¬Buyticket(x)
3. Enters(Vinod, theatre)
 CNF representation of the above FOPL sentences:
1. ¬ Enters(x1, theatre) ∨Buyticket(x1)
2. ¬ Person(x2) ∨ Havemoney(x2) ∨¬Buyticket(x2)
3. Enters(Vinod, theatre)

 Goal in negative form is: 4. ¬Buyticket(Vinod)

Clause 4 Clause 1
Vinod/ x1
Clause 3 ¬ Enters(Vinod, theatre)

Nil

11/15/2021 49
 An agent often has only uncertain information about its task & about its environment

 Techniques discussed so far had limited abilities for representing & reasoning about
uncertain knowledge

 Sentence like P ∨ Q allows to express uncertainty about which of P or Q is true

 But it fails to represent how certain we are about either P or Q

11/15/2021 50
 Let’s consider a collection of random variables V1, V2,…., Vk

 Joint probability that the values of V1, V2,…., Vk are v1, v2,…, vk, respectively can be
represented by the expression p(V1= v1, V2= v2, …, Vk= vk).

 If a fair coin is flipped five times, then we might have p(H, T, T, H, T) = 1/32

 Probability functions must satisfy the following properties:

❑ 0≤ p(V1, V2, …, Vk)≤1,  p(V 1,V 2,...,Vk ) = 1

11/15/2021 51
 Let’s consider 4 binary valued variables, B, M, L & G, there are 16 joint
probabilities over these variables, each of the form p(B=b, M=m, L=l, G=g)

B M L G Joint
Prob.
T T T T 0.5686
T T T F 0.0299
T T F T 0.0135
T T F F 0.0007
…. …. …. …. ….

11/15/2021 52
 When we know the values of all of the joint prob. for a set of random
variables, then we can compute what is called the marginal prob. of one of
these random var.

 E.g. p(B=true) = Σ p(B=true, M, L, G)

 p(B=b, M=m) = Σ p(B=b, M=m, L, G)

 Thus, given the full joint prob. function for a collection of random var., it’s
possible to compute all of the marginal & lower order joint prob.

 However, when the no of random vars. are too large, then the task of
specifying all of the joint prob becomes intractable.

11/15/2021 53
 For any values of the variables Vi & Vj, the conditional prob function is given
by:
p (Vi , V j )
p (Vi | V j ) =
p (V j )
Where p(Vi , Vj) is the joint prob. of Vi & Vj, and p(Vj) is the marginal prob. Of
Vj.

 So, we can have, p(Vi , Vj) = p(Vi| Vj) p(Vj)

p ( B , M )
 E.g. p ( B | M ) =
p ( M )
 Conditional prob. is thus a normalized version of a joint prob.

11/15/2021 54
 Joint prob. can be expressed in terms of a chain of conditional prob as follows:
k
p(V1 ,V2 ,...,Vk ) =  p(Vi | Vi −1 ,.....,V1 )
i =1

 e.g. p(B, L, G, M) = p(B|L, G, M)p(L|G, M)p(G|M)p(M)

 Since the way in which we order variables in a joint prob. function is

unimportant, then we can write:

 p(Vi , Vj) = p(Vi| Vj) p(Vj) = p(Vj| Vi) p(Vi) = p(Vj , Vi)
p(V j | Vi ) p(Vi )
 Then, p(Vi | V j ) =
p(V j )

 This is called Bayes’ rule

11/15/2021 55
 Consider the following joint probabilities:
p(P, Q, R) = 0.3, p(P, Q, ¬R) = 0.2, p(P, ¬Q, R) = 0.2, p(P, ¬Q, ¬R) = 0.1,
p(¬P, Q, R) = 0.05, p(¬P, Q, ¬R) = 0.1, p(¬P, ¬Q, R) = 0.05, p(¬P, ¬Q, ¬R) = 0.0

❑ Calculate p(Q| ¬R)

11/15/2021 56
 Consider the following joint probabilities:
p(P, Q, R) = 0.3, p(P, Q, ¬R) = 0.2, p(P, ¬Q, R) = 0.2, p(P, ¬Q, ¬R) = 0.1,
p(¬P, Q, R) = 0.05, p(¬P, Q, ¬R) = 0.1, p(¬P, ¬Q, R) = 0.05, p(¬P, ¬Q, ¬R) = 0.0

❑ Calculate p(Q| ¬R)

❑ p(Q| ¬R) = 0.75

11/15/2021 57
 A variable V is conditionally independent of a set of variables Vi, given a set Vj, if
p (V | Vi , V j ) = p (V | V j )
 This fact is represented by the notation I (V , Vi | V j )
 The intuition is that if I (V , Vi | V j ) , then Vi tells us nothing more about V than we
already knew by knowing Vj

 As a generalization of pair-wise independence, we say that the variables V1, V2,

…, Vk are mutually conditionally independent, given a set V, if each of the
variables is conditionally independent of all of the others, given V.
k

 Since p (V1 , V2 ,...,Vk | V ) =  p (Vi | Vi −1 ,...,V1 , V )

and since each V i is conditionally independent
i =1

of the others given V, we have the following:

k
p (V1 ,..., Vk | V ) =  p(V
i =1
i |V)

11/15/2021 58
 Conditional independencies can be conveniently represented by Bayes Networks,
also called belief networks
 A Bayes network is a DAG, where nodes are labeled by random variables
 The intuitive meaning of an arrow from a parent to a child is that the parent
directly influences the child.
 These influences are quantified by conditional probabilities
 BNs are graphical representations of joint distributions.
 In a Bayes network, each node Vi , in the graph is conditionally independent of any
subset of the nodes that are not descendants of Vi, given the parents of Vi.
 Let’s consider A(Vi) – any set of nodes that are not descendants of Vi,
P(Vi) – immediate parents of Vi
 With the above assumptions, we can say I(Vi, A(Vi)| P(Vi)), ∀ Vi in the graph
 In turn the above expression is as equivalent as p(Vi| A(Vi), P(Vi)) = p(Vi| P(Vi))

11/15/2021 59
 Let’s assume that V1, V2, …, Vk be the nodes in a Bayes network

 Given the conditional independence assumptions made by the network, we have

the following joint prob. of all of the nodes in the network:
k
p (V1 , V2, ..., Vk ) =  p(V
i =1
i | P (Vi ))

 L p(L) = 0.7

 B p(B) = 0.95

p(M|B, L) = 0.9
p(M|B, ¬L) = 0.05
p(M| ¬B, L) = 0.0
p(M| ¬B, ¬L) = 0.0

 P(G|B) = 0.95G M
P(G| ¬B) = 0.1

p( G, B, M, L) = p(G|B)p(M|B, L)p(B)p(L)

11/15/2021 60
 Considering the Bayes network of last slide, calculate
p(M|L)

11/15/2021 61
 Considering the Bayes network of last slide, calculate p(M|L)

 p(M|L) = p(M|B, L)p(B|L)+p(M|¬B, L)p(¬B|L)

= p(M|B, L)p(B)+p(M|¬B, L)p(¬B) = 0.855

11/15/2021 62
 Q p(Q) = 0.05
 p(R) = 0.01 R

p(P|R, Q) = 0.95
p(P|R, ¬Q) = 0.90 p(S|Q) = 0.95
 p(P| ¬R, Q) = 0.80 P S p(S| ¬Q) = 0.05
p(P| ¬R, ¬Q) = 0.01

 P(U|P) = 0.7 U V P(V|S) = 0.99

P(V| ¬S) = 0.1
P(U| ¬P) = 0.2

 Calculate p(Q|U).

11/15/2021 63
 From Bayes rule we have p(Q|U) = kp(U|Q)p(Q), where k=1/p(U)
 Now, p(U | Q) =  p(U | P) p( P | Q)
p

p ( P | Q) =  p ( P | R, Q) p ( R ) = p ( P | R, Q) p ( R ) + p ( P | R, Q) p (R ) = 0.80

R

 So p (P | Q) = 0.20
 p(U|Q)=p(U|P)p(P|Q)+p(U|¬P)p(¬P|Q)=0.60
 p(Q|U)=k*0.60*0.05=k*0.03------------------(1)

 Similarly, p(Q | U ) = kp(U | Q) p(Q)

 Now, p(U | Q) =  p(U | P) p( P | Q) = 0.21
p

 Therefore p(¬Q|U)=k*0.21*0.95=k*0.20 ---------(2)

 so from (1) & (2), we have k=4.35
 Now from (1), p(Q|U) = 0.13

11/15/2021 64
 bird(tweety).
 fly(X) :- bird(X).
 ?- fly(tweety).
 Yes

 A way to handle knowledge representation in real problems is to extend logic by using

certainty factors.
 In other words, replace
IF condition THEN fact
with
IF condition with certainty x THEN fact with certainty f(x)
 Uncertainty is represented explicitly and quantitatively within probability theory
 A probabilistic model describes the world in terms of a set S of possible states - the
sample space
 Since it is not possible to know the true state of the world, so we come up with a
probability distribution over S which gives the probability of any state being the true
one

11/15/2021 65
 Let’s consider the following situation:

You have a new burglar alarm installed at home. It is fairly reliable at

detecting a burglary, but also responds on occasion to minor
earthquakes. You also have two neighbors, John and Mary, who have
promised to call you at work when they hear the alarm. John quite
reliably calls when he hears the alarm, but sometimes confuses the
telephone ringing with the alarm and calls then too. Mary, on the
other hand, likes loud music and misses the alarm altogether
sometimes.

11/15/2021 66
11/15/2021 67
 A generic entry in the joint probability distribution
P(X1, …, Xn) is given by:
n
P ( X 1 , X 2 ,..., X n ) =  P ( X i | Parents( X i ))
i =1

11/15/2021 68
 Probability of the event that the alarm has
sounded but neither a burglary nor an
earthquake has occurred, and both Mary and
John calls:

11/15/2021 69
 Probability of the event that the alarm has sounded
but neither a burglary nor an earthquake has
occurred, and both Mary and John calls:

P(J ∧ M ∧ A ∧ ¬B ∧ ¬E)
= P(J | A) P(M | A) P(A | ¬B ∧ ¬E) P(¬B) P(¬E)
=0.9 X 0.7 X 0.001 X 0.999 X 0.998
=0.00062

11/15/2021 70
 Let A, B, C, D be Boolean random variables. Given that: A and B are
(absolutely) independent. C is independent of B given A. D is independent of
C given A and B.
Prob(A=T) = 0.3, Prob(B=T) = 0.6, Prob(C=T|A=T) = 0.8,
Prob(C=T|A=F) = 0.4, Prob(D=T|A=T,B=T) = 0.7, Prob(D=T|A=T,B=F) = 0.8,
Prob(D=T|A=F,B=T) = 0.1, Prob(D=T|A=F,B=F) = 0.2

 Compute the following quantities:

1) Prob(D=T)
2) Prob(D=F,C=T)
3) Prob(A=T|C=T)
4) Prob(A=T|D=F)
5) Prob(A=T,D=T|B=F).

11/15/2021 71
1. P(D=T) = P(D=T,A=T,B=T) + P(D=T,A=T,B=F) + P(D=T,A=F,B=T) +
P(D=T,A=F,B=F)
= P(D=T|A=T,B=T) P(A=T,B=T) + P(D=T|A=T,B=F) P(A=T,B=F) +
P(D=T|A=F,B=T) P(A=F,B=T) + P(D=T|A=F,B=F) P(A=F,B=F)
(since A and B are independent absolutely)
=P(D=T|A=T,B=T) P(A=T) P(B=T) + P(D=T|A=T,B=F) P(A=T) P(B=F) +
P(D=T|A=F,B=T) P(A=F) P(B=T) + P(D=T|A=F,B=F) P(A=F) P(B=F)
= 0.7*0.3*0.6 + 0.8*0.3*0.4 + 0.1*0.7*0.6 + 0.2*0.7*0.4 = 0.32

2. P(D=F,C=T) = P(D=F,C=T,A=T,B=T) + P(D=F,C=T,A=T,B=F) + P(D=F,C=T,A=F,B=T) + P(D=F,C=T,A=F,B=F)

= P(D=F,C=T|A=T,B=T) P(A=T,B=T) + P(D=F,C=T|A=T,B=F) P(A=T,B=F) + P(D=F,C=T|A=F,B=T) P(A=F,B=T) +
P(D=F,C=T|A=F,B=F) P(A=F,B=F) (since C and D are independent given A and B)
 =P(D=F|A=T,B=T) P(C=T|A=T,B=T) P(A=T,B=T) + P(D=F|A=T,B=F) P(C=T|A=T,B=F) P(A=T,B=F) +
 P(D=F|A=F,B=T) P(C=T|A=F,B=T) P(A=F,B=T) + P(D=F|A=F,B=F) P(C=T|A=F,B=F) P(A=F,B=F)
 (since C is independent of B given A and A and B are independent absolutely)
=P(D=F|A=T,B=T) P(C=T|A=T) P(A=T) P(B=T) + P(D=F|A=T,B=F) P(C=T|A=T) P(A=T) P(B=F) +
 P(D=F|A=F,B=T) P(C=T|A=F) P(A=F) P(B=T) + P(D=F|A=F,B=F) P(C=T|A=F) P(A=F) P(B=F)
= 0.3*0.8*0.3*0.6 + 0.2*0.8*0.3*0.4 + 0.9*0.4*0.7*0.6 + 0.8*0.4*0.7*0.4 = 0.3032

11/15/2021 72
 Consider the following Bayesian Network
containing 3 Boolean random variables:

Compute the following quantities:

(i) P(~B, C | A)
(ii)P(A | ~B, C)

11/15/2021 73
 (i) P (~B,C | A) = P (~B | A) P (C | A) = (0.15)(0.75) = 0.1125

 (ii)

11/15/2021 74