Week 2: Learning

Objectives
■ Conditional Probability -
Permutations/Combinations
■ At the end of Week 2 you
should be able to:
■ Understand the concept of
conditional probability
■ Apply conditional probability and event
independence for solving problems
■ Appreciate some applications of conditional
probability in computer science
■ Understand the concept of
permutations and combinations
■ Solve probability problems involving 1
 Conditional
Probability
■ The probability that an event A occurs given
that event E has already occurred, is called
the conditional probability of A given E, and is
written: P(A|E)
■ Some examples:
■ What is the probability that the die will come up 3,
given that it has come up odd?
P(die is 3 | die is odd) = 1/3

■ We roll two dice. Find the probability that one of the

dice is a 2, given that the sum is 6.
P(die is 2 | sum of 2 dice is 6) = 2/5

2
 Conditional
Probability
■ A formula to help us in the
calculations:
P( A  E )
P(A | E) = P(E ) , where P(E)
s■ 0essentially we are measuring the relative
probability
of event A with respect to the reduced space
E
■ you will also see this with P(EA) - this is
fine because P(EA) = P(AE)
■ As a consequence, if in our probability
space all number
events ofhave
elemets 
in Aprobability
equal
P( A | E ) =
of
E occurrence: 3
 Exampl
e
■ We toss a fair coin 3 times. I win
(event A) if 2 or more Hs come up,
you win (event B) if 2 or more Ts
come up.
■ What is the probability that I win
given that the first toss came up as
H?

■ What is the probability that you win

given that the first toss came up as
H? 4
 A graphical
illustration
S={HHH, HHT, HTH, HTT, THH, THT, TTH,
TTT}

A={2 or more Hs} B={2 or more

Ts} P(A  E
THH P(A | E ) =
TH ) P(E
HTH =
HHH HTT T TTT )
(because the space is equiprobable)
HHT TTH =
| A E | = 3/
E={first toss was |E
H} |
4
P( B  E )
P(B | E) = P(E )
(because
= the space is
equiprobable)
|B E| =
=
5
 The general multiplication
rule
■ We saw in week 1 the multiplication rule
for independent events: P(AB)=P(A)P(B)
■ What if the events are not independent?
P( B  A)
P( A | B) = P(B)  P(B  A) = P( A |
B)P(B)
■ P(B) must not be zero

■ We can use this formula regardless of

whether events A and B are
independent
■ Why?
■ The multiplication rule can be
extended to include 3 or more events 6
 The 2nd test for examining
event independence
■ If events A and B are independent, by
definition, the knowledge that one event
has happened does not affect the
probability of the other event happening
■ So, if A and B are independent:
■ P(A|B) = P(A) (equally P(B|A)= P(B))

■ This is our 2nd test for independence

between two events

7
 Using the new test for
independence

■ Rolling a single die. Events A={die

comes up odd}, B={die comes up
1}, C={die comes up 1 or 2}
■ A and B independent?

■ A and C independent?

■ B and C independent?

8
 Sample
answer
■ Events A and B: P(A)=3/6 P(A|B)=1, A and B
NOT independent
■ Events A and C: P(A)=3/6, P(A|C)=1/2, A and C
are independent
■ Events B and C: P(B)=1/6, P(B|C)=1/2, B and C
NOT independent

9
 Conditional probabilities in
practice
■ Conditional probability, and especially
Bayes theorem (next week), are extensively
applied to various computer science areas
■ Information retrieval (e.g. retrieve web
pages in response to some query words)
■ Automatic Classification (e.g. is this email
spam or not? is this shape a human head?)
■ Automatic translation of one language to
another
■ etc. etc. etc.

10
 Conditional probabilities in
practice
■ Probabilistic language models
■ Spelling correction: P(about tonihgt’s plans)
<<P(about tonight’s plans)
■ Sentence correction: P(I saw a van)>P(eyes
awe a fan)
■ Conditional probability =>
probability of an upcoming word
■ w1, w2, w3,….
■ P(“The sea was cold”) =P(the)P(sea|
the)P(was|the sea)P(cold|the sea was)
■ Markov chains, Unigrams, bigrams, …n-
grams
■ Bigrams: P(w_i| w1, w2, ..w_i-1) ~=P(w_i| 11
 Counting: Permutations and
combinations
■ Remember the very first formula we saw
for the classical method last week?
number of ways A can
P(A)
number occur
of ways the experiment can proceed
=
■ When the probability problems become
more complex, it becomes difficult to list
all elements of event sets
■ We need to find more general methods for
“enumerating” the number of ways that a
certain outcome can occur

12
 Counting: Permutations and
combinations
■ Permutation: an arrangement of objects
in a specific order (order matters)
■ Example: the number of ways that from a group of
10 students, we can select 3 to place in a specific
seating order
■ Example: selecting characters for a password, or
digits for a pin (pins 2345 and 2435 are different)
■ Combination: an arrangement of objects
without regard to order (order does not
matter)
■ Example: the number of ways that from a
group of 10 students, we can make
committees of 3 students 13
 Permutations: with / without
replacement (repetitions)

■ In general we can have 2 types of

permutations
■ With replacement (or with repetitions): e.g.
passwords, pins, etc. allow the same
characters/digits to be repeated
■ Without replacement (or without repetitions): e.g.
take the 20 football teams in the premier league
and calculate all possible orderings of these
teams – which team will be 1st, which 2nd, etc. - a
team can not appear twice in such ordering
■ Depending on whether replacements are 14
 Permutations with
replacement
■ When we have n things to choose
from, we have n choices every time
■ When we choose r of these, the total
number of permutations is: n x n … (r
times) = nr

15
 Permutations without
replacement
■ In this case, we have to reduce the
number of available choices every time
(once an item is used, it can not be
used again)
■ We are asked to arrange r items chosen
from n items
■ Notation: (with
P(n,r) or nno replacements), r ≤ n
Pr or
Pnr n
P(n,r)
= (n !-
r)!

16
 Counting permutations:
factorials
■ n factorial
n  (n -1) (n - 2) ... 2 1 = n!
The quantity n! is called in mathematics
n factorial.
■ by definition, 0! = 1 and also n! = n(n-1)!
■ so 0! = 1, 1! = 1, 2!=2, 3!=6, 4!=24, 5!
=120, 6!=720
.
. .10! = 3,628,800
■ numbers increase VERY steeply

17
 Permutations without
■
replacement
In how many ways can you arrange n people to sit in r
chairs?
■ No. of ways filling the 1st chair

■ 2nd chair

■ 3rd chair

■ …

■ rth chair

Chair Chair Chair Chair

1 2 3 r
P(n,r) = n x n-1 x n-2 x ... x n-r+1 = n!/(n-r)!
18
 Permutations of n items
chosen from n items (without
replacement)
■ In how many ways can you arrange 5 people
to sit in 5 chairs?
■ By applying the general formula as:
n!
P(n, n) = n!
=
= (n - n)!
0! n!
x n-1 x n-2 x ... x
P(n,n)= n 1 = n!
x 4 x 3 x 2 x

P(5,5)=
1 = 5! = 120
ways
5

19
 Exercise
s
■ Suppose you have 6 people in a group
going to the theatre, but you can select
only 4 to sit in a row together. How many
ways do you have to select the 4 people,
and arrange them in one row in the
theatre?

■ In how many ways can you choose 5

distinct numbers from 0-9 to put them
on a license plate?

20
 Sample
answers
■ We are asked to calculate the possible
permutations of r = 4 items chosen from
a total of n = 6 items, P(6,4)=6!/(6-4)!
=6!/2!=
(6x5x4x3x2x1)/(2x1)=6x5x4x3=360
■ This is essentially asking us in how many
ways can we arrange 5 numbers chosen
from 10 to put in the plate, n=10 and
r=5, P(10,5)=10!/(10- 5)! =
(10x9x8x7x6x5x4x3x2x1)/(5x4x3x2x1)=
(10x9x8x7x6)= 30,240
21
 When some items are the
same
(Permutations of indistinguishable
■ In how many ways can you arrange the
objects)
letters of the word “Mississippi”? (note
that the arrangements do not have to
make sense!!!)
■ In general, when we have n items being
permuted, and n1, n2, …, nk the number of each
of the k types of objects being involved (e.g.
the sets of letters in the example), then the
number
n of distinguishable arrangements is
given by: , where n = n1 + n2 + ...
!
n1!n2 !...nk !
+ nk
Mississippi: 11! / 4!2!4!1! =
34,650 22
 Probability problems
involving permutations
■ If a problem asks you to calculate
the probability of an event, and
not just the number of
permutations:
■ you will need to calculate a numerator
and a denominator
■ Numerator: number of ways
(permutations) in which the event of
interest can happen
■ Denominator: total number of
ways (permutations) to 23
 Exercis
e
■ You choose 4 different letters of the alphabet
and re- arrange them in a 4-letter word. What
is the probability of making the word with
letters
Sampleonly from a-m?
answer
The alphabet has 26 letters, so n=26.
The problem asks us for a probability which we will
need to calculate as:
(number of ways in which we can select 4 letters
from 13 letters (a-m))/
(number of ways we can select 4 letters from
the entire alphabet of 26 letters)
Numerator = P(13,4) and denominator =
P(26,4) So p = P(13,4)/P(26,4)
 Combinations (without
replacement)
■ Similar to permutations, but here we are
not interested in the order in which we can
arrange the items
■ Intuitively, the number of possible ways
should be less than for permutations
■ For example, the combinations of the 4
letters a, b, c, d, taken 3 at a time, are:
■ {a,b,c}, {a,b,d}, {a,c,d}, {b,c,d}

■ We will use C(n,r) to mean combinations of

r
objects taken from n objects 25
 Counting
combinations
■ The general formula
is: n
C(n,r)
= (n -! r)!
r!
■ Notice the relationship with P(n,r)?
■ to calculate C(n,r) we take P(n,r) and we
divide it with the possible ways to
rearrange the r objects (r!)
■ We can also count combinations with
replacement, but we will not cover
this here 26
 To sum up with
counting
■ The following matrix summarises the main
counting methods:
Selecting r objects out of a total of n objects
(n>=r)
Replacements Replacements “Special” cases
are allowed are not allowed you can easily
Order work out:
nr
matters C(n,n) =
Order does Try to work
not matter it out!!!
1

+ Permutations with P(n,n) =

indistinguishable objects
(“Mississippi” formula) 27
 Probability problems
involving combinations
■ If a problem asks you to calculate
the probability of an event, and
not just the number of
combinations:
■ you will need to calculate a numerator
and a denominator
■ Numerator: number of ways
(combinations) in which the event of
interest can happen
■ Denominator: total number of
ways (combinations) to 28
 Exercis
■
e
A supplier ships 20 memory chips to a purchaser, of
which 5 have some flaw. The purchaser will select 3
chips at random and test them, and he will accept
the lot if no flaws are found. In how many ways can
the supplier be saved (i.e. in how many ways can the
purchaser select the 3 chips from among the working
chips)?
Sample answer
In this problem the order in which the chips are
selected does not matter, so this is a combination
problem. In how many ways can the purchaser select
the 3 chips from among the working chips - the
working chips are 15 (20 minus the 5 problematic
ones), so we are asked to calculate C(15,3) - the final
answer should be 455
 Exercis
e
■ A supplier ships 20 memory chips to a
purchaser, of which 5 have some flaw. The
purchaser will select 3 chips at random and
test them, and he will accept the lot if no
flaws are found. What is the probability p
that the lot will be accepted?

Sample answer:
We calculated in previous slides the number of
ways in which the purchaser will select the 3
chips from the 15 working chips: C(15,3)=455.
The total umber of ways in which 3 chips can
be chosen from 20 chips is C(20,3)=1140, so 30
 Summary of
lecture
■ In Week 2 we covered:
■ Conditional probability
■ Event independence and
conditional probability
■ Probability problems using
permutations
■ Probability problems using
combinations

31