1

200052 INTRODUCTION TO ECONOMIC METHODS

SUMMARY NOTES - WEEK 3

Required Reading:
Ref. File 4: Sections 4.3, 4.4, 4.6

4. PROBABILITY THEORY CONTINUED

4.4 Discrete Bivariate Probability Distributions

Definitions (Joint and Marginal Probabilities)

Suppose a statistical experiment for which simple events
take the form of intersections of outcomes with respect to
two or more variables. For such a statistical experiment:

 The probabilities of the simple events are referred to

as joint probabilities
 The probabilities of events representing outcomes
with respect to one of the variables only are called
marginal probabilities.
 A listing or other representation of the joint
probabilities is called a joint probability distribution.

Marginal probabilities are also referred to as

‘unconditional’ probabilities.
 2

Example 4.5:
Suppose we have the following data on all 1950 first year
students at a particular university.

Work Status
Age in Not Part- Full- Row
Years Working Time Time Total
Under 25 1200 200 250 1650
25 - 34 100 75 100 275
35 or over 10 5 10 25
Column 1310 280 360 1950
Total

Consider the experiment of selecting one of the students at

random. Define the following events for the experiment:

A: Under 25
B: 25 - 34
C: 35 or over
D: Not working
E: Part-time worker
F: Full-time worker

Calculate the following probabilities:

P ( A) , P (C) , P ( D) , P ( D  A) , P (C  F) , P ( C ) , P ( C  E)

(See video for solution)

 3

4.5 Useful Counting Techniques

The solution of probability problems often requires

enumeration of all the possible simple events of an
experiment as well as knowledge of the numbers of simple
events belonging to particular events of interest. There
are a number of rules which facilitate this enumeration by,
for example, allowing us to avoid actually listing all
possible simple events.
 4

(a) The Multiplicative Rule

The most fundamental counting rule is the multiplicative

rule. Several other rules are derived from this rule.

Theorem (Multiplicative Rule of Counting)

Suppose two sets of elements, sets A and B, consist of n A
and n B distinct elements, respectively: n A and n B need not
be equal. Then it is possible to form n A  n B distinct pairs
of elements consisting of one element from set A and one
element from set B, without regard to order within a pair.

Example 4.6:
If a take-away food store sells 10 different food items and
5 different types of drink, 5  10  50 distinct food/drink
pairs are possible.

The multiplicative rule can be extended naturally. Thus

n1n 2 ...n k different sets of ‘k’ elements are possible if one
selects an element from each of ‘k’ groups consisting of
n1 , n 2 ,...,n k distinct elements, respectively.

Example 4.7:
Suppose we select 5 people at random. What is the
probability that they were born on different days of the
week, assuming an individual has an equal probability of
being born on any of the seven days of the week?
(Approx. 0.1499)
 5

A simple event here is an ordered sequence of 5 elements, the

first representing the day of the week the first person was born
on, the second the day the second person was born on, and so
forth.

The sample space of this experiment contains

7  7  7  7  7  (7) 5 possible simple events.

The number of simple events in the event of interest is

7  6  5  4  3.

(Since the set from which the first element of the 5-tuple can
be taken consists of 7 elements; the set from which the second
element can be chosen consists of 6 elements, etc.)

Therefore the probability sought equals

7.6.5.4.3 2520
  0.1499
(7) 5 16807

(b) Permutations

In many experiments, each simple event can be

represented by a sequence of elements (numbers or
symbols), with the sample space being the set of all
possible distinct ordered sequences.

Definition (Permutations)
A permutation is an ordered sequence of elements.
 6

To introduce a rule for counting permutations we must

first introduce factorial notation.

Definition (Factorial Notation)

If ‘N’ is a non-negative integer, we define:

 N!  N(N  1)(N  2).......(3)(2)(1) (‘N-factorial’)

And

 0!  1

Theorem (Number of Permutations)

The total number of possible distinct permutations
(ordered sequences) of ‘R’ elements selected (without
replacement) from ‘N’ distinct elements, denoted P , is N R

given by

N!
N P 
R
( N  R )!

Example 4.8:
Consider the numbers 1, 2, 3, 4. How many permutations
of these four numbers taken 2 at a time can be found?
(12)

4! (4)(3)(2)(1)
4 P 
2   12
2! (2)(1)
 7

Let’s list them:

(1,2) (1,3) (1,4)
(2,1) (2,3) (2,4)
(3,1) (3,2) (3,4)
(4,1) (4,2) (4,3)

(c) Combinations

In numerous circumstances we can identify the simple

events of an experiment by arrangements of elements in
which order is irrelevant

For example, if we take a random sample of 2 people from

10 people numbered 1 to 10, it is normally irrelevant in
which order the people are chosen.

Definition (Combinations)
A set of ‘R’ elements selected from a set of ‘N’ distinct
elements without regard to order is called a combination.

Theorem (Number of Combinations)

The total number of possible combinations of ‘R’ elements
selected from a set of ‘N’ distinct elements is given by.

N!
C 
N R
R! ( N  R )!

Example 4.9:
In how many ways can a committee of 4 people be chosen
from a group of 7 people? (35)
 8

7 ! 7.6.5 210
   35 ways
4!3! 3.2.1 6

(d) Permutations of ‘N’ Non-Distinct Elements

Theorem (Number of Permutations of ‘N’ Non-Distinct

Elements)
Consider a set of ‘N’ elements of which N 1 are alike, N 2
are alike,....., and N r are alike, where N i  1 ( i  1,...,r )
r
and  Ni  N . Then the number of distinct permutations
i 1
of these ‘N’ elements is given by

N!
N 1 ! N 2 !......N r !

If the above result is specialized to the case where ‘x’ is the

number of distinct arrangements (or distinct
permutations) of ‘N’ objects where ‘R’ are alike and
( N  R ) are alike, then

N!
x  C N R
R! ( N  R )!

(Thus the combinations formula has two interpretations)

 9

Example 4.10:
Say we have 3 black flags and 2 red flags. How many
distinct ways are there of arranging these flags in a row?
(10)

5! 5 (4)
  10
3!2! 2!

For example, BBBRR is one of the distinct permutations. If

we numbered the B’s and R’s arbitrarily, this permutation
could be represented, say, by B1B2 B3 R 1R 2 .

Example 4.11:
Suppose there are 6 applicants for 2 similar jobs. As the
personnel manager is too lazy he simply selects 2 of the
applicants at random and gives them each a job. What is
the probability that he selects one of the 2 best applicants,
and 1 of the four worst applicants?

(See video for solution)

 10

4.6 Conditional Probability

In certain statistical experiments we may wish to calculate

probabilities where we have more information than that
implied by the basic experimental information. For
example, if we are told that a card selected at random
from a pack of playing cards is a heart (extra
information), then the probability of the card being the
ace of hearts is not 1 52 but 1 13 .

In effect, knowing that some event has occurred (a heart

drawn) may affect the probability of some other event
occurring (an ace of hearts drawn). The knowledge that
some event has occurred effectively reduces the sample
space (e.g. from 52 simple events to 13 simple events in our
card example), provided of course that the event does not
represent the set of all simple events.

1
For example, P(ace of hearts/heart drawn) 
13

Definition (Conditional Probability)

The probability of event ‘A’ occurring given that event ‘B’
occurs, or the conditional probability of ‘A’ given ‘B’ (has
occurred) is denoted P ( A | B) . Provided P (B)  0 , this
conditional probability is defined to be

P ( A  B)
P ( A | B) 
P (B)
 11

That is, to find P ( A | B) we divide the probability of both

‘A’ and ‘B’ occurring by the probability of ‘B’ occurring.

P ( A  B) can be interpreted as the probability of

obtaining a basic outcome in ‘A’ which is also in the
reduced sample space represented by ‘B’.

P (B) can be interpreted as the probability of the reduced

sample space represented by simple events in ‘B’.

AB B
A

P ( A | B) = probability of outcome ( A  B) given that we

are in ‘B’.

If P (B)  0 , we simply say that P ( A | B) is not defined.

Note: P ( A | B) need not equal P (B | A) .

Example 4.12:
Suppose that a survey of women aged 20-30 years suggests
the following joint probability table relating to marital
status and desire to become pregnant within the next 12
months.
 12

Desire
Marital status Pregnancy No pregnancy Total
Married 0.08 0.47 0.55
Unmarried 0.02 0.43 0.45
Total 0.10 0.90 1.00

If a woman aged 20-30 years is selected at random, then

P(married and desires pregnancy)

= P(married  desires pregnancy) = 0.08

If, however, we know that the woman is married

P(desires pregnancy|married)

P(married  desires pregnancy)


P(married )
0.08

0.55
 0.1455

The multiplicative law of probability results from a simple

rearrangement of the conditional probability formula
 13

Theorem (Multiplicative Law of Probability)

Suppose events ‘A’ and ‘B’ defined on a sample space.
Then

P ( A  B)  P ( A ) P ( B | A )  P ( B) P ( A | B)

Example 4.13:
Define events ‘A’ and ‘B’ in the following way:

‘A’: A student achieves a mark of over 65% in a first year

statistics exam
‘B’: A student goes on to complete her bachelors degree.

Suppose past experience indicates

P ( A )  0.7
P (B | A )  0.88

Then

P(A  B)  P(A) P(B | A)  0.7(0.88)  0.616

Note: The multiplicative rule can be extended to cases

where we consider more than 2 events. For example if we
have the events ‘A’, ‘B’, and ‘C’.

P ( A  B  C )  P ( A) P (B | A) P (C | ( A  B))
 14

4.7 Independence of Events

Sometimes, whether an event ‘B’ has occurred or not will

have no effect on the probability of ‘A’ occurring. In this
case we say events ‘A’ and ‘B’ are independent.

Definition (Independent and Dependent Events)

Events ‘A’ and ‘B’ are said to be statistically independent if

P ( A  B)  P ( A ) P ( B)

If P ( A  B)  P ( A) P (B) , the events are said to be

statistically dependent.

Alternative Definition (Independent and Dependent

Events)
Events ‘A’ and ‘B’ are said to be statistically independent if

P ( A | B)  P ( A )

P (B | A )  P (B)

Otherwise the events are said to be statistically dependent.

Note that the above alternative definition of independence

of events ‘A’ and ‘B’ implies (and is implied by) the first
definition.
 15

Notes:
 Independent events cannot be demonstrated on a
Venn diagram. To show independence one must
actually calculate the required probabilities.
 Mutually exclusive events are dependent, since in this
case the probability of an event occurring given the
occurrence of another must be 0.

It is not always obvious whether two events are

independent, as the following example illustrates.

Example 4.14:
Consider the single die tossing experiment again and
define the following events:

‘A’: an odd number of dots results

‘B’: a number of dots greater than 2 results

Are ‘A’ and ‘B’ independent?

(See video for solution)

 16

4.8 More Useful Probability Rules

(a) The Additive Law of Probability

Theorem (Additive Law of Probability)

For two events ‘A’ and ‘B’ defined on a sample space

P ( A  B)  P ( A )  P ( B)  P ( A  B)

Diagrammatically
S

AB
B

If we did not subtract P ( A  B) , the probability of the

shaded area would be included twice.

Example 4.15:
Again suppose that for S  {1,2,3,4,5,6,7,8} :

P (1)  P ( 2)  P (3)  P (6)  0.1

P (4)  P (7)  P (8)  0.08
P (5)  0.36

with A  {1,3,5,6} , B  {2,3,4,5,8}

 17

Previously (last week) we found:

P(A)  P(1)  P(3)  P(5)  P(6)

 0.1  0.1  0.36  0.1  0.66

P(B)  P(2)  P(3)  P(4)  P(5)  P(8)

 0.1  0.1  0.08  0.36  0.08  0.72

P(A  B)  P({3,5})  P(3)  P(5)  0.1  0.36  0.46

Then

P(A  B)  P(A)  P(B)  P(A  B)

 0.66  0.72  0.46
 0.92

(b) The Complementation Rule

Theorem 4.7 (Complementation Rule)

Suppose an event ‘E’ and its complement E defined on
some sample space S. Then

P (E)  1  P ( E )

For instance, with respect to the last example:

P( B)  1  P(B)  1  0.72  0.28

 18

(b) The Law of Total Probability

“In some experiments the sample space can be

conveniently considered as the union of a set of mutually
exclusive events.” In these cases the law of total
probability can help in calculating the probabilities of
other events defined on the sample space.

Theorem (Law of Total Probability)

Suppose a sample space S and a set of ‘k’ events
E1 , E 2 ,...,Ek such that
 P (E i )  0 ( i  1,...,k )
 E i  E j   ( i  j ) (i.e. the events are mutually
exclusive)
 S  E1  E2  ...  Ek (i.e. the events are exhaustive
on S)

Then for any event ‘A’ defined on S:

P ( A )  P (E1  A )  P (E2  A )  ....  P (Ek  A )

 P (E1 ) P ( A | E1 )  P (E2 ) P ( A | E2 )  ....  P (Ek ) P ( A | Ek )
k
  P (E j ) P ( A | E j )
j 1

The law of total probability is useful in contexts such as

the application of Bayes’ rule (cf. Reference file Section
4.10 optional reading).
 19

MAIN POINTS

 In some statistical experiments the number of basic

outcomes in the sample space or event of interest can be
enumerated by using the ‘multiplicative rule’,
permutation or combination formulae, depending on
how a simple event can be represented most
appropriately.

 P ( A | B) means the probability event ‘A’ occurs given

that event ‘B’ has occurred. The conditional
probability definition is
P ( A  B)
P ( A | B) 
P (B)

 Multiplicative law of probability:

P ( A  B)  P ( A ) P ( B | A )  P ( B) P ( A | B)

 Events ‘A’ and ‘B’ are statistically independent if the

probability of ‘A’ occurring is not affected by whether
‘B’ has occurred.

 Events ‘A’ and ‘B’ are independent if

P ( A  B)  P ( A ) P (B)
or equivalently P ( A | B)  P ( A)

 Additive law of probability:

P ( A  B)  P ( A )  P ( B)  P ( A  B)

 Complementation rule:
P (E)  1  P ( E )