Mathematical Statistics 1

STAT 1003A

School of Statistics and Actuarial Science

Wits University

DR. HERBERT HOVE

E-mail: [email protected]

Presented by Dr. Herbert Hove

 COUNTING AND PROBABILITY

LECTURE SERIES

2|Page
Prepared by Dr. Herbert Hove
 3.5. Counting and Probability
Under the equally likely model, calculating the probability of some event, say E amounts to
𝑛(𝐸)
counting the number of outcomes in E together with those in S. That is, 𝑃(𝐸) = . This
𝑛(𝑆)

method of calculating probabilities works well for the sample spaces considered thus far since
they have been relatively small. Often, there is no need to generate the sample space of interest.
Rather, it suffices to simply count the number of outcomes. To be able to count outcomes in
events of all kinds however (including for prohibitively large sample spaces), an understanding
of some basic counting techniques is required.

Three basic counting techniques:

❖ Multiplication Principle
❖ Permutations
▪ Permutations of distinct objects
▪ Permutations with some identical objects
❖ Combinations

Results of these counting techniques/ methods are useful when (but not limited to) constructing
probability assignments on finite sample spaces.

3.5.1. Multiplication Principle

Counting problems are generally complicated, and often they must be solved subject to many
restrictions. They are best solved by breaking them down into a series of simple tasks that are
easy to count and employ known rules of combining tasks. The Multiplication Principle,
sometimes, known as the Fundamental Theorem of Counting is a first step in such a process.

The Multiplication Principle: Suppose that a composite experiment (procedure, process etc)
to be performed consists of two random experiments, namely Experiment 1 and Experiment 2.
If there are 𝑛1 outcomes of Experiment 1 and 𝑛2 outcomes of Experiment 2, then together there
are 𝑛1 𝑛2 joint outcomes of the two-stage experiment.

3|Page
Prepared by Dr. Herbert Hove
PROOF:
The Multiplication principle may be proved by enumerating all the possible outcomes of the
composite experiment. That is:

1 2 ... n2
1 (1,1) (1, 2) ... (1, n2 )
2 (2,1) (2, 2) ... (2, n2 )
. . . .
. . . .
n1 ( n1 ,1) ( n1 , 2) ... ( n1 , n2 )
where the outcome is (𝑖, 𝑗) if Experiment 1 results in its 𝑖𝑡ℎ possible outcome and Experiment
2 results in its 𝑗𝑡ℎ possible outcome. Thus, the set of possible outcomes consists of 𝑛1 rows,
each containing 𝑛2 elements. Altogether, there are 𝑛1 𝑛2 possible outcomes.

Remark: These 𝑛1 × 𝑛2 joint outcomes represent the number of elements in the Cartesian
product (ordered pairs) of 𝐺 and 𝐻 where 𝐺 = {𝐺1 , 𝐺2 , … , 𝐺𝑛1 } and 𝐻 = {𝐻1 , 𝐻2 , … , 𝐻𝑛2 }.

Considering the proof as a matter of careful counting, Experiment 1 can be done in 𝑛1 ways.
For each of these, there are 𝑛2 choices/ outcomes for Experiment 2. That is:

(1 × 𝑛2 ) + (1 × 𝑛2 )+. . . +(1 × 𝑛2 ) = 𝑛1 𝑛2
⏟
𝑛1 𝑇𝑖𝑚𝑒𝑠

Illustration: Consider and experiment where a fair 4 – sided die is rolled at the first stage and
a fair coin is tossed at the second stage. How many possible outcomes are there for this
experiment?

SOLUTION:
A tree diagram is often used in connection with the multiplication principle.

4|Page
Prepared by Dr. Herbert Hove
Stage 1: 4 ways (𝑛1 = 4) Stage 2: 2 ways (𝑛2 = 2)

Total # of outcomes: For each outcome at the 1st stage, there are two outcomes at the sec o nd stage
(1 × 2) + (1 × 2) + (1 × 2) + (1 × 2)
⇒⏟
4𝑇𝐼𝑀𝐸𝑆

= 4 × 2(𝑛1 𝑛2 )
=8

The Multiplication principle is generalised as follows:

Consider an experiment (procedure, process etc) consisting of k separate stages (sub-

experiments). Suppose that:

❖ There are 𝑛1 outcomes (results) at the first stage: 1st event can occur in 𝑛1 ways  𝑛1
choices for 1st stage

5|Page
Prepared by Dr. Herbert Hove
 ❖ For each outcome at the first stage, there are 𝑛2 outcomes at the second stage: 2nd event
can occur in 𝑛2 ways
•
•
•
❖ In general, for any sequence of possible outcomes of the first (k-1) stages, there are 𝑛𝑘
outcomes at the kth stage: kth event can occur in 𝑛𝑘 ways

In all, there are 𝑛1 × 𝑛2 ×. . .× 𝑛𝑘 = ∏𝑘𝑖=1 𝑛𝑖 joint outcomes of this composite k – stage

experiment. This is known as the “Fundamental Counting Principle”

Remark: The objects being counted are distinguishable (can be told apart) k-tuples (ordered
sequences) of the form (𝑐1 , 𝑐2 , … , 𝑐𝑘 )

Remarks:
The complication when using the Multiplication principle is determining 𝑛𝑖 , the number of
possible outcomes for each stage (sub-experiment) performed. It may be helpful to pay
particular attention to:

❖ Whether replication is permitted

❖ If other restrictions have been imposed

❖ Whether illustration of the experiment by means of a tree diagram will help

Discussion Problem: Susan banks with Standard Bank. What is the probability that Susan
chooses a 4-digit pin number for her bank card such that:

i) The pin number cannot start with zero and repetitions are allowed?
ii) The pin number cannot start with zero and repetitions are not allowed?
iii) The pin number is even, cannot start with zero and repetitions are not allowed?

6|Page
Prepared by Dr. Herbert Hove
SOLUTION:

i) There are 10 × 10 × 10 × 10 = 104 ways of selecting 4-digit pin numbers when

repetitions are allowed
There are 9 × 10 × 10 × 10 = 9 × 103 ways of selecting 4-digit pin numbers
that does not start with zero and repetitions are allowed
9 × 103
Re q uired probability = = 0.9
104

ii) There are 10 × 9 × 8 × 7 = 5040 ways of selecting 4-digit pin numbers when
repetitions are NOT allowed
There are 9 × 9 × 8 × 7 = 4536 ways of selecting 4-digit pin numbers that does
not start with zero and repetitions are NOT allowed
4536
Re q uired probability = = 0.9
5040

iii) There are 10 × 9 × 8 × 7 = 5040 ways of selecting 4-digit pin numbers when
repetitions are NOT allowed

Even pin #s that cannot start with 0 either end with 0 or with one of 2, 4, 6, 8.
Ending with 0: There are 9 × 8 × 7 × 1 = 504 ways
Ending with 2, 4, 6 or 8: There are 8 × 8 × 7 × 4 = 1792 ways
Total number of ways = 504+1792 = 2296

OR

Even pin #s that cannot start with 0 either start with an odd or an even number.
Starting with odd #: There are 5 × 8 × 7 × 5 = 1400 ways
Starting with even #: There are 4 × 8 × 7 × 4 = 896 ways
Total number of ways = 1400+896 = 2296
2296
Re q uired probability = = 0.4556
5040

7|Page
Prepared by Dr. Herbert Hove
 3.5.2. Permutations
The word “permutation” means “ordered sequence of objects”. It refers to the number of ways
of arranging a set of objects. That is, we use permutations when we are concerned about order.
The number of ways of arranging a set of objects can be determined by applying the
multiplication principle.

Example 3.5.2.1 Imagine an urn with 𝑛 distinguishable balls inside.

CASE 1 Linear Permutations: List all the 𝑛 distinguishable balls in the urn in order. The
number of choices at each step is as follows:

❖ There are 𝑛 possible choices for the ball in the 1st position
❖ Having made this choice, there are (𝑛 − 1)choices for the ball in the 2nd position, then
(𝑛 − 2) choices for the ball in the 3rd position and so forth until the last ball for which
we have only 1 choice for the ball in the nth position

Then by the multiplication principle, the number of ways in which all the 𝑛 distinguishable
balls in the urn are listed in order is 𝑛(𝑛 − 1)(𝑛 − 2) ⋅⋅⋅ (3)(2)(1). We adopt the notation 𝑛!
for this product and define, for any positive integer 𝑛 = 1,2, . ..;

𝑛! = 𝑛(𝑛 − 1)(𝑛 − 2) ⋅⋅⋅ (3)(2)(1)

Definition 3.5.2.1: For a positive integer 𝑛, 𝑛! (read n factorial) is the product of all the positive
integers less than or equal to 𝑛.

The expression 𝑛! will enter into many of our equations. Factorials grow very large quickly:
1! = 1
2! = 2 × 1 = 2
3! = 3 × 2 × 1 = 6
4! = 4 × 3 × 2 × 1 = 24
5! = 5 × 4 × 3 × 2 × 1 = 120
•
•
•

8|Page
Prepared by Dr. Herbert Hove
Some estimate of the magnitude (approximation) of 𝑛! will therefore be required when 𝑛 is
large. Stirling’s formula provides an approximation for 𝑛! in terms of elementary functions.

Stirling’s Theorem: For large 𝑛, 𝑛! ≈ 𝑛𝑛 𝑒 −𝑛 √2𝜋𝑛

PROOF: See Analysis Texts (Not examinable in this course)

CASE 2 Permutations of subsets: Suppose now that interest is in the number of arrangements
of only some (say 𝑘) of the 𝑛 distinguishable elements/ balls in an urn for which 𝑘 < 𝑛. These
are called ordered samples of size 𝑘. In how many ways can this be done?

The answer depends on whether the draws are with or without replacement.

Ordered, without replacement: As before, the number of possible choices for each ball drawn
is as follows:
❖ There are 𝑛 possible choices for the 1st ball drawn
❖ Having drawn the 1st ball, there are (𝑛 − 1) choices for the 2nd ball drawn, then (𝑛 − 2)
choices for the 3rd ball drawn and so forth until the 𝑘𝑡ℎ ball for which there are
(𝑛 − (𝑘 − 1)) = (𝑛 − 𝑘 + 1) choices

Multiplication principle: The number of ways of selecting an ordered sample of 𝑘 balls at

random from an urn with 𝑛 distinguishable balls without replacement, that is, the number of
permutations of the 𝑛 objects taken 𝑘 at a time without replacement is:

𝑛(𝑛 − 1)(𝑛 − 2) ⋅⋅⋅ (𝑛 − 𝑘 + 1)[(𝑛 − 𝑘) ⋅⋅⋅ (2)(1)]

𝑛(𝑛 − 1)(𝑛 − 2) ⋅⋅⋅ (𝑛 − 𝑘 + 1) =
(𝑛 − 𝑘) ⋅⋅⋅ (2)(1)

𝑛!
= (𝑛−𝑘)! = 𝑛𝑃𝑘

9|Page
Prepared by Dr. Herbert Hove
Definition 3.5.2.2: The number of permutations of 𝑛 distinct objects taken 𝑘 at a time without
replacement, denoted 𝑛𝑃𝑘 is

𝑛 𝑛!
𝑃𝑘 = (𝑛−𝑘)!

For Definition 3.5.2.2 to hold when 𝑘 = 𝑛, 0! = 1 by definition.

(Special case):- When 𝑘 = 𝑛, the number of permutations of the 𝑛 distinguishable objects

taken 𝑛 at a time, denoted 𝑛𝑃𝑛 is

𝑛
𝑛! 𝑛!
𝑃𝑛 = = = 𝑛!
(𝑛 − 𝑛)! 0!

since 0! = 1 by definition.

Ordered, with replacement: The number of possible choices for each ball drawn is as
follows:

❖ There are still 𝑛 possible choices for the 1st ball drawn

❖ Having drawn the 1st ball, 𝑛 choices still remain for the 2nd ball since the drawn ball is
replaced. The same applies for the 3rd ball and so on until the kth ball.

Multiplication principle: The number of ways of selecting an ordered sample of 𝑘 balls at

random from an urn with 𝑛 distinguishable balls with replacement, that is, the number of
permutations of the 𝑛 objects taken 𝑘 at a time with replacement is:

𝑛 × 𝑛 ×⋅⋅⋅× 𝑛 = 𝑛𝑘

10 | P a g e
Prepared by Dr. Herbert Hove
In summary:

n distinguishable objects

CASE 1 Linear Permutations CASE 2 Permutations of subsets

(Arranging all objects) 𝑛!

Draws without replacement Draws without replacement

𝑛 𝑛!
𝑃𝑘 = (𝑛−𝑘)! 𝑛𝑘

11 | P a g e
Prepared by Dr. Herbert Hove
Exercises:

1. Prove the identity 𝑛𝑃𝑛−1 =𝑛 𝑃𝑛

RTP 𝑛𝑃𝑛−1 =𝑛 𝑃𝑛
𝑛! 𝑛!
LHS: 𝑛𝑃𝑛−1 = (𝑛−(𝑛−1))! = = 𝑛! =𝑛 𝑃𝑛 by definition
1!

2. Four people are available to form a committee of four comprising of positions of

Chairperson, Secretary, Treasurer and Spokesperson. Each person can take up only
one position on the committee. How many possible committees can be formed?
Answer: 24

4 × 3 × 2 × 1 = 4! = 4𝑃4 = 24

3. Sipho rents seven movies to watch over two nights. He decides to watch four movies
on the first night. How many distinct sequences of four movies could he possibly
watch on the first night?

7 × 6 × 5 × 4 = 7𝑃4 = 840

4. A coin is tossed five times. How many sequences of “Heads” and “Tails” are
possible? Answer: 32

Each coin toss has two possible outcomes, “Heads” and “Tails”. Hence 𝑛 = 2. The
coin is tossed five times. Thus 𝑘 = 5. Number of sequences = 𝑛𝑘 = 25 = 32

12 | P a g e
Prepared by Dr. Herbert Hove
 3.5.3. Combinations
Permutations have applications to the problem of finding the number of ordered subsets of a
set. In many problems however, interest is in the number of subsets of a certain size (say 𝑘)
that can be selected from a larger collection of objects (say 𝑛), where order does not matter.

Thus, a combination is a technical term meaning a selection.

Example 3.5.3.1: Consider the set 𝑆 = {1,2,3,4,5} consisting of all integers 1 to 5. Observe
that S can be regarded as an urn containing 5 distinguishable balls numbered 1 to 5. How many
subsets of size 3 can be formed from the set S?

SOLUTION:
There are 5 ways to select the 1st ball, 4 ways to select the 2nd and 3 ways to select the 3rd. Thus,
there are 5*4*3 = 60 ways of selecting 3 balls from an urn with 5 balls under the following
conditions:

• Order in which the letters are selected is important

• Selection is without replacement

Observe that each subset of 3 balls will be counted 3! = 6 times in this count. For example,
assume a subset consisting of balls numbered 2, 4 and 5. Because order is important, all six
arrangements ({2,4,5}, {2,5,4}, {4,2,5}, {4,5,2}, {5,4,2}, {5,2,4}) are counted. But all these 6 are
just one selection of a group of 3 balls, namely balls numbered say 2, 4 and 5. Hence the number
of different subsets of 3 balls that can be selected from the 5 numbered balls in the urn without
regard to order is
5⋅4⋅3
= 10
3!

These are called unordered samples of size 𝑘 taken from a population (urn) with 𝑛
distinguishable objects. This form of counting plays a central role in Statistics so much so that
it has earned its own notation

13 | P a g e
Prepared by Dr. Herbert Hove
In general:
• 𝑘 objects can be selected from n objects in 𝑛(𝑛 − 1) ⋅⋅⋅ (𝑛 − (𝑘 − 1)) different ways when
order of selection is important
• But each group of 𝑘 objects is counted 𝑘! times in this count

Thus the number of groups of size 𝑘 that can be selected from the 𝑛 objects is

n(n − 1) ⋅⋅⋅ (n − (k − 1)) n(n − 1) ⋅⋅⋅ (n − (k − 1))(n − k)!

=
k! k! (n − k)!
n! n
= = ( ) (or nCk )
k! (n − k)! k

𝑛 5 5! 5!
Regarding Example 3.5.3.1, 𝑛 = 5 and 𝑘 = 3. Hence ( ) = ( ) = = = 10 as
𝑘 3 3!(5−3)! 3!2!
before.

Definition: The number of unordered subsets (groups), called a combination of n objects taken
𝑛
k at a time (without replacement) denoted by ( ), for 𝑘 ≤ 𝑛 is:
𝑘
𝑛 𝑛!
( )=
𝑘 𝑘! (𝑛 − 𝑘)!
𝑛
That is, ( )is the number of different subsets of size 𝑘 that could be selected from the set of 𝑛
𝑘
objects without replacement and without regard to order. It is read “ 𝒏 choose 𝒌 ”.

PROOF:

• Let C be the number of groups (subsets) of size 𝑘 that can be selected from the 𝑛 different
objects
• The 𝑘 objects in each group (subset) can be ordered in 𝑘𝑃𝑘 ways

• Thus 𝐶( 𝑘𝑃𝑘 ) = 𝑛𝑃𝑘  Permutation: Select, then order

𝑘! 𝑛!
⇒𝐶( )= , 0! = 0
(𝑘 − 𝑘)! (𝑛 − 𝑘)!

𝑛! 𝑛
∴𝐶= =( )
𝑘! (𝑛 − 𝑘)! 𝑘

14 | P a g e
Prepared by Dr. Herbert Hove
Exercises:
1. A company has 5 male and 4 female employees with the necessary skills for a special
project. 5 employees are randomly selected from the 9 to work on the project. Find
the probability that at least 2 male and at least 2 female employees are selected to
work on the project. Answer: 0.7937

Restriction: Males on the team

Males: 2 | 3
Females: 3 | 2
5 4 5 4
( )( ) + ( )( )
𝑅𝑒 𝑞 𝑢𝑖𝑟𝑒𝑑 𝑝𝑟𝑜𝑏𝑎𝑏𝑖𝑙𝑖𝑡𝑦 = 2 3 3 2
9
( )
5
(10)(4) + (10)(6) 100
= = = 0.7937
126 126

2. A bag contains 2 green balls, 3 orange balls and 5 red balls. If 3 balls are randomly
drawn from the bag, what is the probability that at least one orange ball is included
in the draw? Answer: 0.708333

Restriction: At least one orange ball

Orange: 1 | 2 | 3
Other: 2 | 1 | 0
3 7 3 7 3 7
( )( ) + ( )( ) + ( )( )
𝑅𝑒 𝑞 𝑢𝑖𝑟𝑒𝑑𝑝𝑟𝑜𝑏𝑎𝑏𝑖𝑙𝑖𝑡𝑦 = 1 2 2 1 3 0
10
( )
3
(3 × 21) + (3 × 7) + (1 × 1) 85
= = = 0.70833
120 120

3. The birthday problem: How large must a group of people (say, in a room) be for
there to be at least a 50% chance that two or more of them share a birthday? Answer:
𝒌 = 𝟐𝟑. When 𝒌 = 𝟓𝟕, we are almost guaranteed (99% probability) to find a
match

15 | P a g e
Prepared by Dr. Herbert Hove
The Birthday Problem is a standard problem that has earned the status of being a classic in
elementary probability courses.

General Setup
• n = 365 days in a year – ignore the concept of leap years
• k people in a room – days chosen to be birthdays of the k people in a room
• Birthdays are uniformly distributed (each person has probability 1/n for each possible
day) – equally likely
• Birthdays of different people are independent – no twins/ multiples

Why is it popular and applications

• It is true, fun and its implications contradicts our intuition considerably
• Has relevance to the security of hashing protocols – Cryptographic hash functions
• The hash of the object is some sort of “fingerprint” of an object usually designed to be
a random looking collection of bits. Changing the object ever so slightly changes the
hash pretty much apparently at random.
o Avoiding hash collisions in computing systems

The numbers (k = 23 for least a 50% chance of a match AND k = 57 to be almost guaranteed
a match) are sufficiently small to challenge intuition. We can however make sense of them if
we consider the problem with care. It maybe that this result evades our naïve expectation
because we are unknowingly pondering a different problem:

How large must the number of people in a room be for there to be at least a 50% chance that
two or more of them share a given birthday?

16 | P a g e
Prepared by Dr. Herbert Hove
Solution
Different problem (…..share a given birthday?)

❖ Let p be the probability of finding a match

❖ Consider instead the probability that there is not a match: 𝑝 = 1 − 𝑝

❖ All birthdays in a year assumed to be equally likely

➢ The 1st person has a birthday different from the given birthday with probability
𝑛−1
, and so is the 2nd, …, kth person
𝑛

𝑛−1 𝑘 364 𝑘 364 𝑘

➢ 𝑝=( ) =( ) ⇔𝑝 =1−( )
𝑛 365 365

➢ With 𝑝 = 𝑝 = 0.5, there must be k = 253 people (comparisons with a given

birthday). This is large enough to seem plausible

NB:- Interest is not in matching a given birthday, but rather matching any birthday

Actual problem (…..share a birthday?) – EXACT SOLUTION

❖ Similarly, considering the probability that there is not a match
𝑛
➢ The 1st person has a unique birthday with probability = 1 i.e. birthday can fall
𝑛

on any one of the 365 days

𝑛−1
➢ The 2nd person has a birthday different from the 1st with probability
𝑛

➢ Given that the first two birthdays were different, the 3rd person has a birthday
𝑛−2
different from those with probability
𝑛

➢ Given that the first (k – 1) birthdays were different, the kth person has a birthday
𝑛−(𝑘−1)
different from those with probability
𝑛
𝑛(𝑛−1)(𝑛−2)...(𝑛−(𝑘−1))[(𝑛−𝑘)!] 𝑛! 𝑛 𝑘!
▪ 𝑝=1− =1− =1−( ) 𝑘
𝑛𝑘 [(𝑛−𝑘)!] 𝑛𝑘 (𝑛−𝑘)! 𝑘 𝑛

365 𝑘!
▪ Substituting n = 365, 𝑝 = 1 − ( ) 𝑘
𝑘 365

Solving this equation for different values of k yields the results in Table 3.5.3.1.

17 | P a g e
Prepared by Dr. Herbert Hove
Table 3.5.3.1. Calculated probabilities of a match for different values of k
k pk
5 0.0271
10 0.1169
15 0.2529
20 0.4114
22 0.4757
23 0.5073
30 0.7063
40 0.8912
50 0.9703
60 0.9941

Actual problem (…..share a birthday?) – APPROXIMATE SOLUTION

By Stirling’s approximation,
𝑛! 𝑛𝑛 𝑒 −𝑛 √2𝜋𝑛
𝑝=1− 𝑘 ⇔ (1 − 𝑝) ≈
𝑛 (𝑛 − 𝑘)! 𝑛𝑘 (𝑛 − 𝑘)𝑛−𝑘 𝑒 −(𝑛−𝑘) √2𝜋(𝑛 − 𝑘)
1
𝑛𝑛−𝑘+2 𝑒 −𝑘
= 1
(𝑛 − 𝑘)𝑛−𝑘+2
1
𝑛𝑛−𝑘+2 𝑒 −𝑘
= 1
1 𝑘 𝑛−𝑘+2
𝑛𝑛−𝑘+2 (1 − )
𝑛
1
−𝑘
𝑘 𝑘−𝑛−2
= 𝑒 (1 − )
𝑛

Taking natural logarithms on both sides yields:

1 𝑘
ln(1 − 𝑝) ≈ −𝑘 + (𝑘 − 𝑛 − ) [ln (1 − )]
2 𝑛

From the Taylor series for logs

18 | P a g e
Prepared by Dr. Herbert Hove
 𝑥2 𝑥3
ln(1 + 𝑥) ≈ 𝑥 − + −⋅⋅⋅ for small x, it follows that (watch out for the minus sign)
2 3

1 𝑘 1 𝑘 2 1 𝑘 3
ln(1 − 𝑝) ≈ −𝑘 + (𝑘 − 𝑛 − ) [(− ) − (− ) + (− ) −⋅⋅⋅]
2 𝑛 2 𝑛 3 𝑛
1 𝑘 𝑘2 𝑘3
= −𝑘 + (𝑛 − 𝑘 + ) [ + 2 + 3 +⋅⋅⋅]
2 𝑛 2𝑛 3𝑛

Assume k is very small compared with n. Specifically, assume k is about √𝑛

⇒ √𝑛 << 𝑛

𝑘2 𝑘3 𝑘2 𝑘3 𝑘4 𝑘 𝑘2 𝑘3
𝑙𝑛(1 − 𝑝) = −𝑘 + 𝑘 + + − − − + + +
2𝑛 3𝑛2 𝑛 2𝑛2 3𝑛3 2𝑛 4𝑛2 6𝑛3
𝑘2 𝑘3 𝑘4 𝑘 𝑘2 𝑘3
=− − 2− 3+ + 2+ 3
2𝑛 6𝑛 3𝑛 2𝑛 4𝑛 6𝑛
𝑘2 𝑘 2𝑘 2 1 1 𝑘
=− [1 + + 2+ + + 2 +⋅⋅⋅]
2𝑛 3𝑛 3𝑛 𝑘 2𝑛 3𝑛

𝑘2
⇒ 𝑙𝑛(1 − 𝑝) ≈ − (First order Stirling approximation)
2𝑛
𝑘2
−2𝑛
∴𝑝 ≈1− 𝑒

𝑘2
The approximate solution 𝑝 ≈ 1 − 𝑒 −2𝑛 yields values of p that are very close to those in the
above table. Thus, we have demonstrated the truth of the Birthday problem.

Remarks:
• The number of different pairs that can be formed from a group of 23 people is
23
( ) = 253 - which is also the number of people (comparisons) required for there to
2
be at least a 50% chance that two or more of them share a given birthday
• It is the number of comparisons (not people) that matter
➢ When looking to match a given birthday, 253 comparisons would be required
to achieve a 50% chance of a match, that is, compare one person’s birthday
(given birthday) with 253 others
➢ When looking to match any birthday, 253 comparisons are still required.
However, these 253 comparisons are distributed among the 23 people with

19 | P a g e
Prepared by Dr. Herbert Hove
 23×22 23
each person compared against 22 others. That is =( ) = 253 (it is
2 2
the same whether we compare Joe to Ann or Ann to Joe – double counting)
• Exponential growth rapidly decreases the chance of picking unique items (increases
the chance of a match) and exponents are non-intuitive

3.5.4. Counting Methods with R

𝑛
The factorial 𝑛! and ( ) are computed with the respective commands factorial(n) and
𝑘
choose(n, k). When sampling k from n objects made by the urnsamples() function
however, the nsamp() function is used. The nsamp() function has the following arguments:
❖ n: number of distinguishable object in the urn
❖ k: the sample size
❖ replace, ordered: As in the urnsamples() function

Taking all the discussed considerations into account, a 2 by 2 table of possibilities is

constructed as follows:
ordered = TRUE Ordered = FALSE
(𝑛 − 1 + 𝑘)!
replace = TRUE 𝑛𝑘
𝑘! (𝑛 − 1)!

𝑛
𝑛! 𝑛 𝑛!
replace = FALSE 𝑃𝑘 = ( )=
(𝑛 − 𝑘)! 𝑘 𝑘! (𝑛 − 𝑘)!

The nsamp() function is vectorised. Thus, entering vectors instead of numbers for n, k,
replace and ordered yields a vector of corresponding answers.

Example 3.5.4.1 Compute the number of outcomes for each of the four urnsamples()
examples in Example 2.1.2.1. Recall that samples of size two were taken from an urn with
three distinguishable balls labeled 1, 2 and 3 respectively. Compare the answers to the length
of data frames generated in Example 2.1.2.1.

> nsamp(n = 3, k = 2, replace = TRUE, ordered = TRUE)

[1] 9

> nsamp(n = 3, k = 2, replace = FALSE, ordered = TRUE)

[1] 6

20 | P a g e
Prepared by Dr. Herbert Hove
> nsamp(n = 3, k = 2, replace = FALSE, ordered = FALSE)
[1] 3

> nsamp(n = 3, k = 2, replace = TRUE, ordered = FALSE)

[1] 6

nsamp() calculate number of rows in a sample space made by urnsamples()

3.6. Binomial Identities

𝑛
The quantities ( ) are in general, called binomial coefficients due to their role in the binomial
𝑘
theorem, stated as follows: For any two real numbers x and y, and any positive integer n,

(𝑥 + 𝑦)𝑛 = (𝑥 + 𝑦)(𝑥 + 𝑦) ⋅⋅⋅ (𝑥 + 𝑦)

𝑛
𝑛
= ∑ ( ) 𝑥 𝑛−𝑘 𝑦 𝑘
𝑘
𝑘=0
𝑛 𝑛 𝑛 𝑛 𝑛 𝑛
= ( ) 𝑥 𝑛 + ( ) 𝑥 𝑛−1 𝑦 + ( ) 𝑥 𝑛−2 𝑦 2 +⋅⋅⋅ + ( ) 𝑥 2 𝑦 𝑛−2 + ( ) 𝑥𝑦 𝑛−1 + ( ) 𝑦 𝑛
0 1 2 𝑛−2 𝑛−1 𝑛

Each term in the expansion is a product of 𝑥′𝑠 and 𝑦′𝑠 with an x or a y coming from each of the
𝑛
n factors (𝑥 + 𝑦). The coefficient of 𝑥 𝑛−𝑘 𝑦 𝑘 is ( ), the number of ways in which the k factors
𝑘
providing the 𝑦′𝑠 can be chosen.

Illustration:
The expansion (𝑥 + 𝑦)3 = ∑3𝑘=0 (𝑛) 𝑥 𝑛−𝑘 𝑦 𝑘 = (3) 𝑥 3 + (3) 𝑥 2 𝑦 + (3) 𝑥𝑦 2 + (3) 𝑦 3
𝑘 0 1 2 3
3
yields terms of the form 𝑥 3 , 𝑥 2 𝑦, 𝑥𝑦 2 and 𝑦 3 . The coefficient of 𝑥𝑦 2 for example is ( ) = 3,
2
the number of ways in which the 2 factors providing the 𝑦′𝑠 can be chosen. Thus, the
𝑛
relationship of the quantities ( ) with binomial expansions is that they are the coefficients.
𝑘

21 | P a g e
Prepared by Dr. Herbert Hove
Exercises:
Prove the following results (Theorems) which simplify the calculation of binomial coefficients:

𝑛 𝑛
1. ( )=( ) ⇒ Equidistant Relationship (from the central term(s))
𝑘 𝑛−𝑘
4 4 4 4 4
e.g. (𝑎 + 𝑏)4 = ( ) 𝑎4 + ( ) 𝑎3 𝑏 + ( ) 𝑎2 𝑏 2 + ( ) 𝑎𝑏 3 + ( ) 𝑏 4
0 1 2 3 4

Entries on each line are symmetric about the middle entry (or entries)

RHS:

𝑛 𝑛! 𝑛! 𝑛! 𝑛
( )= = = = ( ) = 𝐿𝐻𝑆
𝑛−𝑘 (𝑛 − 𝑘)! [𝑛 − (𝑛 − 𝑘)]! (𝑛 − 𝑘)! (𝑛 − 𝑛 − 𝑘)! 𝑘! (𝑛 − 𝑘)! 𝑘

𝑛 𝑛−1 𝑛−1
2. ( )=( )+( ) ⇒ Pascal’s Relationship
𝑘 𝑘−1 𝑘
𝑛
Think of the number ( ) as the 𝑘𝑡ℎ term in the 𝑛𝑡ℎ row of Pascal’s triangle.
𝑘
1
1 1
1 2 1
1 3 3 1
1 4 6 4 1

That is, each term is the sum of two terms above it.

22 | P a g e
Prepared by Dr. Herbert Hove
 𝑛−1 𝑛−1 (𝑛−1)! (𝑛−1)!
RHS: ( )+( ) = (𝑘−1)![(𝑛−1)−(𝑘−1)]! +
𝑘−1 𝑘 𝑘![(𝑛−1)−𝑘]!

(𝑛 − 1)! (𝑛 − 1)!
= +
(𝑘 − 1)! (𝑛 − 𝑘)! 𝑘! ((𝑛 − 𝑘) − 1)!

𝑘 (𝑛 − 1)! (𝑛 − 𝑘)(𝑛 − 1)!

= +
𝑘(𝑘 − 1)! (𝑛 − 𝑘)! 𝑘! (𝑛 − 𝑘)((𝑛 − 𝑘) − 1)!

𝑘(𝑛 − 1)! (𝑛 − 𝑘)(𝑛 − 1)!

= +
𝑘! (𝑛 − 𝑘)! 𝑘! (𝑛 − 𝑘)!

(𝑛 − 1)! [𝑘 + (𝑛 − 𝑘)]
=
𝑘! (𝑛 − 𝑘)!

𝑛!
=
𝑘! (𝑛 − 𝑘)!

𝑛
= ( ) = 𝐿𝐻𝑆
𝑘

23 | P a g e
Prepared by Dr. Herbert Hove