BASIC PROBABILITY

LEARNING OBJECTIVES

• In this chapter, you learn:

• Basic probability concepts
• Conditional probability
• To use Bayes’ Theorem to revise probabilities
• Various counting rules

• Witte, R.S. and Witte, J.S. (2017). Statistics. 11th Edition, Wiley

• Berenson, M.L., Levine, D.M. and Krehbiel,T.C (2012). Basic Business Statistics: Concepts and
applications, 12th Edition (International Edition), Prentice Hall, New Jersey (BLK).
 BASIC PROBABILITY CONCEPTS

Probability – the chance that an uncertain event will occur

(always between 0 and 1)

Impossible Event – an event that has no chance of occurring

(probability = 0)

Certain Event – an event that is sure to occur (probability =

1)
 EVENTS

• Each possible outcome of a variable is an event.

• Simple event
• An event described by a single characteristic
• e.g., A day in January from all days in 2010
• Joint event
• An event described by two or more characteristics
• e.g. A day in January that is also a Wednesday from all days in 2010
• Complement of an event A (denoted A’)
• All events that are not part of event A
• e.g., All days from 2010 that are not in January
 SAMPLE SPACE

• The Sample Space is the collection of all possible events

• e.g. All 6 faces of a die:

• e.g. All 52 cards of a bridge deck:

 DEFINITION: SIMPLE PROBABILITY

• Simple Probability refers to the probability of a simple event.

• P(Jan.)
• P(Wed.)

Jan. Not Jan. Total P(Wed.) = 52 / 365

Wed. 4 48 52
Not Wed. 27 286 313
Total 31 334 365

P(Jan.) = 31 / 365
MUTUALLY EXCLUSIVE EVENTS

Mutually exclusive events

Events that cannot occur simultaneously

Example: Randomly choosing a day from

2010
A = day in January; Events A and B are
B = day in February mutually exclusive
 COLLECTIVELY EXHAUSTIVE EVENTS

• Collectively exhaustive events

1. One of the events must occur
2. The set of events covers the entire sample space

• Example: Randomly choose a day from 2010

A = Weekday; B = Weekend;
C = January; D = Spring;

• Events A, B, C and D are collectively exhaustive (but not mutually exclusive – a weekday can be in January
or in Spring)
• Events A and B are collectively exhaustive and also mutually exclusive
 JOINT PROBABILITIES AND MARGINAL PROBABILITIES

• The probability of a joint event, A and B:

number of outcomes satisfying A and B

P( A and B) =
total number of elementary outcomes
• Computing a marginal (or simple) probability:

P(A) = P(A and B1 ) + P(A and B 2 ) +  + P(A and Bk )

• Where B1, B2, …, Bk are k mutually exclusive and collectively exhaustive events
 JOINT PROBABILITY EXAMPLE MARGINAL PROBABILITY EXAMPLE

P(Jan. and Wed.) P(Wed.)

4 48 52
= P(Jan. and Wed.) + P(Not Jan. and Wed.) = + =
number of days that are in Jan. and are Wed. 4 365 365 365
= =
total number of days in 2010 365

Jan. Not Jan. Total Jan. Not Jan. Total

Wed. 4 48 52 Wed. 4 48 52

Not Wed. 27 286 313 Not Wed. 27 286 313

Total 31 334 365 Total 31 334 365

 PROBABILITY SUMMARY SO FAR

• Probability is the numerical measure of the likelihood that an

event will occur 1 Certai
• The probability of any event must be between 0 and 1, n
inclusively

0 ≤ 𝑃(𝐴) ≤ 1 For any event A

0.5
• The sum of the probabilities of all mutually exclusive and
collectively exhaustive events is 1
P(A) + P(B) + P(C) = 1
If A, B, and C are mutually exclusive and
collectively exhaustive
0 Impossible
 GENERAL ADDITION RULE

• If A and B are mutually exclusive, then P(A and B) = 0, so the

• rule can be simplified:

𝑷(𝑨 𝒐𝒓 𝑩) = 𝑷(𝑨) + 𝑷(𝑩) − 𝑷(𝑨 𝒂𝒏𝒅 𝑩)

𝑷(𝑨 𝒐𝒓 𝑩) = 𝑷(𝑨) + 𝑷(𝑩)

For mutually exclusive events A and B
GENERAL ADDITION RULE EXAMPLE

𝑷(𝑱𝒂𝒏. 𝒐𝒓 𝑾𝒆𝒅. ) = 𝑷(𝑱𝒂𝒏. ) + 𝑷(𝑾𝒆𝒅. ) − 𝑷(𝑱𝒂𝒏. 𝒂𝒏𝒅 𝑾𝒆𝒅. )

= 𝟑𝟏/𝟑𝟔𝟓 + 𝟓𝟐/𝟑𝟔𝟓 − 𝟒/𝟑𝟔𝟓 = 𝟕𝟗/𝟑𝟔𝟓

Don’t count
the four
Wednesdays in
Jan Not Jan Total January twice!

Wed 4 48 52

Not Wed 27 286 313

Total 31 334 365

 COMPUTING CONDITIONAL PROBABILITIES

• A conditional probability is the probability of one event, given that another event has
occurred:
P(A and B) The conditional
P(A | B) = probability of A given
P(B) that B has occurred

P(A and B) The conditional

P(B | A) = probability of B given
P(A) that A has occurred

Where P(A and B) = joint probability of A and B

P(A) = marginal or simple probability of A
P(B) = marginal or simple probability of B
 CONDITIONAL PROBABILITY EXAMPLE

• Of the cars on a used car lot, 90% have air conditioning (AC) and 40% have a GPS. 35% of the
cars have both.

• What is the probability that a car has a GPS given that it has AC ?
i.e., we want to find P(GPS | AC)
 ANSWER

• Of the cars on a used car lot, 90% have air conditioning (AC) and 40% have a GPS.
35% of the cars have both.

GPS No GPS Total

AC 0.35 0.55 0.90

No AC 0.05 0.05 0.10

Total 0.40 0.60 1.00

 ANSWER: CONTD…

• Given AC, we only consider the top row (90% of the cars). Of these, 35% have a GPS. 35% of
90% is about 38.89%.
GPA No GPS Total

AC 0.35 0.55 0.90

No AC 0.05 0.05 0.10

Total 0.40 0.60 1.00

P(GPS and AC) 0.35

P(GPS | AC) = = = 0.3889
P(AC) 0.90
DECISION TREES
 GPA No GPS Total
AC 0.35 0.55 0.90
No AC 0.05 0.05 0.10

Total 0.40 0.60 1.00 .35

Given AC or no .90 P(AC and GPS) = 0.35
AC:

.55 P(AC and GPS’) = 0.55

.90
Conditional
All
Probabilities
Cars
.05
.10 P(AC’ and GPS) = 0.05

.05 P(AC’ and GPS’) = 0.05

.10
 GPA No GPS Total
AC 0.35 0.55 0.90
No AC 0.05 0.05 0.10

Total 0.40 0.60 1.00 .35

Given GPS or no .40 P(GPS and AC) = 0.35
GPS:

.05 P(GPS and AC’) = 0.05

.40
Conditional
All
Probabilities
Cars
.55
.60 P(GPS’ and AC) = 0.55

.05 P(GPS’ and AC’) = 0.05

.60
 INDEPENDENCE

• Two events are independent if and only if:

𝑃 𝐴 𝐵 = 𝑃(𝐴)
• Events A and B are independent when the probability of one event is not affected by the fact
that the other event has occurred
 MULTIPLICATION RULES

• Multiplication rule for two events A and B:

𝑃 𝐴 𝑎𝑛𝑑 𝐵 = 𝑃 𝐴 𝐵 𝑃(𝐵)

• Note: If A and B are independent, then 𝑃(𝐴│𝐵)=𝑃(𝐴), and the multiplication rule simplifies to

𝑃 𝐴 𝑎𝑛𝑑 𝐵 = 𝑃(𝐴)𝑃(𝐵)
 MARGINAL PROBABILITY

• Marginal probability for event A:

𝑃 𝐴 = 𝑃 𝐴 𝐵1 𝑃(𝐵1 )+ 𝑃 𝐴 𝐵2 𝑃(𝐵2 )+…+ 𝑃 𝐴 𝐵𝑘 𝑃(𝐵𝑘 )

• Where B1, B2, …, Bk are k mutually exclusive and collectively exhaustive events
 BAYES’ THEOREM

• Bayes’ Theorem is used to revise previously calculated probabilities based on new

information.

• Developed by Thomas Bayes in the 18th Century.

• It is an extension of conditional probability.

 BAYES’ THEOREM

P(A | B i )P(B i )
P(B i | A) =
P(A | B 1 )P(B 1 ) + P(A | B 2 )P(B 2 ) +    + P(A | B k )P(B k )

• where:
𝐵𝑖 = 𝑖𝑡ℎ event of 𝑘 mutually exclusive and collectively
exhaustive events
𝐴 = new event that might impact 𝑃(𝐵𝑖)
 BAYES’ THEOREM EXAMPLE

• A drilling company has estimated a 40% chance of striking oil for their new well.
• A detailed test has been scheduled for more information. Historically, 60% of successful wells
have had detailed tests, and 20% of unsuccessful wells have had detailed tests.
• Given that this well has been scheduled for a detailed test, what is the probability that the
well will be successful?
 EXAMPLE: CONTD…

• Let S = successful well

• U = unsuccessful well
• 𝑃(𝑆) = 0.4 , 𝑃(𝑈) = 0.6 (prior probabilities)
• Define the detailed test event as D
• Conditional probabilities:
• 𝑃(𝐷|𝑆) = 0.6 𝑃(𝐷|𝑈) = 0.2
• Goal is to find 𝑃(𝑆|𝐷)
 APPLY BAYES’ THEOREM:

P(D | S)P(S)
P(S | D) =
P(D | S)P(S) + P(D | U)P(U)
(0.6)(0.4)
=
(0.6)(0.4) + (0.2)(0.6)
0.24
= = 0.667
0.24 + 0.12

• So the revised probability of success, given that this well has been scheduled for a detailed
test, is 0.667
 Prior Conditional Joint Revised
Event
Prob. Prob. Prob. Prob.
S (successful) 0.4 0.6 (0.4)(0.6) = 0.24 0.24/0.36 = 0.667
U (unsuccessful) 0.6 0.2 (0.6)(0.2) = 0.12 0.12/0.36 = 0.333
Sum = 0.36

Given the detailed test, the revised probability of a successful well has risen to 0.667 from the
original estimate of 0.4
 COUNTING RULES: RULES FOR COUNTING THE
NUMBER OF POSSIBLE OUTCOMES

• Counting Rule 1:
• If any one of k different mutually exclusive and collectively exhaustive events can occur on
each of n trials, the number of possible outcomes is equal to

𝑘𝑛
• Example
• If you roll a fair die 3 times, then there are 63 = 216 possible outcomes
• Counting Rule 2:
• If there are 𝑘1 events on the first trial, 𝑘2 events on the second trial, … and 𝑘𝑛 events on the
𝑛𝑡ℎ trial, the number of possible outcomes is

𝑘1 𝑘2 … (𝑘𝑛 )

• Example: You want to go to a park, eat at a restaurant, and see a movie. There are 3 parks, 4
restaurants, and 6 movie choices. How many different possible combinations are there?
• Answer: (3)(4)(6) = 72 different possibilities
• Counting Rule 3:
• The number of ways that n items can be arranged in order is

𝑛! = 𝑛 𝑛 − 1 … (1)
• Example: You have five books to put on a bookshelf. How many different ways can these
books be placed on the shelf?
• Answer: 5! = (5)(4)(3)(2)(1) = 120 different possibilities
• Counting Rule 4: Permutations
• The number of ways of arranging X objects selected from n objects in order is

𝑛!
𝑛𝑃𝑋 =
𝑛−𝑋 !

• Example: You have five books and are going to put three on a bookshelf. How many different
ways can the books be ordered on the bookshelf?

= 60 different possibilities
n! 5! 120
Answer: n Px = = =
(n − X)! (5 − 3)! 2
• Counting Rule 5: Combinations
• The number of ways of selecting X objects from n objects, irrespective of order, is

𝑛!
𝑛𝐶𝑋 =
𝑋! (𝑛 − 𝑋)!

Example: You have five books and are going to randomly select three to read. How many
different combinations of books might you select?

n! 5! 120
Answer: n Cx = = = = 10 different possibilities
X!(n − X)! 3! (5 − 3)! (6)(2)
 CHAPTER SUMMARY

• Discussed basic probability concepts

• Sample spaces and events, contingency tables, simple probability, and joint probability
• Examined basic probability rules
• General addition rule, addition rule for mutually exclusive events, rule for collectively
exhaustive events
• Defined conditional probability
• Statistical independence, marginal probability, decision trees, and the multiplication rule
• Discussed Bayes’ theorem
• Discussed various counting rules