A Survey of Probability Concepts

Definitions

A probability is a measure of the likelihood that an event in

the future will happen. It can only assume a value between
0 and 1.

 A value near zero means the event is not likely to happen.

A value near one means it is likely.

 There are three ways of assigning probability:

– classical,
– empirical, and
– subjective.
2
 Definitions continued

 An experiment is a situation involving chance or

probability that leads to results called outcomes.

 An outcome is the result of a single trial of an

experiment.

 An event is one or more outcomes of an

experiment.

3
 Assigning Probabilities

Three approaches to assigning probabilities

– Classical
– Empirical
– Subjective

4
 Classical Probability

Consider an experiment of rolling a six-sided die. What is the probability

of the event “an even number of spots appear face up”?
The possible outcomes are:

There are three “favorable” outcomes (a two, a four, and a six) in the
collection of six equally likely possible outcomes.

5
 Mutually Exclusive Events

 Events are mutually exclusive if the occurrence of any one event means that none
of the others can occur at the same time.

 Events are independent if the occurrence of one event does not affect the
occurrence of another.

6
 Collectively Exhaustive Events

 Eventsare collectively exhaustive if at

least one of the events must occur
when an experiment is conducted.

7
 Empirical Probability

The empirical approach to probability is based on what is

called the law of large numbers. The key to establishing
probabilities empirically is that more observations will
provide a more accurate estimate of the probability.

8
 Law of Large Numbers

Suppose we toss a fair coin. The result of each toss is either a head or a
tail. If we toss the coin a great number of times, the probability of the
outcome of heads will approach .5. The following table reports the
results of an experiment of flipping a fair coin 1, 10, 50, 100, 500,
1,000 and 10,000 times and then computing the relative frequency of
heads

9
 Empirical Probability - Example

On February 1, 2003, the Space Shuttle Columbia exploded.

This was the second disaster in 113 space missions for
NASA. On the basis of this information, what is the
probability that a future mission is successfully completed?

Number of successful flights

Probability of a successful flight 
Total number of flights
111
  0.98
113

10
Problem

Two dice are rolled, find the probability that the

sum is

a) equal to 1
b) equal to 4
c) less than 13
Solution

a) The sample space S of two dice is shown below.

S = { (1,1),(1,2),(1,3),(1,4),(1,5),(1,6)
 (2,1),(2,2),(2,3),(2,4),(2,5),(2,6)
 (3,1),(3,2),(3,3),(3,4),(3,5),(3,6)
 (4,1),(4,2),(4,3),(4,4),(4,5),(4,6)
 (5,1),(5,2),(5,3),(5,4),(5,5),(5,6)
 (6,1),(6,2),(6,3),(6,4),(6,5),(6,6) }
 Let E be the event "sum equal to 1". There are no
outcomes which correspond to a sum equal to 1, hence
 P(E) = n(E) / n(S) = 0 / 36 = 0
 b) Three possible ouctcomes give a sum equal to
4: E = {(1,3),(2,2),(3,1)}, hence.
 P(E) = n(E) / n(S) = 3 / 36 = 1 / 12
 c) All possible ouctcomes, E = S, give a sum less
than 13, hence.
 P(E) = n(E) / n(S) = 36 / 36 = 1
Problem

 A die is rolled and a coin is tossed, find the

probability that the die shows an odd number and
the coin shows a head.
Solution

 The sample space S of the experiment is as follows

 S = { (1,H),(2,H),(3,H),(4,H),(5,H),(6,H)
 (1,T),(2,T),(3,T),(4,T),(5,T),(6,T)}
 Let E be the event "the die shows an odd number and the
coin shows a head". Event E may be described as follows
 E={(1,H),(3,H),(5,H)}
 The probability P(E) is given by
 P(E) = n(E) / n(S) = 3 / 12 = 1 / 4
 Rules for Computing Probabilities

Rules of Addition
 Special Rule of Addition - If two events A
and B are mutually exclusive, the
probability of one or the other event’s
occurring equals the sum of their
probabilities.
P(A or B) = P(A) + P(B)

 The General Rule of Addition - If A and B

are two events that are not mutually
exclusive, then P(A or B) is given by the
following formula:
P(A or B) = P(A) + P(B) - P(A and B)

16
 Addition Rule - Example

What is the probability that a card chosen at random

from a standard deck of cards will be either a king
or a heart?

P(A or B) = P(A) + P(B) - P(A and B)

= 4/52 + 13/52 - 1/52
= 16/52, or .3077
17
Example

 If the probabilities are respectively 0.09, 0.15,

0.21 and 0.23 that a person purchasing a new
automobile will choose the color green, white, red
or blue, what is the probability that a given buyer
will purchase a new automobile that comes in one
of those colors?
Solution

 Let G,W,R and B be the events that a buyer

selects respectively, a green, white, red, or blue
automobiles. Since these four events are mutually
exclusive, the probability is

P(G U W U R U B)=P(G)+P(W)+P(R)+P(B)
=0.09+0.15+0.21+0.23
=0.68
 The Complement Rule

The complement rule is used to determine the

probability of an event occurring by subtracting
the probability of the event not occurring from
1.
P(A) + P(~A) = 1
or P(A) = 1 - P(~A).

20
Example

 If the probabilities that an automobile mechanic

will service 3,4,5,6,7, or 8 or more cars on any
given workday are respectively, 0.12, 0.19, 0.28,
0.24, 0.10 and 0.07, what is the probability that he
will service at least 5 cars on his next day at
work?
Solution

 Let E be the event that at least 5 cars are serviced.

Now, P(E)=1-P(Ec)
where, Ec is the event that fewer than 5 cars are
serviced.
Since
P(Ec)=0.12+0.19=0.31
Hence, P(E)=1-0.31=0.69
 Joint Probability – Venn Diagram

JOINT PROBABILITY A probability that

measures the likelihood two or more events will
happen concurrently.

23
 Special Rule of Multiplication

 The special rule of multiplication requires that two events A and B are
independent.
 Two events A and B are independent if the occurrence of one has no effect on the probability
of the occurrence of the other.
 This rule is written: P(A and B) = P(A)P(B)

24
 Multiplication Rule-Example

A survey by the American Automobile association (AAA)

revealed 60 percent of its members made airline
reservations last year. Two members are selected at
random. What is the probability both made airline
reservations last year?

Solution:
The probability the first member made an airline reservation last year is .60,
written as P(R1) = .60
The probability that the second member selected made a reservation is also .60, so
P(R2) = .60.
Since the number of AAA members is very large, you may assume that
R1 and R2 are independent.

P(R1 and R2) = P(R1)P(R2) = (.60)(.60) = .36

25
 Conditional Probability

A conditional probability is the probability of a

particular event occurring, given that another
event has occurred.

The probability of the event A given that the event B

has occurred is written P(A|B).

26
 General Multiplication Rule

It states that for two events, A and B, the joint probability that both
events will happen is found by multiplying the probability that
event A will happen by the conditional probability of event B
occurring given that A has occurred.

27
Conditional Probability

•• Conditional
ConditionalProbability
Probability--Probability
Probabilityof
ofAAgiven
givenBB

P( A  B)
P ( A B)  , where P( B)  0
P ( B)

 Independent
Independentevents:
events:
P( A B)  P( A)
P( B A)  P( B)
Conditional Probability (continued)

Rulesof
Rules ofconditional
conditionalprobability:
probability:

P( A B)  P( A  B) so P( A  B)  P( A B) P( B)
P( B)
 P( B A) P( A)

If events A and D are statistically independent:

P ( A D )  P ( A)
so P( A  D)  P( A) P( D)
P ( D A)  P ( D )
 General Multiplication Rule - Example

A golfer has 12 golf shirts in his closet. Suppose 9 of these shirts are white
and the others blue. He gets dressed in the dark, so he just grabs a shirt and
puts it on. He plays golf two days in a row and does not do laundry.
What is the likelihood both shirts selected are white?

30
 General Multiplication Rule - Example

 The event that the first shirt selected is white is W1.

The probability is P(W1) = 9/12
 The event that the second shirt selected is also white is
identified as W2. The conditional probability that the
second shirt selected is white, given that the first shirt
selected is also white, is P(W2 | W1) = 8/11.
 To determine the probability of 2 white shirts being
selected we use formula: P(AB) = P(A) P(B|A)
 P(W1 and W2) = P(W1)P(W2 |W1) = (9/12)(8/11) = 0.55

31
 Contingency Tables

A CONTINGENCY TABLE is a table used to classify sample

observations according to two or more identifiable
characteristics

E.g. A survey of 150 adults classified each as to gender and the

number of movies attended last month. Each respondent is
classified according to two criteria—the number of movies
attended and gender.

32
 Contingency Tables - Example

A sample of executives were surveyed about their loyalty to their

company. One of the questions was, “If you were given an offer by
another company equal to or slightly better than your present position,
would you remain with the company or take the other position?” The
responses of the 200 executives in the survey were cross-classified
with their length of service with the company.

What is the probability of randomly selecting an executive who is loyal to

the company (would remain) and who has more than 10 years of
service?
33
 Contingency Tables - Example

Event A1 happens if a randomly selected executive will remain with

the company despite an equal or slightly better offer from
another company. Since there are 120 executives out of the
200 in the survey who would remain with the company
P(A1) = 120/200, or .60.
Event B4 happens if a randomly selected executive has more than
10 years of service with the company. Thus, P(B4| A1) is the
conditional probability that an executive with more than 10
years of service would remain with the company. Of the 120
executives who would remain 75 have more than 10 years of
service, so P(B4| A1) = 75/120.

34
Contingency Table - Example
Counts
AT& T IBM Total

Telecommunication 40 10 50 Probability that a project

Computers 20 30 50
is undertaken by IBM
given it is a
Total 60 40 100
telecommunications
Probabilities project:
P ( IBM T )
AT& T IBM Total P ( IBM T ) 
P (T )
Telecommunication .40 .10 .50
0.10
  0.2
Computers .20 .30 .50 0.50

Total .60 .40 1.00

Product Rules for Independent Events
The probability of the intersection of several independent events
is the product of their separate individual probabilities:
P( A  A  A  An )  P( A ) P( A ) P( A ) P( An )
1 2 3 1 2 3

The probability of the union of several independent events

is 1 minus the product of probabilities of their complements:
P( A  A  A  An )  1  P( A ) P( A ) P( A ) P( An )
1 2 3 1 2 3
The Law of Total Probability and Bayes’
Theorem

The law of total probability:

P( A)  P( A  B)  P( A  B )

In terms of conditional probabilities:

P( A)  P( A  B)  P( A  B )
 P( A B) P( B)  P( A B ) P( B )

More generally (where Bi make up a partition):

P( A)   P( A  B )
i
  P( AB ) P( B )
i i
The Law of Total Probability-
Example

Event U: Stock market will go up in the next year

Event W: Economy will do well in the next year
P(U W ) .75
P(U W ) .30
P(W ) .80 P(W ) 1.8 .2
P(U )  P(U W )  P(U W )
 P(U W )P(W )  P(U W )P(W )
 (.75)(.80)  (.30)(.20)
.60 .06 .66
 Bayes’ Theorem

 Bayes’ Theorem is a method for revising a

probability given additional information.

 It is computed using the following formula:

39
 Bayes’ Theorem Extended - Example

 An economist believes that during periods of high economic growth, the U.S.
dollar appreciates with probability 0.70; in periods of moderate economic
growth, the dollar appreciates with probability 0.40; and during periods of
low economic growth, the dollar appreciates with probability 0.20.
 During any period of time, the probability of high economic growth is 0.30,
the probability of moderate economic growth is 0.50, and the probability of
low economic growth is 0.50.
 Suppose the dollar has been appreciating during the present period. What is
the probability we are experiencing a period of high economic growth?
Partition: Event A  Appreciation
H - High growth P(H) = 0.30 P( A H )  0.70
M - Moderate growth P(M) = 0.50 P( A M )  0.40
L - Low growth P(L) = 0.20 P( A L)  0.20
Example 2-11 (continued)

P ( H  A)
P ( H A) 
P( A)
P ( H  A)

P( H  A)  P ( M  A)  P ( L  A)
P( A H ) P( H )

P ( A H ) P ( H )  P ( A M ) P ( M )  P ( A L) P ( L )
( 0.70)( 0.30)

( 0.70)( 0.30)  ( 0.40)( 0.50)  ( 0.20)( 0.20)
0.21 0.21
 
0.21 0.20  0.04 0.45
 0.467
 Bayes Theorem - Example

42
 Bayes Theorem – Example (cont.)

43
 Bayes Theorem – Example (cont.)

44
 Bayes Theorem – Example (cont.)

45
 Bayes Theorem – Example (cont.)

46
Example

 Marie is getting married tomorrow, at an outdoor

ceremony in the desert. In recent years, it has
rained only 5 days each year. Unfortunately, the
weatherman has predicted rain for tomorrow.
When it actually rains, the weatherman correctly
forecasts rain 90% of the time. When it doesn't
rain, he incorrectly forecasts rain 10% of the time.
What is the probability that it will rain on the day
of Marie's wedding?
Solution:

 The sample space is defined by two mutually-exclusive events - it rains

or it does not rain. Additionally, a third event occurs when the
weatherman predicts rain. Notation for these events appears below.

 Event A1. It rains on Marie's wedding.

 Event A2. It does not rain on Marie's wedding
 Event B. The weatherman predicts rain.
 In terms of probabilities, we know the following:
 P( A1 ) = 5/365 =0.0136985 [It rains 5 days out of the year.]
 P( A2 ) = 360/365 = 0.9863014 [It does not rain 360 days out of the
year.]
 P( B | A1 ) = 0.9 [When it rains, the weatherman predicts rain 90% of
the time.]
 P( B | A2 ) = 0.1 [When it does not rain, the weatherman predicts rain
10% of the time.]
 We want to know P( A1 | B ), the probability it will rain on
the day of Marie's wedding, given a forecast for rain by the
weatherman. The answer can be determined from Bayes'
theorem, as shown below.

 P( A1 | B ) = P( A1 ) P( B | A1 )
 ---------------------------------------------
 P( A1 ) P( B | A1 ) + P( A2 ) P( B | A2 )

 P( A1 | B ) = (0.014)(0.9) / [ (0.014)(0.9) + (0.986)(0.1)]


 P( A1 | B ) = 0.111