Basics of Probability

Basic Probability Concepts

• Probability : the chance that an uncertain event will occur
always between 0 and 1
• Impossible Event : an event that has no chance of occurring
probability is equal to 0
• Certain Event – an event that is sure to occur
probability is equal to 1
 Assessing Probability
• There are three approaches to assessing the probability of an
uncertain event
a priori : based on prior knowledge of the process

empirical probability : based on historical occurrence

subjective probability : judgment based on experience

Example of a priori probability
When randomly selecting a day from the year 2015 what
is the probability the day is in January?

X number of days in January

Probability of Day In January  
T total number of days in 2015

X 31 days in January 31
 
T 365 days in 2015 365
Example of empirical probability
Find the probability of selecting a male taking statistics
from the population described in the following table:

Taking Stats Not Taking Total

Stats
Male 84 145 229
Female 76 134 210
Total 160 279 439

number of males taking stats 84

Probability of male taking stats    0.191
total number of people 439
Subjective probability
• Subjective probability may differ from person to person
• A media development team assigns a 60% probability of success to
its new ad campaign.
• The chief media officer of the company is less optimistic and assigns
a 40% of success to the same campaign
• The assignment of a subjective probability is based on a
person’s experiences, opinions, and analysis of a particular
situation
• Subjective probability is useful in situations when an empirical
or a priori probability cannot be computed
Events
Each possible outcome of a variable is an event.

• Simple event
• An event described by a single characteristic
• e.g., A day in January from all days in 2015
• Joint event
• An event described by two or more characteristics
• e.g. A day in January that is also a Wednesday from all days in 2015
• Complement of an event A (denoted A’)
• All events that are not part of event A
• e.g., All days from 2015 that are not in January
Sample Space
The Sample Space is the collection of all
possible events

e.g. All 6 faces of a die:

e.g. All 52 cards of a deck:

Organizing & Visualizing Events
• Venn Diagram For All Days In 2015
Sample Space (All Days In 2015) Days That Are In January and Are Wednesdays

January Days

Wednesdays
Organizing & Visualizing Events
• Contingency Tables -- For All Days in 2015

• Decision Trees
Total Number

Sample
Wed. 4 Of Sample Space
Outcomes
J an .
Space
All Days Not Wed. 27
In 2015 Wed. 48
N o t Ja
n.
Not W 286
ed .
Definition: Simple Probability
• Simple Probability refers to the probability of a simple event.
• ex. P(Jan.)
• ex. P(Wed.)

Jan. Not Jan. Total P(Wed.) = 52 / 365

Wed. 4 48 52
Not Wed. 27 286 313

Total 31 334 365

P(Jan.) = 31 / 365
Definition: Joint Probability
• Joint Probability refers to the probability of an occurrence of two
or more events (joint event).
• ex. P(Jan. and Wed.)
• ex. P(Not Jan. and Not Wed.)

Jan. Not Jan. Total

P(Not Jan. and Not Wed.)
Wed. 4 48 52 = 286 / 365
Not Wed. 27 286 313

Total 31 334 365

P(Jan. and Wed.) = 4 / 365

Mutually Exclusive Events
• Mutually exclusive events
• Events that cannot occur simultaneously

Example: Randomly choosing a day from 2015

A = day in January; B = day in February

• Events A and B are mutually exclusive

Collectively Exhaustive Events
• Collectively exhaustive events
• One of the events must occur
• The set of events covers the entire sample space

Example: Randomly choose a day from 2015

A = Weekday B = Weekend
C = January; D = Spring;

• Events A, B, C and D are collectively exhaustive (but not mutually

exclusive – a weekday can be in January or in Spring)
• Events A and B are collectively exhaustive and also mutually exclusive
Computing Joint and Marginal Probabilities
• The probability of a joint event, A and B:
number of outcomes satisfying A and B
P( A and B) 
total number of elementary outcomes

• Computing a marginal (or simple) probability:

P(A)  P(A and B1 )  P(A and B 2 )    P(A and Bk )

• Where B1, B2, …, Bk are k mutually exclusive and collectively exhaustive events
Joint Probability Example

Jan. Not Jan. Total

Wed. 4 48 52
Not Wed. 27 286 313

Total 31 334 365

Marginal Probability Example

Jan. Not Jan. Total

Wed. 4 48 52
Not Wed. 27 286 313

Total 31 334 365

Marginal & Joint Probabilities in a Contingency Table

Event
Event B1 B2 Total
A1 P(A1 and B1) P(A1 and B2) P(A1)

A2 P(A2 and B1) P(A2 and B2) P(A2)

Total P(B1) P(B2) 1

Joint Probabilities Marginal (Simple) Probabilities

Probability Summary So Far
• Probability is the numerical measure of the likelihood
that an event will occur 1 Certain

• The probability of any event must be between 0 and 1,

inclusively
0 ≤ P(A) ≤ 1 For any event A
0.5
• The sum of the probabilities of all mutually exclusive
and collectively exhaustive events is 1

If A, B, and C are mutually exclusive and collectively

exhaustive
P(A)  P(B)  P(C)  1 0 Impossible
General Addition Rule
General Addition Rule:
P(A or B) = P(A) + P(B) - P(A and B)

If A and B are mutually exclusive, then

P(A and B) = 0, so the rule can be simplified:

P(A or B) = P(A) + P(B)

For mutually exclusive events A and B
General Addition Rule Example
P(Jan. or Wed.) = P(Jan.) + P(Wed.) - P(Jan. and Wed.)
= 31/365 + 52/365 - 4/365 = 79/365
Don’t count
the four
Wednesdays
in January
Jan. Not Jan. Total twice!
Wed. 4 48 52
Not Wed. 27 286 313

Total 31 334 365

Computing Conditional Probabilities
• A conditional probability is the probability of one
event, given that another event has occurred:

P(A and B) The conditional

P(A | B)  probability of A given
P(B) that B has occurred

P(A and B) The conditional

P(B | A)  probability of B given
P(A) that A has occurred

Where P(A and B) = joint probability of A and B

P(A) = marginal or simple probability of A
P(B) = marginal or simple probability of B
Conditional Probability Example
 Of the cars on a used car lot, 70% have air
conditioning (AC) and 40% have a GPS. 20%
of the cars have both.

• What is the probability that a car has a GPS, given

that it has AC ?

i.e., we want to find P(GPS | AC)

Conditional Probability Example
 Of the cars on a used car lot, 70% have air conditioning
(AC) and 40% have a GPS and
20% of the cars have both.
GPS No GPS Total
AC 0.2 0.5 0.7
No AC 0.2 0.1 0.3
Total 0.4 0.6 1.0

P(GPS and AC) 0.2

P(GPS | AC)    0.2857
P(AC) 0.7
Conditional Probability Example
 Given AC, we only consider the top row (70% of the cars). Of
these, 20% have a GPS. 20% of 70% is about 28.57%.

GPS No GPS Total

AC 0.2 0.5 0.7
No AC 0.2 0.1 0.3
Total 0.4 0.6 1.0

P(GPS and AC) 0.2

P(GPS | AC)    0.2857
P(AC) 0.7
Independence
• Two events are independent if and only if:

P(A | B)  P(A)
• Events A and B are independent when the probability of one event is not affected
by the fact that the other event has occurred
Multiplication Rules
• Multiplication rule for two events A and B:

P(A and B)  P(A | B) P(B)

Note: If A and B are independent, then P(A | B)  P(A)

and the multiplication rule simplifies to

P(A and B)  P(A) P(B)

Marginal Probability
• Marginal probability for event A:

P(A)  P(A | B1 ) P(B1 )  P(A | B 2 ) P(B 2 )    P(A | Bk ) P(B k )

• Where B1, B2, …, Bk are k mutually exclusive and collectively exhaustive events
Counting Rules Are Often Useful In Computing
Probabilities

• In many cases, there are a large number of possible outcomes.

• Counting rules can be used in these cases to help compute

probabilities.
Counting Rules
• Rules for counting the number of possible outcomes

• Counting Rule 1:
• If any one of k different mutually exclusive and collectively exhaustive events
can occur on each of n trials, the number of possible outcomes is equal to

kn
• Example
• If you roll a fair die 3 times then there are 63 = 216 possible outcomes
 Counting Rules
• Counting Rule 2:
• If there are k1 events on the first trial, k2 events on the second trial,
… and kn events on the nth trial, the number of possible outcomes
is (k1)(k2)…(kn)

• Example:
• You want to go to a park, eat at a restaurant, and see a movie. There are 3
parks, 4 restaurants, and 6 movie choices. How many different possible
combinations are there?
• Answer: (3)(4)(6) = 72 different possibilities
Counting Rules
• Counting Rule 3:
• The number of ways that n items can be arranged in order is

n! = (n)(n – 1)…(1)

• Example:
• You have five books to put on a bookshelf. How many different ways can
these books be placed on the shelf?

• Answer: 5! = (5)(4)(3)(2)(1) = 120 different possibilities

Counting Rules
• Counting Rule 4:
• Permutations: The number of ways of arranging X objects selected from n objects in
order is
n!
n Px 
(n  X)!
• Example:
• You have five books and are going to put three on a bookshelf. How
many different ways can the books be ordered on the bookshelf?
n! 5! 120
• Answer: n Px     60 different possibilities
(n  X)! (5  3)! 2
Counting Rules
• Counting Rule 5:
• Combinations: The number of ways of selecting X objects from n objects,
irrespective of order, is
n!
n Cx 
X!(n  X)!
• Example:
• You have five books and are going to select three are to read. How many
different combinations are there, ignoring the order in which they are
selected?
n! 5! 120
n Cx     10 different possibilities
• Answer: X!(n  X)! 3!(5  3)! (6)(2)