Introduction to Probability

1
Research questions
• What is the probability of students that took probability course?
• What is the probability that a randomly selected graduate student
gets more than seven hours of sleep each night?
• Does a machine break down in a year?
• What will be the market share of your new product?

• How can we answer our research questions if we can't ask every

person in the population our research question?
2
Population versus sample
• Take a random sample from the population, and use the resulting
sample to learn something, or make an inference, about the
population

3
Random Experiment
• An experiment that can result in different outcomes, even though it is
repeated in the same manner every time, is called a random
experiment.

4
Sample space
• The sample space (or outcome space or universe), denoted S, is the
collection of all possible outcomes of a random study.
• S = {yes, no} did you take probability course before?
• S = {h: h ≥ 0 hours} h: number of hours slept
• S = {0, 1, 2, ..., 31} how many times did the machine break down last
year?

• For each of the research questions you created:

• Formulate the question you would ask (or describe the measurement
technique you would use).
• Define the resulting sample space.

5
Types of data
• Did you take probability course before? yes yes yes no no no yes yes yes yes
yes no no yes yes no yes yes yes yes

• Quantitative data are called discrete if the sample space contains a finite or
countably infinite number of values.
S = {0, 1, 2, ...} countably infinite
• Quantitative data are called continuous if the sample space contains an interval
or continuous span of real numbers.
S = {h: h ≥ 0 hours}
• Qualitative data are called categorical if the sample space contains objects that
are grouped or categorized based on some qualitative trait. When there are only
two such groups or categories, the data are considered binary.
S = {yes, no}

6
(Discrete) probability mass function

• h(x0) = f0/n is the relative frequency (or proportion) of students, in a

sample of size n, taking probability class x0 times
• As the sample size n increases, f0/n tends to stabilize and approach
some limiting probability p0 = f(x0)

7
Event
• An event — denoted with capital letters A, B, C, ... — is just a subset of the sample space S. That is, for
example A ⊂ S, where "⊂" denotes "is a subset of.“

• "how many pairs of jeans do you own?". Sample space S is S = {0, 1, 2, 3, ...}

• A is the event that a randomly selected student owns no jeans:

• A = student owns none = {0}

• B is the event that a randomly selected student owns some jeans:

• B = student owns some = {1, 2, 3, ...}

• C is the event that a randomly selected student owns no more than five pairs of jeans:
• C = student owns no more than five pairs = {0, 1, 2, 3, 4, 5}

• D is the event that a randomly selected student owns an odd number of pairs of jeans:
• D = student owns an odd number = {1, 3, 5, ...} 8
Set review
• Ø is the "null set" (or "empty set")
• C ∪ D = "union" = the elements in C or D or both
• A ∩ B = "intersection" = the elements in A and B. If A ∩ B = Ø,
then A and B are called "mutually exclusive events" (or "disjoint
events").
• D' = Dc = "complement" = the elements not in D
• If E ∪ F ∪ G ∪ ... = S, then E, F, G, and so on are called "exhaustive
events."

9
Basic Concepts
• Two events are mutually exclusive (independent) if,
when one event occurs, the other cannot, and vice
versa.

•Experiment: Toss a die Not Mutually

–A: observe an odd number Exclusive

–B: observe a number greater than 2

–C: observe a 6 B and C?
Mutually
–D: observe a 3 Exclusive B and D?
Examples
• The union of events C and D is the event that a randomly selected student either
owns no more than five pairs or owns an odd number. That is:
C ∪ D = {0, 1, 2, 3, 4, 5, 7, 9, ...}
• The intersection of events A and B is the event that a randomly selected student
owes no pairs and owes some pairs of jeans. That is:
A ∩ B = {0} ∩ {1, 2, 3, ...} = the empty set Ø
• The complement of event D is the event that a randomly selected student owes
an even number of pairs of jeans. That is:
Dc = {0, 2, 4, 6, ...}
• If E = {0, 1}, F = {2, 3}, G = {4, 5} and so on, so that:
E ∪ F ∪ G ∪ ... = S
then E, F, G, and so on are exhaustive events.

11
Probability
Probability is a number between 0 and 1, where:
• a number close to 0 means "not likely"
• a number close to 1 means "quite likely"

If the probability of an event is exactly 0, then the event can't occur.

If the probability of an event is exactly 1, then the event will definitely
occur.

12
How does an event get assigned a particular
probability value?
• the personal opinion approach
• "I think there is an 80% chance of rain today."
• What about?
• one day you will die?
• you can swim around the world in 30 hours?
• you will win the lottery some day?
• a randomly selected student will get an A in this course?
• you will get an A in this course?

• the relative frequency approach

• the classical approach

13
Calculating probabilities: How likely is it?

14
The Relative Frequency Approach (Statistical Reasoning)
• To determine P(A), the probability of an event A:
• Perform an experiment a large number of times, n, say.
• Count the number of times the event A of interest occurs, call the
number N(A), say.
• Then, the probability of event A equals: P(A) = N(A) / n

• When you toss a fair coin with one side designated as a "head" and the
other side designated as a "tail", what is the probability of getting a head?
n, the number of tosses N(H), the number of
Coin Tosser P(H)
made heads tossed
Count Buffon 4,040 2,048 0.5069
John Kerrich 10,000 5,067 0.5067
Karl Pearson 15
24,000 12,012 0.5005
 Examples
Some workers in a factory were showing signs of disease.
• What is the probability that one worker selected at random is large? 68/200 = 0.34.
• What is the probability that one worker selected at random is diseased?
• 37/200 = 0.185.
• What is the probability that one worker selected at random both works in small
factory and diseased?
• 8/200 = 0.04.
• What is the probability that one worker selected at random is either Works in small
factory or disease-free?
• (35 + 46 + 24 + 8 + 8) /200 = 121 / 200 = 0.605.
• What is the probability that one worker selected at random from the population of
medium sized factory worker is doubtful of disease?
• 32/92 = 0.348.
16
Classical approach (Probabilistic Reasoning)
• the probability of event A is: P(A) = N(A) / N(S)
• N(A) is the number of elements in the event A, and N(S) is the number of
elements in the sample space S.

• Suppose you draw one card at random from a standard deck of 52 cards
• 13 face values (Ace, 2, 3, 4, 5, 6, 7, 8, 9, 10, Jack, Queen, and King) in 4
different suits (Clubs, Diamonds, Hearts, and Spades) for a total of 52
cards.
• A = {x: x is a 2, 3, or 7}
• B = {x: x is 2H, 3D, 8S, or KC}

17
Example
• What is the probability that a 2, 3, or 7 is
drawn?
• What is the probability that the card is a 2 of
hearts, 3 of diamonds, 8 of spades or king of
clubs?
• What is the probability that the card is either
a 2, 3, or 7 or a 2 of hearts, 3 of diamonds, 8
of spades or king of clubs?
• What is P(A∩B)?

18
Axioms of probability
1.The probability of any event A must be nonnegative, that is, P(A) ≥ 0.
2.The probability of the sample space is 1, that is, P(S) = 1.
3.Given mutually exclusive events A1, A2, A3, ... that is, where Ai ∩ Aj = Ø, for i ≠ j,

a.the probability of a finite union of the events is the sum of the probabilities of the individual events,
that is:
P(A1∪A2∪⋯∪Ak)=P(A1)+P(A2)+⋯+P(Ak)

b.the probability of a countably infinite union of the events is the sum of the probabilities of the
individual events, that is:
P(A1∪A2∪⋯)=P(A1)+P(A2)+⋯

19
Example
Status Fresh Soph Jun Sen Grad Total
Count 1 4 20 9 9 43
Proportion 0.02 0.09 0.47 0.21 0.21
Randomly select one student from the class.
Defining the following events:
•Fr = the event that a Freshman is selected
•So = the event that a Sophomore is selected
•Ju = the event that a Junior is selected
•Se = the event that a Senior is selected
•Gr = the event that a Graduate student is selected
The sample space is S = (Fr, So, Ju, Se, Gr}. Using the relative
frequency approach to assigning probability to the events:
•P(Fr) = 0.02
•P(So) = 0.09
•P(Ju) = 0.47
•P(Se) = 0.21
•P(Gr) = 0.21 20
Theorems of probability
P(A) = 1 − P(A').

21
Theorems of probability
• P(Ø) = 0.

22
Theorems of probability
• If events A and B are such that A ⊆ B, then P(A) ≤ P(B).

23
Theorems of probability
• P(A) ≤ 1.

24
Theorems of probability
• For any two events A and B, P(A ∪ B) = P(A) + P(B) − P(A ∩ B).

25
Example
• A company has bid on two large
construction projects. The company
president believes that the probability of
winning the first contract is 0.6, the
probability of winning the second contract is
0.4, and the probability of winning both
contracts is 0.2.
• What is the probability that the company
wins at least one contract?
• What is the probability that the company
wins the first contract but not the
second contract?
• What is the probability that the company
wins neither contract?
• What is the probability that the company
wins exactly one contract?

26
Example
• f it is known that A ⊆ B, what can be definitively said about P(A ∩ B)?

27
Example
• If 7% of the population smokes cigars, 28% of the population smokes
cigarettes, and 5% of the population smokes both, what percentage
of the population smokes neither cigars nor cigarettes?

28
Multiplication principle
A Six and a head S = {1H, 1T, 2H, 2T, ...}
6 x 2 = 12 possible outcomes

P(A) = N(A) / N(S)

The Multiplication Principle. If there are:

n1 outcomes of a random experiment E1
n2 outcomes of a random experiment E2
... and ...
nm outcomes of a random experiment Em
then there are n1 × n2 × ... × nm outcomes of the composite experiment E1E2 ... Em.

29
Permutations
• Suppose there are n positions to be filled with n different objects, in which
there are:
• n choices for the 1st position
• n − 1 choices for the 2nd position
• n − 2 choices for the 3rd position
• ... and ...
• 1 choice for the last position
• The Multiplication Principle tells us there are then in general:
• n × (n − 1) × (n − 2) × ... × 1 = n!
• ways of filling the n positions.

Definition. A permutation of n objects is an ordered arrangement of the n objects.

Example: If there are 4 positions and 4 people to take them, for the first position there are 4 options, for the
next there are 3 options since the first one is already chosen, etc.
4x3x2x1 possible ways
30
Permutations
• Suppose there are r positions to be filled
with n different objects, in which there are:
• n choices for the 1st position
• n − 1 choices for the 2nd position A permutation of n objects taken r at a time
• n − 2 choices for the 3rd position nPr=n! / (n−r)!
• ... and ...
• n − (r − 1) choices for the last position
• The Multiplication Principle tells us there are in
general:
• n × (n − 1) × (n − 2) × ... × [n − (r − 1)]
• ways of filling the r positions.

Example:
If there are 4 people but 2 positions to take:
4x3 ways 31
Combinations
• Example:
• Maria has three tickets for
a concert. She'd like to
use one of the tickets
herself. She could then
offer the other two tickets
to any of four friends
(Ann, Beth, Chris,
Dave). How many ways
can 2 people be selected
from 4 to go to a concert?

32
Combinations

Definition. The number of unordered subsets, called a combination of n objects

taken r at a time, is:

We say “n choose r.”

The r represents the number of objects you'd like to select (without

replacement and without regard to order) from the n objects you have.

33
Example
• Twelve (12) patients are available for use in a research study. Only
seven (7) should be assigned to receive the study treatment. How
many different subsets of seven patients can be selected?

34
Example
• Let's use a standard deck of
cards containing 13 face values
(Ace, 2, 3, 4, 5, 6, 7, 8, 9, 10,
Jack, Queen, and King) and 4
different suits (Clubs,
Diamonds, Hearts, and
Spades) to play five-card
poker. If you are dealt five
cards, what is the probability
of getting a "full-house" hand
containing three kings and two
aces (KKKAA)?

35
Binomial coefficients

36
Distinguishable permutations

37
 Distinguishable permutations

distinguishable permutations of the n objects.

38
Distinguishable permutations

39
Conditional probability

• P(T+) = 54/137
• P(D) = 67/137
• If a person has renal disease, what is the probability that he/she tests
positive for the disease?
• P(T+ | D) = 44/67 = 0.65

40
Conditional Probability

41
Conditional probability
• Sensitivity
• If a person has renal disease, what is
the probability of testing positive?

• "positive predictive value" of a

diagnostic test
• If I receive a positive test, what is the
probability that I actually have the
disease?

42
Properties of conditional probability
• Because conditional probability is
just a probability, it satisfies the
three axioms of probability. That is,
as long as P(B) > 0:
• P(A | B) ≥ 0
• P(B | B) = 1
• If A1, A2, ... Ak are mutually
exclusive events, then P(A1 ∪ A2 ∪
... ∪ Ak| B) = P(A1 | B) + P(A2 | B) +
... + P( Ak| B) and likewise for
infinite unions.
43
 Examples
• A box contains 6 white balls and 4 red
balls. We randomly (and without
replacement) draw two balls from the
box. What is the probability that the
second ball selected is red, given that
the first ball selected is white?

• What is the probability that both balls

selected are red?

44
Multiplication rule
• Definition. The probability that two events A and B both occur is
given by the multiplication rule as:
• P(A ∩ B) = P(A | B) × P(B)
• or by:
• P(A ∩ B) = P(B | A) × P(A)

45
Example
• A box contains 6 white
balls and 4 red balls. We
randomly (and without
replacement) draw two
balls from the box. What is
the probability that the
second ball selected is red?

46
Extended multiplication rule

47
Example
• Medical records reveal that of the 937 men who died in a particular region in 1999:
• 212 of the men died of causes related to heart disease,
• 312 of the men had at least one parent with heart disease
• Of the 312 men with at least one parent with heart disease, 102 died of causes related to
heart disease. Using this information, if we randomly select a man from the region, what
is the probability that he dies of causes related to heart disease given that neither of his
parents died from heart disease? If we define two events as such:
• Let H = the event that at least one of the parents of a randomly selected man died of
causes related to heart disease
• Let D = the event that a randomly selected man died of causes related to heart disease
• then we are looking for the following conditional probability:
• P(D | H')

48
Solution

49
Independent Events
• A couple plans to have three
children. What is the
probability that the second
child is a girl? And, what is
the probability that the
second child is a girl given
that the first child is a girl?

50
Independent events
• Definition. Events A and B are independent events if the occurrence
of one of them does not affect the probability of the occurrence of
the other. That is, two events are independent if either:
P(B|A) = P(B)
• (provided that P(A) > 0) or:
• P(A|B) = P(A)
• (provided that P(B) > 0).

51
Independent events

Now, since independence tells us that P(B|A) = P(B), we can

substitute P(B) in for P(B|A) in the formula given to us by the
multiplication rule:
P(A ∩ B) = P(A) × P(B |A)
yielding:
P(A ∩ B) = P(A) × P(B).

• Definition. Events A and B are independent events if and only

if :
• P(A ∩ B) = P(A) × P(B)
• Otherwise, A and B are called dependent events.
52
Theorems
• If A and B are independent events,
then the events A and B' are also
independent.
• If A and B are independent events,
then the events A' and B are also
independent.
• If A and B are independent events,
then the events A' and B' are also
independent.

53
Example
• A nationwide poll determines that 72% of the American population
loves eating pizza. If two people are randomly selected from the
population, what is the probability that the first person loves eating
pizza, while the second one does not?
• P(A ∩ B') = 0.72 × (1 − 0.72) = 0.202

54
Mutual independence
• Definition. Three events A, B, and C are mutually independent if and
only if the following two conditions hold:
• (1) The events are pairwise independent. That is,
• P(A ∩ B) = P(A) × P(B) and...
• P(A ∩ C) = P(A) × P(C) and...
• P(B ∩ C) = P(B) × P(C)
• (2) P(A ∩ B ∩ C) = P(A) × P(B) × P(C)

55
56
Bayes’ theorem
• A desk lamp produced by The Luminar Company was found to be
defective (D). There are three factories (A, B, C) where such desk lamps
are manufactured. A Quality Control Manager (QCM) is responsible for
investigating the source of found defects. This is what the QCM knows
about the company's desk lamp production and the possible source of
defects:
Factory % of total production Probability of defective lamps
A 0.35 = P(A) 0.015 = P(D | A)
B 0.35 = P(B) 0.010 = P(D | B)
C 0.30 = P(C) 0.020 = P(D | C)
• If a randomly selected lamp is defective, what is the probability that the
lamp was manufactured in factory C?
57
Bayes’ theorem

58
Bayes’ theorem
• if a randomly selected lamp is defective, what is the probability that
the lamp was manufactured in factory A? And, if a randomly selected
lamp is defective, what is the probability that the lamp was
manufactured in factory B?

59
Bayes’ theorem
• The probabilities P(A), P(B) and P(C) are often referred to as prior
probabilities, because they are the probabilities of events A, B, and C that
we know prior to obtaining any additional information. The conditional
probabilities P(A | D),P(B | D), and P(C | D) are often referred to
as posterior probabilities, because they are the probabilities of the
events after we have obtained additional information.
• As a result of our work, we determined:
• P(C | D) = 0.407
• P(B | D) = 0.237
• P(A | D) = 0.356

60