Probability Theory

Probability – Models for random

phenomena
 Phenomena

Non-deterministic
Deterministic
Deterministic Phenomena
• There exists a mathematical model that allows
“perfect” prediction the phenomena’s
outcome.
• Many examples exist in Physics, Chemistry
(the exact sciences).
Non-deterministic Phenomena
• No mathematical model exists that allows
“perfect” prediction the phenomena’s
outcome.
Non-deterministic Phenomena
• may be divided into two groups.

1. Random phenomena
– Unable to predict the outcomes, but in the long-
run, the outcomes exhibit statistical regularity.

2. Haphazard phenomena
– unpredictable outcomes, but no long-run,
exhibition of statistical regularity in the
outcomes.
 Phenomena

Non-deterministic

Deterministic
Haphazard

Random
Haphazard phenomena
– unpredictable outcomes, but no long-run,
exhibition of statistical regularity in the
outcomes.
– Do such phenomena exist?
– Will any non-deterministic phenomena exhibit
long-run statistical regularity eventually?
Random phenomena
– Unable to predict the outcomes, but in the long-
run, the outcomes exhibit statistical regularity.

Examples
1. Tossing a coin – outcomes S ={Head, Tail}
Unable to predict on each toss whether is Head or
Tail.
In the long run can predict that 50% of the time
heads will occur and 50% of the time tails will occur
2. Rolling a die – outcomes
S ={ , , , , , }

Unable to predict outcome but in the long run can

one can determine that each outcome will occur 1/6
of the time.
Use symmetry. Each side is the same. One side
should not occur more frequently than another side
in the long run. If the die is not balanced this may
not be true.
Definitions
 The sample Space, S
The sample space, S, for a random phenomena
is the set of all possible outcomes.
Examples
1. Tossing a coin – outcomes S ={Head, Tail}

2. Rolling a die – outcomes

S ={ , , , , , }

={1, 2, 3, 4, 5, 6}
 An Event , E
The event, E, is any subset of the sample space,
S. i.e. any set of outcomes (not necessarily all
outcomes) of the random phenomena
Venn
S diagram
E
The event, E, is said to have occurred if after
the outcome has been observed the outcome lies
in E.

S
E
Examples

1. Rolling a die – outcomes

S ={ , , , , , }
={1, 2, 3, 4, 5, 6}

E = the event that an even number is

rolled
= {2, 4, 6}
={ , , }
 Special Events
The Null Event, The empty event - f

f = { } = the event that contains no outcomes

The Entire Event, The Sample Space - S
S = the event that contains all outcomes

The empty event, f , never occurs.

The entire event, S, always occurs.
 Set operations on Events
Union
Let A and B be two events, then the union of A
and B is the event (denoted by AB) defined by:
A  B = {e| e belongs to A or e belongs to B}

AB

A B
The event A  B occurs if the event A occurs or
the event and B occurs .

AB

A B
Intersection

Let A and B be two events, then the intersection

of A and B is the event (denoted by AB) defined
by:
A  B = {e| e belongs to A and e belongs to B}

AB

A B
The event A  B occurs if the event A occurs and
the event and B occurs .

AB

A B
Complement

Let A be any event, then the complement of A

(denoted by A ) defined by:

A = {e| e does not belongs to A}

A
A
The event A occurs if the event A does not
occur

A
A
In problems you will recognize that you are
working with:

1. Union if you see the word or,

2. Intersection if you see the word and,
3. Complement if you see the word not.
Definition: mutually exclusive
Two events A and B are called mutually
exclusive if:
A B 

A B
If two events A and B are are mutually
exclusive then:
1. They have no outcomes in common.
They can’t occur at the same time. The outcome
of the random experiment can not belong to both
A and B.

A B
Probability
Definition: probability of an Event E.
Suppose that the sample space S = {o1, o2, o3, …
oN} has a finite number, N, of oucomes.
Also each of the outcomes is equally likely
(because of symmetry).
Then for any event E
n E  nE  no. of outcomes in E
P E =  
n S  N total no. of outcomes
Note : the symbol n  A  = no. of elements of A
Thus this definition of P[E], i.e.

nE  nE  no. of outcomes in E

P E =  
n S  N total no. of outcomes

Applies only to the special case when

1. The sample space has a finite no.of
outcomes, and
2. Each outcome is equi-probable
If this is not true a more general definition
of probability is required.
Rules of Probability
Rule The additive rule
(Mutually exclusive events)
P[A  B] = P[A] + P[B]
i.e.
P[A or B] = P[A] + P[B]

if A  B = f
(A and B mutually exclusive)
If two events A and B are are mutually
exclusive then:
1. They have no outcomes in common.
They can’t occur at the same time. The outcome
of the random experiment can not belong to both
A and B.

A B
P[A  B] = P[A] + P[B]
i.e.
P[A or B] = P[A] + P[B]

A B
Rule The additive rule
(In general)

P[A  B] = P[A] + P[B] – P[A  B]

or
P[A or B] = P[A] + P[B] – P[A and B]
Logic A B
A B

A B

When P[A] is added to P[B] the outcome in A  B

are counted twice
hence
P[A  B] = P[A] + P[B] – P[A  B]
 P  A  B   P  A  P  B   P  A  B 

Example:
DSU and Iqra are two of the universities competing for
the University Olympics. (There are also many others).
The organizers are narrowing the competition to the
final 5 universities.
There is a 20% chance that DSU will be amongst the
final 5. There is a 35% chance that Iqra will be amongst
the final 5 and an 8% chance that both DSU and Iqra
will be amongst the final 5. What is the probability that
DSU or Iqra will be amongst the final 5.
Solution:
Let A = the event that DSU is amongst the final 5.
Let B = the event that Iqra is amongst the final 5.
Given P[A] = 0.20, P[B] = 0.35, and P[A  B] = 0.08
What is P[A  B]?
Note: “and” ≡ , “or” ≡  .
P  A  B   P  A  P  B   P  A  B 
 0.20  0.35  0.08  0.47
 Rule for complements

2. P  A   1  P  A

or
P not A  1  P  A
Complement

Let A be any event, then the complement of A

(denoted by A ) defined by:

A = {e| e does not belongs to A}

A
A
The event A occurs if the event A does not
occur

A
A
Logic:
A and A are mutually exclusive.
and S  A  A

A
A

thus 1  P  S   P  A  P  A 
and P  A   1  P  A