Probability Theory and Random

Processes – Lecture-01
Course code: ECE-215

UNIT-1
Probability and Random Variables

Title :
The axiomatic definition of probability
 Outline
• Introduction
• Set –definitions
• Set- operations
• Probability- axiomatic definition through
sets.
 Introduction
• Engineering systems are designed to operate well in the face
of uncertainty
• Examples
– Amplifiers – Maintains a specified “GAIN”
• Inspite of manufacturing variations in its components.
• Inspite of ambient temperature variations .
– The Power distribution grid- Provides
• fluctuation-free voltage to homes.
• Inspite of unpredictable load variations.

– Communication systems- reliably transfer information

• Inspite of noise.
 Introduction…
• How do they achieve it? • Three steps in modeling uncertainty
– handle the various uncertainties • Step1: defining the idea of a
• First step is sample space -‘S’.
• Step2: defining the idea of an
to mathematically model it
event -‘A’.
• The idea of a “random experiment” • Step3: Assignment of a
–basis for a mathematical model of “Probability” P(A) (likelihood
uncertainty number) to each of such events A,
such that it follows three basic
• The three axioms form the basis of rules/ axioms.
probability theory. • Axiom1:
P( A)  0
• The ideas of
• Axiom2:
• sample space
• Event and
P( S )  1
• Axiom3:
• The axioms
N N
Are best understood using P (  An )   P( An ) if Am  An  
n 1
set theory n 1
 Set- definitions
• Set – a collection of objects (elements). • subset
• If a is an element of set A, then we denote • Every element of one set A is
also an element of another set B
a A
• If a is not an element of A, then we denote A B
a A • and a proper subset.
• Specifying a set
– Tabular method:
• If at least one element exists in B
{2, 3, 5, 7, 11, 13, 17, 19} which is not in A
– Rule method:
{ prime numbers between 1 and 20}
A B
• Disjoint or mutually exclusive sets.
• Countable and uncountable sets.
– Eg: is uncountable.
{1  c  20} • The universal set – S
• All-encompassing set of objects
• The null set .
 under discussion in a given
situation
• Finite and infinite sets- Countable
elements with the counting process
terminating.
 Set- Operations
• Equality • Algebra of sets-Commutative law
A  B if A  B and B  A A B  B  A A B  B  A
• Difference • Distributive law
A  B  { all elements of A
A  ( B  C )  ( A  B)  ( A  C )
that are not present in B}
A  ( B  C )  ( A  B)  ( A  C )
• Union / sum • Associative law
C  A B ( A  B)  C  A  ( B  C )  A  B  C
• Intersection /product ( A  B)  C  A  ( B  C )  A  B  C
D  A B
A  B    A, B mutually • De Morgan’s Laws: The complement of
exclusive
a union (intersection) of two sets A and B
• For N sets An , n  1,2,3.....N
N
is the intersection (union) of their
C  A1  A2  ...  AN   An complements
n 1
• ( A  B)  A  B , ( A  B)  A  B
N
D  A1  A2  ...  AN   AN
n 1
• Duality Principle: In an identity on sets if
• complement A  S  A Unions replace intersections , intersections
replace unions, S replaces null set and null set
replaces S, then the identity is preserved.
 Axiomatic definition of probability-revisit
• Three steps in modeling uncertainty • Sample space – the set of all possible
• Step1: defining the idea of a outcomes in a random experiment
sample space -‘S’. • Event- a subset of the sample space
• Step2: defining the idea of an • The three axioms
event -‘A’. – Axiom1:
• Step3: Assignment of a – we desire to work with non-negative numbers

“Probability” P(A) (likelihood

number) to each of such events A, – Axiom2:
such that it follows three basic – Normalization
rules/ axioms. – Recognizes that the sample space itself is an
event
• Axiom1:
P( A)  0 – Should have the highest probability since it is
all-encompassing.
• Axiom2: – Arbitrarily selected as unity.
P( S )  1
• Axiom3: – Axiom3:
– Additivity
N N
– The probability of the event equal to the union
P (  An )   P( An ) if Am  An   of any number of mutually exclusive events is
n 1 n 1 equal to the sum of the individual event
probabilities
• Any queries?

Thank you