EEF 271E

PROBABILITY AND STATISTICS

FALL 2022

Lecture 05
Random Variables
Dr. Ramazan Çağlar

Lecture Notes 05 Random Variables 1

 Lesson Overview

 Random variable

 Probability Distribution Function or

Cumulative Distribution Function “cdf”

 Probability Density Function

 The Probability Mass Function

Lecture Notes 05 Random Variables 2

 RANDOM VARIABLES

Ω
ω4 ωn
ω1 ω2 …
ω3

X(ω1) X(ω2) X(ω3)= X(ω4) … X(ωn) Real line

A random variable (r.v.) is a function

from Ω to ℝ .

X : Ω → ℝ

3
Lecture Notes 05 Random Variables
 Why Random variable ?

 Often, in a random experiment, we are not

interested in outcomes themselves, rather in a
numerical attributes of the outcomes. (e.g., toss
five coins; what is the # of heads ?)
 Often, the outcomes of a random experiment
are themselves real numbers. (e.g., voltage or
temperature measurements).
 With random variables the ‘’new’’ sample space
is the real line, so we can manipulate Probability
related quantities easily (we will see soon. …)

Lecture Notes 05 Random Variables 4

 Random variable

 Note that the function X mapping Ω

to ℝ is not random, but a fixed, well
defined quantity (e.g., ‘’# of heads’’).

 The ‘’random’’ in a r.v. comes from

the outcomes of the underlying
sample space Ω being random.

Lecture Notes 05 Random Variables 5

 Events
 X maps the events in the original
sample space to subsets of ℝ .

A
XA

A = { ω: ω ∈ A }  XA = { ω: ω ∈ XA }

Subset of Ω Subset of ℝ

A = { ω: ω ∈ XA }
Lecture Notes 05 Random Variables 6
Probability

The Notion of equivalent events helps us define

probability in the context of a random variable.
P({ω: ω ∈ A }) = P({ω: ω ∈ XA }) = P(A)
Therefore, we can assign a probability directly to XA:
P(XA) = P(A)

Note that strictly speaking, we need to use different

symbols to denote these two probability functions;
they are different function because their domains of
definition are different!

Lecture Notes 05 Random Variables 7

Probability

We will no longer make any mention of

this careful distinction. Instead, we will
just use of
P(X ∈ XA ) or
P( 5 ≤ X ≤ 7) or
P(XA) … etc.

Lecture Notes 05 Random Variables 8

 Probability Distribution Function
or
Cumulative Distribution Function ‘’cdf’’

The cumulative distribution function for a real

random variable X is defined as:
𝑭𝑿 𝒙 = 𝑷(𝑿 ≤ 𝒙)
Given x, 𝑭𝑿 𝒙 is simply the probability that the
random variable X lies in the interval (-∞, x ].
(-∞, x ]
-∞ 𝒙 +∞
Random Variables
Lecture Notes 05 9
 Properties of the cdf

(1) 0 ≤ 𝐹𝑋 𝑥 ≤ 1) for all 𝑥 ∈ ℝ

(2) lim 𝐹𝑋 𝑥 = 1
𝑥 → +∞

(3) lim 𝐹𝑋 𝑥 = 0
𝑥 → −∞

(4) If 𝑥2 ≥ 𝑥1 ,
𝐹𝑋 𝑥1 ≤ 𝐹𝑋 𝑥2
‘’ 𝐹𝑋 𝑥 is a non-decreasing function ’’

Random Variables
Lecture Notes 05 10
 Properties of the cdf

(5) 𝐹𝑋 𝑥 is right continuous function

lim 𝐹𝑋 𝑏 + ℎ = 𝐹𝑋 𝑏
ℎ→0
ℎ>0

(6) 𝑃 𝑎 < 𝑥 ≤ 𝑏 = 𝐹𝑋 𝑏 − 𝐹𝑋 (𝑎)

(7) 𝑃 𝑋 = 𝑏 = 𝐹𝑋 𝑏 − 𝐹𝑋 𝑏 −
The ‘’jump in the cdf at 𝑥 = 𝑏, gives
𝑃 𝑋=𝑏

Random Variables
Lecture Notes 05 11
 Properties of the cdf

𝑃 𝑋 = 𝑏 = 𝐹𝑋 𝑏 − 𝐹𝑋 𝑏 −
The ‘’jump in the cdf at 𝑥 = 𝑏, gives 𝑃 𝑋 = 𝑏

Example:
𝑭𝑿 𝒙
1
𝐹𝑋 3
𝐹𝑋 2 , 𝐹𝑋 3−
𝐹𝑋 2−
𝐹𝑋 0
𝐹𝑋 −2

−2 −1 0 𝒙
1 2 3
𝑷 𝑿 ≤ −𝟐 𝑷 𝑿≤𝟎 𝑷 𝑿=𝟐 𝑷 𝑿=𝟑
𝐹𝑋 −2 𝐹𝑋 0 𝐹𝑋 2 − 𝐹𝑋 2− 𝐹𝑋 3 − 𝐹𝑋 3−
Random Variables
Lecture Notes 05 12
Example:

𝐹𝑋 𝑥 = 0 −∞<𝑥 ≤0
𝐹𝑋 𝑥 = 1 − 𝑒 −𝑥 0<𝑥≤∞
or with the another representation
This is the
0 −∞ < 𝑥 ≤ 0 ‘’exponential
𝐹𝑋 𝑥 = ቊ
1 − 𝑒 −𝑥 0<𝑥≤∞ distribution’’.

(a) What is 𝑃 𝑋 > 0.5 ?

𝑃 𝑋 ≤ 0.5 + 𝑃 𝑋 > 0.5 = 1.0
𝑃 𝑋 > 0.5 = 1.0 − 𝑃 𝑋 ≤ 0.5
𝑃 𝑋 > 0.5 = 1.0 − 𝐹𝑋 0.5
𝑃 𝑋 > 0.5 = 1.0 − 1 − 𝑒 −0.5 = 𝑒 −0.5 = 0.6065
Random Variables 13
Lecture Notes 05
Example:

0 −∞ < 𝑥 ≤ 0
𝐹𝑋 𝑥 = ቊ
1 − 𝑒 −𝑥 0<𝑥≤∞

(b) What is 𝑃 𝑋 ≤ 0.25 ?

𝑃 𝑋 ≤ 0.25 = 1 − 𝑒 −0.25

(c) What is 𝑃 0.3 < 𝑋 ≤ 0.7 ?

𝑃 0.3 < 𝑋 ≤ 0.7 = 𝐹𝑋 0.7 − 𝐹𝑋 0.3
= 1 − 𝑒 −0.7 − 1 − 𝑒 −0.3
= 𝑒 −0.3 − 𝑒 −0.7 = 0.2442

Random Variables
Lecture Notes 05 14
Example:
Expt: Toss a fair coin three times, and count number of
heads. Here, the random variable is the number of heads.
Ω = { HHH, HHT, THH, HTH, HTT, TTH, THT, TTT }
# of heads { 3, 2, 2, 2, 1, 1, 1, 0}

X = Number of heads in three tosses of a fair coin.

X = { 0, 1, 2, 3}
1 3
𝑃 𝑋=0 = 𝑃 𝑋=1 =
8 8
3 1
𝑃 𝑋=2 = 𝑃 𝑋=3 =
8 8
15
Lecture Notes 05 Random Variables
 X = Number of heads in three tosses of a fair coin.

Outcomes Probability

1/2 H 3H 1/8

H T 2H 1/8
1/2
1/2
H 2H 1/8
1/2
H
1/2 1/2 T
T 1H 1/8
1/2

1/2 H 2H 1/8
1/2
1/2 H
T
1/2 T 1H 1/8

1/2 T 1/2 H 1H 1/8

1/2 T 0H 1/8

P(H=0) = 1/8 P(H=1) = 3/8 P(H=2) = 3/8 P(H=3) = 1/8

Lecture Notes 05 Random Variables 16

Example:
Expt: Toss a fair coin three times, and count number of
heads. X = { 0, 1, 2, 3}
𝑃 𝑋 = 0 = 1/8 𝑃 𝑋 = 1 = 3/8
𝑃 𝑋 = 2 = 3/8 𝑃 𝑋 = 3 = 3/8
P(X = x) P(X x)
P(X 8) =8/8 = 1
8/8
P(X 2) =7/8 P(X = 3) =1/8
7/8

6/8
P(X = 2) =3/8 The cdf is “ piecewise
5/8
P(X 1) =4/8
constant or shows a
4/8 “staircase behavior”.
3/8 Random variables with
2/8
P(X = 1) =3/8
such cdfs are called
discrete random variables.
1/8
P(X = 0) =1/8
x
0 1 2 3

Random Variables
Lecture Notes 05 17
Example:
Exponential cumulative distribution function (cdf)

0 𝑖𝑓 𝑥 < 0
𝐹𝑋 𝑥 = ቊ
1 − 𝑒 −𝑥 𝑖𝑓 𝑥 ≥ 0
FX (x)

1.0

x
0.0

The cdf is a continuous function. Random variables

with continuous cdf will be called continuous random
variables.
Lecture Notes 05 Random Variables 18
Example:

A random variable W is the waiting time in a queue

(note that W is non-negative!)

𝟎 if the system is idle (immediate service)

𝑊=ቊ
𝟏 − 𝒆−𝝀𝒙 otherwise

Lecture Notes 05 Random Variables 19

 𝑃 𝑊 ≤ 𝑥 = 𝑃 𝑊 ≤ 𝑥 | System idle 𝑃 System idle
+𝑃 𝑊 ≤ 𝑥 | System not idle 𝑃 System not idle

Suppose that

𝑃 System idle = 𝑝 is given, therefore

𝑃 System not idle = 1 − 𝑝 then

0 𝑖𝑓 𝑥 < 0
𝐹𝑊 𝑥 =൝
𝑝 + (1 − 𝑝) 1 − 𝑒 −𝜆𝑥 𝑖𝑓 𝑥 ≥ 0

Lecture Notes 05 Random Variables 20

 FX (x)

1.0

0.0 x

This random variable is neither discrete r.v. nor

a continuous r.v. Such random variables are
called mixed random variables.

Lecture Notes 05 Random Variables 21

 Probability Density Function (pdf)

This is a nonnegative function, denoted 𝒇𝑿 𝒙 ,

whose integral is cumulative distribution
function.
𝒙

𝑭𝑿 𝒙 = න 𝒇𝑿 𝒕 𝒅𝒕
−∞
𝒃

𝑭𝑿 𝒃 −𝑭𝑿 𝒂 = 𝑷 𝒂 ≤ 𝑿 ≤ 𝒃 = න 𝒇𝑿 𝒕 𝒅𝒕
𝒂

Lecture Notes 05 Random Variables 22

 Probability Density Function (pdf)

When 𝑭𝑿 𝒙 is differentialable at x,
we obviously have
𝒅
𝒇𝑿 𝒙 = 𝑭𝑿 (𝒙)
𝒅𝒙

Lecture Notes 05 Random Variables 23

 Probability Density Function
Why density ?
𝑷 𝒙 < 𝑿 ≤ 𝒙 + ∆𝒙 = 𝑭𝑿 𝒙 + ∆𝒙 −𝑭𝑿 𝒙
∆𝒙
= 𝑭𝑿 𝒙 + ∆𝒙 −𝑭𝑿 𝒙
∆𝒙
𝑭𝑿 𝒙 + ∆𝒙 −𝑭𝑿 𝒙
= ∆𝒙
∆𝒙

𝑭𝑿 𝒙+𝒉 −𝑭𝑿 𝒙 𝒅
As x  0, lim = 𝑭 𝒙 , therefore
∆𝒙→𝟎 ∆𝒙 𝒅𝒙 𝑿
𝒅
𝑷 𝒙 < 𝑿 ≤ 𝒙 + ∆𝒙 = ∆𝒙 ∙ 𝑭𝑿 𝒙
𝒅𝒙
= ∆𝒙 ∙ 𝒇𝑿 𝒙
Lecture Notes 05 Random Variables 24
 fX (x)
Probability that X [ x, x +x ]

x
x
x x+x
Thus
P(X is in an interval of with x, in the vicinity of x ) x . fX (x)

Lecture Notes 05 Random Variables 25

 Properties of the pdf

(1) 𝑓𝑋 𝑥 ≥ 0 for all 𝑥 ∈ ℝ

𝑏

𝑃 𝑎 < 𝑋 ≤ 𝑏 = න 𝑓𝑋 𝑡 𝑑𝑡
(2)
𝑎+
Careful with the limits !
+∞

(3) න 𝑓𝑋 𝑡 𝑑𝑡 = 1.0
−∞
Important note: For a continuous random variable X we have
for any a and b.
𝑃 𝑎≤𝑋≤𝑏 =𝑃 𝑎≤𝑋<𝑏 =𝑃 𝑎<𝑋≤𝑏 =𝑃 𝑎<𝑋<𝑏
Because 𝑃 𝑋 = 𝑥 = 0 for any 𝑥 ∈ ℝ
Random Variables
Lecture Notes 05 26
 Independence of two random variables

For two events A and B:

𝑃 𝐴 ∩ 𝐵 = 𝑃 𝐴 𝑃(𝐵)

Two random variables X and Y

𝐹𝑋,𝑌 𝑥, 𝑦 = 𝐹𝑋 𝑥 𝐹𝑌 (𝑦)

IID : Independent and Identically Distributed

Random Variables
Lecture Notes 05 27
Example:

Exponential cumulative distribution function (cdf)

FX (x)
1.0
0 𝑖𝑓 𝑥 < 0
𝐹𝑋 𝑥 = ൝
1 − 𝑒 −𝜆𝑥 𝑖𝑓 𝑥 ≥ 0
x
0.0

Exponential density function (pdf)

fX (x)
λ

0 𝑖𝑓 𝑥 < 0
𝑓𝑋 𝑥 = ൝
𝜆𝑒 −𝜆𝑥 𝑖𝑓 𝑥 ≥ 0
x

Lecture Notes 05 Random Variables 28

Example:
X = No. Of heads in 3 tosses of a fair coin.
Recall that cdf looks like:
P(X = xi) P(X x)
8/8

7/8

6/8

5/8

4/8

3/8

2/8

1/8

x
0 1 2 3

Lecture Notes 05 Random Variables 29

Example:
To get the pdf, we need to differentiate FX(x),
but what about the discontinuities ?
Handle these using the Dirac delta function
(δ function). Sometimes, it is only named as the
delta function. …
P(X = xi) P(X x)
8/8

7/8

6/8

5/8

4/8

3/8

2/8 discontinuities
1/8

x
0 1 2 3

Lecture Notes 05 Random Variables 30

The Dirac delta function

1.0

0.5

0.0

- 0.2
-1.0 0.0 1.0

Schematic representation of the Dirac delta

fuction by a line surmounted by an arrow.
The height of the arrow is usually meant to
specify the value of any multiplicative constant,
which will give the area under the function. The
other convention is to write the area next to the
arrowhead. Source: WikipediA.

Lecture Notes 05 Random Variables 31

The Dirac delta function

Recall that δ(.) satisfies

(1) lim δ t = ∞ ‘‘infinite amplitude’’.

t→∞

(2) δ t = 0, for t ≠ 0 ‘‘zero duration’’.

+𝜖

(3) න 𝛿 𝑡 𝑑𝑡 = 1.0 for any 𝜖 > 0 ‘‘unit mass’’

−𝜖

Lecture Notes 05 Random Variables 32

Then, pdf for our example can be given as follows:
P(X = xi)
8/8

7/8

6/8
P(X = xi)

5/8 4/8

4/8 3/8

3/8 2/8

2/8
1/8

1/8 x
0 1 2 3
x
0 1 2 3

The ‘’jump in the cdf at 𝑥 = 𝑥𝑖 Thus, for discrete random

gives 𝑃 𝑋 = 𝑥𝑖 , these are the variables, the pdf is a train
individual probabilities of the of impulses
discrete rvs.

Lecture Notes 05 Random Variables 33

 Then, the sketches of pdf for our example is
given below with two kind of representations,
one uses the arrows and the other uses the
dots with dashed lines.

P(X = xi) P(X = xi)

3/8 3/8

(area) 1/8 1/8

x x
0 1 2 3 0 1 2 3

the pdf is a train of impulses the pdf is a train of dots

with dashed lines

Lecture Notes 05 Random Variables 34

Example: Uniform random variable
Cumulative distribution function, cdf
0 𝑥≤𝑎 cdf
𝑥−𝑎 1
𝐹𝑋 𝑥 = 𝑎≤𝑥≤𝑏
𝑏−𝑎
1 𝑥≥𝑏 a b

Probability densisty function, pdf

pdf
0 𝑥 < 𝑎 or 𝑥 > 𝑏
𝑓𝑋 𝑥 = ቐ 1
𝑎≤𝑥≤𝑏
𝑏−𝑎 a b

We see from the example that for continuous random

variables, the pdf may not be continuous ! But the pdf
cannot have any delta (impulsive) terms.
Lecture Notes 05 Random Variables 35
 Example: Mixed type random variable

Cumulative distribution function, cdf

FX (x)

1.0

0 𝑖𝑓 𝑥 < 0
𝐹𝑊 𝑥 =൝
𝑝 + (1 − 𝑝) 1 − 𝑒 −𝜆𝑥 𝑖𝑓 𝑥 ≥ 0 p

0.0 x

Probability densisty function, pdf

0 𝑥<0
𝑓𝑋 𝑥 = ቊ
𝑝𝛿 𝑥 + 𝜆𝑒 −𝜆𝑥 (1 − 𝑝) 𝑥 ≥ 0
x

(It has both impulse terms and smooth terms;

neither discrete nor continuous.)
Lecture Notes 05 Random Variables
36
The Probability Mass Function
A simpler description of probabilities for a discrete rv.

Let X be a discrete random variable with

𝑃 𝑋 = 𝑥𝑗 = 𝑝𝑗
Of course 𝑝𝑗 ≥ 0 and
෍ 𝑝𝑗 = 1
𝑗
Then the Probability mass function (pms) of X is denoted
𝓟𝑿 and defined as follows:
𝓟𝑿 : 𝑥1 , 𝑥2 , … → 0, 1 with
𝒫𝑋 𝑥𝑗 = 𝑝𝑗

Lecture Notes 05 Random Variables 37

 The pms comes in handy, when it is necessary to write down the
description of cdf and pdf of a discrete random variable:

Let u be the usual unit step function.

0, 𝑥<0
𝒖 𝑥 =ቊ
1, 𝑥≥0

and let d be the usual delta function.

Then
𝐹𝑋 𝑥 = ෍ 𝒑𝑿 (𝑥𝑗 ) 𝒖(𝑥 − 𝑥𝑗 )
𝑗
and
𝑓𝑋 𝑥 = ෍ 𝒑𝑿 (𝑥𝑗 ) 𝜹(𝑥 − 𝑥𝑗 )
𝑗

Lecture Notes 05 Random Variables 38

Example:

X : Number of heads in tossing two coins

then possible outcomes are {0, 1, 2}.
From the definition of the pms,
1
𝑝𝑋 0 = 4
1
𝑝𝑋 1 = 2
1
𝑝𝑋 2 = 4
1 1 1
𝑭𝑿 𝒙 = 4 𝒖 𝑥 + 2 𝒖 𝑥 − 1 + 4 𝒖 𝑥 − 2
1 1 1
𝒇𝑿 𝒙 = 4 𝜹 𝑥 + 2 𝜹 𝑥 − 1 + 4 𝜹(𝑥 − 2)

Lecture Notes 05 Random Variables 39