Probabilistic Methods in

Engineering
Lecture 17
Dr. Maha Ali
• Functions of Random
Variables
• Joint PDF
• Conditional PDF
• Bayes’ formula for
continuous R.V.
 Gaussian random variable
• A Gaussian (also called standard Gaussian, standard normal) random variable X with mean 0 and
variance 1 is given by

• The cdf does not have a closed form and is denoted by Φ(x), i.e.,

• It appears in many natural situations.

• The average of many independent samples of the same random variable tends to the Gaussian
distribution (height of people, exam scores in a large class, noise, etc).

• The noise is often modeled as a Gaussian random variable since it is the sum of many other
small random variables such as thermal motion of electrons.
Gaussian random variable
Gaussian Random Variable cont.
Φ(x) vs Q(x)

Gaussian Distribution: 1σ has probability 0.6827

The central limit theorem and the Gaussian
approximation
• The Gaussian distribution arises frequently in practice, because of the
phenomenon known as the central limit theorem (CLT), and the associated
Gaussian approximation. There are many mathematical formulations of the CLT
which differ in various details, but the main idea is the following: If many
independent random variables are added together, and if each of them is small
in magnitude compared to the sum, then the sum has an approximately
Gaussian distribution.

Normal random variables play an important role in a broad

range of probabilistic models. The main reason is that.
generally speaking, they model well the additive effect of many
independent factors in a variety of engineering, physical. and
statistical contexts. Mathematically, the key fact is that the sum
of a large number independent identically distributed (not
necessarily normal) random variables has an approximately
normal CDF, regardless of the distribution of the individual
Random variable
Example
CO
• A binary message is transmitted as a signal s, which is either - l or + 1. The communication
channel corrupts the transmission with additive normal noise with mean μ = 0 and
variance σ2
• The receiver concludes that the signal -1 (or + 1) was transmitted if the value received is <
0 (or ≥0, respectively); see Figure What is the probability of error?

Solution:

1 V

Receive S N
if voltage o y
volage Co o
C
P error for S N2o PCN 1 1 PCN 1
PC
In's 15
Function of random variable
• Often in engineering problems we are given (or can make a good
assumption about) one random variable, call it X but want to
determine the density of a function of the random variable Y, where
Y = g(X).
For example, we measure the voltage V , and we want to analyze the
power P = V2/R
In such a case we often follow the following procedure:
Step 1: Determine the distribution, FY(y), of Y in terms of the distribution
of X.
Step 2: Take the derivative of FY(y) to obtain fY(y).
Functions of Random Variables cont.
• Given a random variable X with PDF fX(x) and CDF FX(x), and
supposing that Y = g(X) for some function g, what are fY (y) and FY (y)?
• General principle:
• For simplicity, we assume that g is monotonically increasing. In this
case, the CDF of Y can be determined as follows.

P Y y
Fy y
but Y 9 x
P GLX y
Fy b
 Functions of Random Variables cont.

FIs FIL Lx E de

f 5
Fy s day Fx g s

EÉ T
Example
• Let X be a random variable with PDF fX(x) and CDF FX(x). Supposing
that Y = X2 , find fY (y) and FY (y). Express the answers in terms of fX(x)
and FX(x).

F 7 P Y y P X y

PER X F
p a X b
Fca 516 Fx B
Fyly Fx B
the PDF
fy s Fyls FIB Facts
first.fi B
Follow Up. (Square of a uniform random variable)
• Suppose X is a uniform random variable in [a, b] (assume a > 0), and
let Y = X2 . Then using the results from the previous example, the CDF
and PDF of Y are respectively
b
FIN If
a Hx v
uniform R

Fy s Fx T FL Ts
714
III a y b
Function of a Random Variable Summary.
• Let X be a random variable and Y = g(X).
• For a general continuous function ,X= g -1(y) may have several answers.
• For instance, when g(x) = x2 , then for y > 0, g -1 (y) = √y or − √y.

• For a given y, let x1, x2, . . . , xk denote all the answers to g -1 (y). Then the
relation between the pdf of Y and the pdf of X is as follows:

9 s
 Joint PDF
• The continuous version of the joint PMF is called the joint probability
density function (joint PDF), denoted by fX,Y (x, y).

• Let X and Y be two continuous random variables. The joint PDF of X

and Y is a function fX,Y (x, y) that can be integrated to yield a
probability

P A 45 da dy

A city Ry
Joint PDF cont. IIIa
Y
d
e

A joint PDF for a pair of continuous random variables is a surface in the 2D plane. To measure the size of the event A, we
integrate fX,Y (x, y) inside A.

P X b c Y d
a
fx yens dady
1 8 fan 5 dads 1
Event Examples: Joint Event is a region in the 2D plain

Example : GPS Event{ 10 km away from home}

Example of Joint PMF
aisles
Anylast 7s

0 otherwise

A CS

ff_fff
dsstcidoaifftx.in
dads

Hands 41yd
Example/iClicker RE
• Consider a uniform joint PDF fX,Y (x, y) defined on [0, 2] with f
2
X,Y (x, y)
= 1/ 4 .
let B = {X + Y ≤ 2}. Find P[B].
iClicker: what are the limits of the integration

Lay dads YES

B

f dxds 4 1 dy
Marginal PDF

I
fr
4 st

151
fy fry fay I
 Marginal CDF
fSGfdffedfth
• Let X and Y be two random variables. The marginal CDF is
fossorial

Fy Ms
f tx y 157 dads

P X xAY

tx NY
0 Fx
Marginal CDF

x x
F x
Fay
Fy b
Fx y
x Y

If
b da da
get
Fyi aug 1
8 41 Fx x
Conditional PDF
• The conditional PDF fXIA of a continuous random variable X, given an event
A with P(A) > 0, satisfies

P X CB A fx Alt dx

• If A is a subset of the real line with P(X ∈ A) > 0, then

a if cA

a otherwise
• Let A1, A2, . .. , An be disjoint events that form a partition of the sample
Space

totalprob then Ix X P Ai fx Afx

a version of
Conditioning one Random Variable on Another
• Let X and Y be continuous R.V. with joint pdf fX.Y
• with fY(y) > O. Then the conditional PDF of X given that Y= y, is defined
by

1 4157
1 ly
fyl5 f fxigtxisldx
If fxy
for any
b
fixed
dx
5
A
Example
• Let X and Y be two continuous random variables with a joint PDF

• Find the conditional PDFs fX|Y (x|y) and fY|X(y|x). fiiiE.li

In t.fi
f
é é
Ly s
fay Lau dx dx
Solution cont.

é
fy s s e dx 2 e

fay last
É s

t.IE I
Conditional Expectations
• The conditional expectation of X given the event A is defined by

E CX A fx alt dx
• The conditional expectation of X given that Y = y is defined by

fix 8 1,4 15 d
E X 14 5
Total Expectation
• Total expectation theorem: Let AI , A2 , ... ,An be disjoint events that
form a partition of the sample space, and assume that P(Ai > 0)

E X
LIE X Y s fy's dy

E X
If E X Yay
fy Y dy
Independent R.V.
Random variables X and Y are independent if and only if

y
44 f x Y
Joint Expectation

E XY f
f Ry 1,1
4
s 8 14 5 dad

Gorulation

Cov X Y E XY E X EG
Joint Expectation cont.

E Y E X E Y

Var X Y VAR X 2 Cov X4 UARI

f
FARM
If independent then
E XY E X EEY
if 2 Y are independent then Cov X Y o but notviceversa
Bayes' Rule for Continuous Random Variables

f NY f x y fly
f Y x f x

CNS
Effed
KIti.it