Slide #0

ECE673 – Week 3
One Continuous Random Variable and
Mixed Random Variables
Learning Objectives
– Understanding the PDF and CDF of a continuous random
variable
– Understanding the transformation of a continuous random
variable into another random variable
– Calculating the expected value, variance, other moments
and characteristic function of a continuous random variable
– Understanding the usage of probability inequalities
– Understanding mixed random variables
Slide #HOME

Week 3 – Lessons
Lesson 3.1 Lesson 3.3Cumulative Distribution
Probability Density Functions Lesson 3.2Important PDFs Functions
(PDFs)

Lesson 3.5 Lesson 3.6

Lesson 3.4Transformations Mixed random variables Computer Simulation of Rando
m Variables

Lesson 3.7 Lesson 3.8 Lesson 3.9

Determining the Expected Values Variance and Moments of a Rand Characteristic Functions
om Variable

Lesson 3.10 Lesson 3.11

Probability, Moments, and the Che Estimating Mean and Variance
byshev Inequality
Slide #1.1

Lesson 3.1
Probability Density Functions (PDFs)

Roughly speaking, the value of the probability density function

(PDF) of a continuous random variable X, pX(x), at any point x is
the chance of X taking a value in a small interval around x, i.e.,
falling into a small interval around x.
Slide #1.2

Probability Density Functions (PDFs)

• PDF properties
– Property 10.1 – PDF must be nonnegative.

There is no negative probability.

– Property 10.2 – PDF must integrate to one.

All possible probabilities should add up to 100%.

Slide #1.3

Probability Density Functions (PDFs)

• Example - Exponential PDF
- Sometimes used to describe the lifetime of a product.
- The PDF has an exponential form, with λ > 0 as a parameter

- Clearly pX(x) for , and

for λ > 0

- Satisfies property 10.1 and 10.2

See Figure 3.A: Exponential PDF

`
Slide #1.4

Probability Density Functions (PDFs)

clear all;
x=[0:0.1:5];
lambda1=0.7;
lambda2=1.2;
p1=lambda1.*exp(-lambda1*x);
p2=lambda2.*exp(-lambda2*x);
plot(x,p1);
hold all;
plot(x,p2);
xlabel('x');
ylabel('p_X[x]');
axis([-1 5 0 1.4]);

Figure 3.A: Exponential PDF

Slide #1.5

Probability Density Functions (PDFs)

• Example – Exponential PDF
– If X is the failure time in days of a light bulb, then P[X > 100] is the
probability that the light bulb will fail after 100 days.
Slide #End of Lesson

End of Lesson
Click here to return to the main menu.
Slide #2.1

Lesson 3.2

Important PDFs
Slide #2.2

Important PDFs
• Uniform

See Figure 3.B: Examples of uniform PDF

• Gaussian or Normal

“Standard” Gaussian PDF is a Gaussian PDF with and σ2 = 1.

See Figure 3.C: Examples of Gaussian PDF with different s
Slide #2.3
Important PDFs
clear all;
x=[0:0.01:6]; y=x;
a=2; b=5;
for i=1:601 % 601 points from 0 to
6 with step = 0.01
if (x(i)>a && x(i)<b)
y(i)=1/(b-a);
else
y(i)=0;
end;
end;
plot(x,y);
xlabel('x');
ylabel('p_X(x)');
axis([0 8 0 0.5])

(a) a=2, b=5

Figure 3.B: Examples of uniform PDF

Slide #2.4
Important PDFs
clear all;
x=[0:0.01:6]; y=x;
a=1; b=6;
for i=1:601 % 601 points from 0 to
6 with step = 0.01
if (x(i)>a && x(i)<b)
y(i)=1/(b-a);
else
y(i)=0;
end;
end;
plot(x,y);
xlabel('x');
ylabel('p_X(x)');
axis([0 8 0 0.5])

(b) a=1, b=6

Figure 3.B: Examples of uniform PDF

Slide #2.5
Important PDFs
Matlab Code (an example for and σ2 = 2)
clear all;
var = 2;
mean = 1
x=[-6:0.1:6];
y=1/sqrt(2*pi*var).*exp(-(x-mean).^2/(2*var));
%plot the PDF
plot(x,y)
xlabel('x');
ylabel('p_X(x)');
axis([-6 6 0 0.4]);
%generate realizations of the random variable
r=mean+sqrt(var)*randn(50,1); %generate 50 realizations of the random variable in a 50 by 1 matrix
figure;
stem(r);
xlabel('sample number');
ylabel('result');

Figure 3.C: Examples of Gaussian PDF with different s

Slide #2.6
Important PDFs

(a) , σ2 = 2

Figure 3.C: Examples of Gaussian PDF with different s

Slide #2.7
Important PDFs

(b) , σ2 = 2

Figure 3.C: Examples of Gaussian PDF with different s

Slide #End of Lesson

End of Lesson
Click here to return to the main menu.
Slide #3.1

Lesson 3.3

Cumulative Distribution Functions (CDFs)

The value of the CDF FX(x) at any point x is the integral of the PDF
of the random variable X, up to the point x. It is the probability of X
being less than or equal to x (X x).
Slide #3.2

Cumulative Distribution Functions (CDFs)

• Definition

• Integral representation
Slide #3.3

Cumulative Distribution Functions (CDFs)

• Exponential
– Using eq. (10.5), we have

– Also

– Therefore
Slide #3.4

Cumulative Distribution Functions (CDFs)

clear all;
x=[0:0.05:5];
lambda=2;
y=1-exp(-lambda*x);
plot(x,y);
xlabel('x');
ylabel('F_X(x)');
axis([-1 5 0 1.3]);

Figure 3.D: CDF for exponential random variable with λ = 2

Slide #3.5

Cumulative Distribution Functions (CDFs)

• Standard Gaussian (Gaussian with μ = 0 and σ2 =1)

– From eq. (10.7), we have

– CDF of standard Gaussian is typically denoted by Φ(x)

Slide #3.6

Cumulative Distribution Functions (CDFs)

• “Q” function for the standard Gaussian random variable

- In communication and signal processing, usually Q function is used,

instead of the Φ function
(10.17)

 1 1 2(10.18)
 x 2
exp(  t )dt
2

See Figure 3.E: Definitions of and .

Slide #3.7

Cumulative Distribution Functions (CDFs)

Matlab code for creating the graphs:
clear all;
var = 1;
mean = 0
x=[-4:0.1:4];
y=1/sqrt(2*pi*var).*exp(-(x-mean).^2/(2*var));
plot(x,y) % plot the curve
hold all;
xlabel('x');
ylabel('p_X(x)');
axis([-4 4 0 0.5]);
stem(x(1:61),y(1:61)); %make the shaded area for phi
figure;
plot(x,y) % plot the curve
hold all;
xlabel('x');
ylabel('p_X(x)');
axis([-4 4 0 0.5]);
stem(x(61:81),y(61:81)); %make the shaded area for Q

Figure 3.E: Definitions of and .

Slide #3.7

Cumulative Distribution Functions (CDFs)

(a) Shaded area = (b) Shaded area = .

Figure 3.E: Definitions of and .

Slide #3.8

Cumulative Distribution Functions (CDFs)

• Relationship between PDFs and CDFs
– Obtain PDF from CDF by differentiation:

(10.26)

– Example: For X ~ exp(λ), shorthand notation for exponential

distribution, we previously observed

– Now for all x, except x = 0, we have

Slide #End of Lesson

End of Lesson
Click here to return to the main menu.
Slide #4.1

Lesson 3.4

Transformations
Slide #4.2

Transformations
• General result for the PDF of Y = g(X)

(10.30)

- Y is a function of the random variable X

- g-1(y) is the inverse function, gives the value of x such that g(x)=y.

- dg 1
( y ) is actually dxin y domain.
dy dy
Slide #4.3

Transformations
• One-to-one
– Example 10.5: Linear
To determine the PDF of Y = a X + b =g(X)
yb
Solve for x to obtain x  g 1 ( y ) 
a
Using eq. (10.30), PDF of Y can be written as

 y  b  1 (10.31)
pY ( y )  p X  
 a a
-- Consider X being standard Gaussian and let Y  2X 
Then Y becomes another Gaussian with the parameters σ2 and μ
This result allows to simulate an arbitrary Gaussian variable in MATLAB
y=sqrt(sigma2)*randn+mu
Remember: Linear transformation of Gaussian gives another Gaussian!
Slide #4.4

Transformations
• Not one-to-one
– Example: Quadratic y  x2
Obtain multiple solutions by solving for x in y  g ( x)
x1   y  g11 ( y )
x1   y  g 21 ( y )
See Figure 10.23 in the book: Solutions for x in y = g(x) = x2

Now use eq. (10.30) for all the solutions and add them up
1 dg11 ( y ) 1 dg 21 ( y(10.32)
)
pY ( y )  p X ( g ( y ))
1  p X ( g 2 ( y ))
dy dy
See also Example 10.7 for PDF of Y = X 2, when X is a standard Gaussian
random variable.
Slide #End of Lesson

End of Lesson
Click here to return to the main menu.
Slide #5.1

Lesson 3.5

Mixed Random Variables

Slide #5.2

Mixed Random Variables

• If random variable X takes on a continuum of values as well as the discrete
values {x1,x2,…} with the corresponding nonzero probabilities {p1,p2,…},
then X is a mixed random variable with the following PDF


(10.36)
p X ( x )  pc ( x )   pi ( x  xi )
i 1
- To be a valid PDF, it must satisfy

 

 pc ( x )dx   pi  1

i 1
- Note that for the delta function we have 

 p  ( x  x )dx  p

i i i
Slide #End of Lesson

End of Lesson
Click here to return to the main menu.
Slide #6.1

Lesson 3.6

Computer Simulation of Random Variables

Slide #6.2

Computer Simulation of Random Variables

• Theorem 10.9.1 – Inverse probability integral transformation theorem
– If a continuous random variable X is given as

X  FX1 U 
where U ~ U (0,1), then X has the following PDF

p X ( x)  dFX ( x) / dx
-- The theorem implies that if a random variable X is transformed by
applying the transformation FX(.) to it, where FX(.) is its CDF, then the
resulting random variable FX(X) is uniformly distributed between 0 and
1 ! This interesting property can be used to simulate random variables
from a uniform random number generator.
Slide #6.3

Computer Simulation of Random Variables

• Example 10.10: Probability integral transformation of
exponential random variable
– If X has an exponential distribution with λ = 1, we previously learned
that its CDF is given by

The theorem says that – ln(1 – U), the inverse function of the above
CDF applied to U, which is uniformly distributed between 0 and 1, has
in fact the same distribution as X !
This can be used to simulate exponential variable from a uniform
variable. Here is a Matlab code to generate 300 samples of the above
exponential distribution -log(1-rand(1,300))
Slide #End of Lesson

End of Lesson
Click here to return to the main menu.
Slide #7.1

Lesson 3.7

Determining Expected Values

Slide #7.2

Determining Expected Values

• Definition of expected value for a continuous random

variable X

(11.3)
Slide #7.3

Determining Expected Values

• Example 11. 1: Expected value for random variable with
nonuniform PDF

- See Figure 3.F in Slide #7.5: Example of nonuniform PDF and its mean.

According to the graph of the PDF, x / 2 for 0 < x < 2, E[X] can be
calculated using the following integral

2 1  4
E[ X ]   x  x  dx 
0
2  3
Slide #7.4

Determining Expected Values

Matlab code for generating some realizations in Figure 3.F (next

slide), for the nonuniform random variable with the PDF x / 2, 0 <
x<2:

u=rand(40,1);
x=sqrt(4*u); %Obtained using Theorem 10.9.1
of the textbook, p.325
stem(x); xlabel('sample number');
ylabel('result');
Slide #7.5
Determining Expected Values

(b) 40 realizations and expected value

(a) PDF
of 1.33
Figure 3.F: Example of nonuniform PDF and its mean.
Slide #7.6

Determining Expected Values

• Exponential
– If X ~ exp(λ), then

(11.7)
Slide #7.7

Determining Expected Values

• Expected Value for a Function of a Random Variable

(11.9)

- This equation is actually the same as eq. (11.3), with x replaced by y.

However, there is no need to determine pY(y) first. We can use eq. (11.10) in the
next slide to calculate the expected value of Y.
Slide #7.8

Determining Expected Values

• Expected Value for a Function of a Random Variable
– Not necessary to use eq. (11.9)
– Instead, for Y = g(X), use this formula

(11.10)

• Example: See Example 11.3

Slide #7.7

Determining Expected Values

• Assume X is a mixed random variable with the following PDF

where pc(x) is the continuous part of the PDF. For E[X] we have

(11.11)
Slide #End of Lesson

End of Lesson
Click here to return to the main menu.
Slide #8.1

Lesson 3.8

Variance and Moments of a Random Variable

Slide #8.2

Variance and Moments of a Random Variable

• To evaluate variance, use this formula:

(11.12)

• In general, variance is defined as E[(X-E[X])2], and eq.

(11.12) is the variance for a continuous random variable,
written in terms of its PDF.
Slide #8.3

Variance and Moments of a Random Variable

• Example 11.5: Variance of a uniform random variable

– If X~U(a,b), then

– Using we obtain
Slide #End of Lesson

End of Lesson
Click here to return to the main menu.
Slide #9.1

Lesson 3.9

Characteristic Functions
Slide #9.2

Characteristic Functions

• Definition

(11.16)

or

(11.17)
Slide #9.3

Characteristic Functions
• Example 11.7: Moments of the exponential PDF
- Characteristic function of an exponential random variable is given by:
(11.19)

- Moments are given by

n
(11.18)
1 d X ( )
n
E[ X ]  n n
| 0
d  of the exponential PDF can be
j moments
Using eq. (11.18) and eq. (11.19),
shown to be

n n!
E[ X ]  n

Slide #9.4

Characteristic Functions

• Relationship between PDFs and characteristic functions

– PDF can be obtained from the characteristic function via inverse
Fourier transform

(11.20)
Slide #End of Lesson

End of Lesson
Click here to return to the main menu.
Slide #10.1

Lesson 3.10

Probability, Moments, and the Chebyshev

Inequality
Slide #10.2

Probability, Moments, and the Chebyshev

Inequality
• Chebyshev inequality

(11.21)

- It says the probability that a random variable deviates from its

mean by more than γ (in either direction) is less than or equal to
var(X)/γ2.
- If E[X] = 0 and var(X) = 1, then
Slide #10.3

Probability, Moments, and the Chebyshev

Inequality
• Example 11.8: Bounds for different PDFs (both have
E[X] = 0 and var(X) = 1)

-- If , then using the Chebyshev inequality for any PDF of X.

-- If X ~ N(0,1), the first PDF above, then the actual probability is

See slide #3.6 for the definition of the Q function.

Slide #10.4

Probability, Moments, and the Chebyshev

Inequality
• Example 11.8: Bounds for different PDFs (continued)
-- For Laplacian PDF, the second PDF is the previous slide, the actual
probability is
Slide #End of Lesson

End of Lesson
Click here to return to the main menu.
Slide #11.1

Lesson 3.11

Estimating Mean and Variance

Mean and variance of continuous random variables are

estimated using exactly the same method used for discrete
random variables.
Slide #11.2

Estimating Mean and Variance

• M outcomes of a random variable X are observed.

– Mean estimate:
(11.22)

– Variance estimate:

(11.23)
Slide #End of Lesson

End of Lesson
Click here to return to the main menu.
Reference

Kay, S. (2006). Intuitive probability and random

processes using Matlab. Springer.