1

Review of Probability and Random Processes

 2

Importance of Random Processes

• Random variables and processes talk about quantities and

signals which are unknown in advance
• The data sent through a communication system is modeled
as random variable
• The noise, interference, and fading introduced by the
channel can all be modeled as random processes
• Even the measure of performance (Probability of Bit Error)
is expressed in terms of a probability
 3

Random Events

• When we conduct a random experiment, we can use set

notation to describe possible outcomes
• Examples: Roll a six-sided die
Possible Outcomes: S  {1, 2,3, 4,5,6}
• An event is any subset of possible outcomes: A  {1, 2}
 4

Random Events (continued)

• The complementary event: A  S  A  {3, 4,5, 6}

• The set of all outcomes in the certain event: S
• The null event: 
• Transmitting a data bit is also an experiment

Wireless Communication Research Laboratory (WiCoRe)

 5

Probability

• The probability P(A) is a number which measures the

likelihood of the event A
Axioms of Probability
• No event has probability less than zero: P( A)  0
P( A)  1 and P( A)  1  A  S
• Let A and B be two events such that: A  B  
Then: P( A  B)  P( A)  P( B)
• All other laws of probability follow from these axioms
 6

Relationships Between Random Events

• Joint Probability: P( AB)  P( A  B)

- Probability that both A and B occur

P( AB)
• Conditional Probability: P( A | B) 
P( B)

- Probability that A will occur given that B has occurred

Wireless Communication Research Laboratory (WiCoRe)

 7

Relationships Between Random Events

• Statistical Independence:
- Events A and B are statistically independent if:
P( AB)  P( A) P( B)
- If A and B are independence than:
P( A | B)  P( A) and P( B | A)  P( B)

Wireless Communication Research Laboratory (WiCoRe)

 8

Random Variables

• A random variable X(S) is a real valued function of the

underlying even space: s  S
• A random variable may be:
-Discrete valued: range is finite (e.g.{0,1}) or countable
infinite (e.g.{1,2,3…..})
-Continuous valued: range is uncountable infinite (e.g. )
• A random variable may be described by:
- A name: X
- Its range: X 
- A description of its distribution
 9

Cumulative Distribution Function

• Definition: FX ( x)  F ( x)  P( X  x)
• Properties:
 FX ( x) is monotonically nondecreasing
 F ()  0
 F ()  1
 P(a  X  b)  F (b)  F (a)
• While the CDF defines the distribution of a random
variable, we will usually work with the pdf or pmf
• In some texts, the CDF is called PDF (Probability
Distribution function)
Wireless Communication Research Laboratory (WiCoRe)
 10

Probability Density Function

dFX ( x) dF ( x)
• Definition: PX ( x)  or P( x) 
dx dx
• Interpretations: pdf measures how fast the CDF is
increasing or how likely a random variable is to lie around
a particular value
• Properties:

P( x)  0 
P( x)dx  1
b
P(a  X  b)   P( x)dx
a
 11

Expected Values

• Expected values are a shorthand way of describing a

random variable
• The most important examples are:


-Mean: E ( X )  mx   xp( x)dx



-Variance: E ([ X  mx ]2 ) 
  2
( x mx ) p( x)dx

 12

Probability Mass Functions (pmf)

• A discrete random variable can be described by a pdf if we
allow impulse functions
• We usually use probability mass functions (pmf)
p( x)  P( X  x)
• Properties are analogous to pdf
p ( x)  0

 p ( x)  1
X
b
P ( a  X  b)   p ( x )
xa

Wireless Communication Research Laboratory (WiCoRe)

 13

Some Useful Probability Distributions

• Binary Distribution
1  p x0
p ( x)  
 p x 1
• This is frequently used for binary data
• Mean: E ( X )  p

 X  p(1  p)
2
• Variance:

Wireless Communication Research Laboratory (WiCoRe)

 14

Some Useful Probability Distributions

(continued)
n
• Let Y  X
i 1
i
where  X i , i  1,..., n are independent

binary random variables with

1  p x0
p ( x)  
 p x 1

 n y
• Then pY ( y )    p (1  p)n y y  0,1,..., n
 y
• Mean: E ( X )  np
• Variance:  X  np(1 
2
p)
Wireless Communication Research Laboratory (WiCoRe)
 15

Some Useful Probability Distributions

(continued)
• Uniform pdf:
 1 a xb

p( x)   b  a

 0 otherwise
• It is a continuous random variable

1
• Mean: E ( X )  (a  b)
2

1
• Variance: X 2
 ( a  b) 2
12
Wireless Communication Research Laboratory (WiCoRe)
 16

Some Useful Probability Distributions

(continued)

1 ( x mx ) 2 2
• Gaussian pdf: p( x)  e
2

• A gaussian random variable is completely determined by

its mean and variance

Wireless Communication Research Laboratory (WiCoRe)

 17

The Q-function

• The function that is frequently used for the area under the
tail of the gaussian pdf is the denoted by Q(x)

Q( x)   e t 2 2
dt , x0
x

• The Q-function is a standard form for expressing error

probabilities without a closed form

Wireless Communication Research Laboratory (WiCoRe)

 18

A Communication System with Guassian noise

S   a RSN

Transmitter Receiver R 0?


(0,  n )
2
N

• The probability that the receiver will make an error is

  ( x  a )2
1  a
P( R  0 | S  a )   e 2 n2
dx  Q  
0 2 n n 

Wireless Communication Research Laboratory (WiCoRe)

 19

Random Processes

• A random variable has a single value. However, actual

signals change with time
• Random variables model unknown events
• Random processes model unknown signals
• A random process is just a collection of random variables
• If X(t) is a random process, then X(1), X(1.5) and X(37.5)
are all random variables for any specific time t

Wireless Communication Research Laboratory (WiCoRe)

 20

Terminology Describing Random Processes

• A stationary random process has statistical properties

which do not change at all time
• A wide sense stationary (WSS) process has a mean and
autocorrelation function which do not change with time
• A random process is ergodic if the time average always
converges to the statistical average
• Unless specified, we will assume that all random processes
are WSS and ergodic

Wireless Communication Research Laboratory (WiCoRe)

 21

Description of Random Processes

• Knowing the pdf of individual samples of the random

process is not sufficient.
- We also need to know how individual samples are
related to each other
• Two tools are available to decribe this relationship
- Autocorrelation function
- Power spectral density function

Wireless Communication Research Laboratory (WiCoRe)

 22

Autocorrelation

• Autocorrelation measures how a random process changes

with time
• Intuitively, X(1) and X(1.1) will be strongly related than
X(1) and X(100000)
• The autocorrelation function quantifies this
• For a WSS random process,
 X    E  X  t  X  t   
• Note that Power   X (0)

Wireless Communication Research Laboratory (WiCoRe)

 23

Power Spectral Density

•   f  tells us how much power is at each frequency
• Wiener-Khinchine Theorem: ( f )  F{ ( )}
- Power spectral density and autocorrelation are a
Fourier Transform pair
• Properties of Power Spectral Density
 ( f )  0
  ( f )   ( f )

 Power    ( f )df


Wireless Communication Research Laboratory (WiCoRe)

 24

Gaussian Random Processes

• Gaussian random processes have some special properties

- If a gaussian random process is wide-sense stationary,
then it is also stationary
- If the input to a linear system is a Gaussian random
process, then the output is also a Gaussian random
process

Wireless Communication Research Laboratory (WiCoRe)

 25

Linear systems

• Input: x(t )
• Impulse Response: h(t )
• Output: y (t )

x(t ) h(t ) y (t )

Wireless Communication Research Laboratory (WiCoRe)

 26

Computing the Output of Linear Systems

• Deterministic Signals:
- Time domain: y(t )  h(t )* x(t )

- Frequency domain: Y ( f )  F{ y(t )}  X ( f ) H ( f )

• For a random process, we can still relate the statistical

properties of the input and output signal
- Time domain: Y ( )   X ( )* h( )* h( )

- Frequency domain: Y ( f )   X ( f ) | H ( f ) |2