RANDOM VARIABLES

Definitions
A variable is defined as random, or stochastic, when it describes mathematically, in accordance
with probability laws, the numerical outcomes of experiments related to random phenomena.
A continuous random variable can assume continuous values on the real axis.
A discrete random variable can assume values in a discrete set of numbers.
A mixed random variable can assume both continuous and discrete values.
The following notes focus on continuous random variables.

Distribution function
The distribution function, also called cumulative distribution, FX ( x ) , is the probability that the
random variable X assumes values less or equal to x:

FX ( x ) = P ( X x )

(1)

This function is always defined on the whole real axis ( < x < + ) ; x is called the state variable.
The distribution function has several noteworthy properties. In particular:

FX ( ) = P ( X ) = P ( 0 ) = 0
FX ( + ) = P ( X + ) = 1
P {x1 < X x 2 } = FX ( x 2 ) FX ( x1 )
P {x1 < X x 2 } 0 FX ( x1 ) FX ( x 2 ) . So, FX (x ) is a not decreasing function.

Density function
The density function p X (x ) , or simply the density of a random variable X, is the prime derivative
of the distribution function with respect to x:
p X (x ) =

dFX (x )
dx

Since FX (x ) is a not decreasing function, p X (x ) is not negative:

pX ( x ) 0
Remembering that FX ( ) = 0 , the application of the fundamental theorem of the integral calculus
provides the relationship:
FX (x ) = p x () d
x

It derives:

FX (x 2 ) FX (x 1 ) = x 2 p x (x ) dx
x

and then:
P ( x1 < X x 2 ) = p X ( x ) dx
x2

x1

p X ( x ) dx = 1

which represent fundamental properties of the density function.

Indexes of a random variable

The distribution function FX (x ) and the density function p X (x ) give a complete description of X.
The indexes of a random variable are quantities that provide a synthetic description of the random
variable. Indexes are framed into 3 classes:
(1) Position indexes provide the position of the distribution, i.e. the position of the values of x
assumed by X; the most important position index is the mean value.
(2) Variability indexes characterise the variability of the values assumed by X; they comprehend
the mean square value, the variance, the standard deviation and the coefficient of variation.
(3) Shape indexes provide an information on the shape of the distribution; they comprehend the
skewness and the kurtosis.
Let us define as the mean value of the random variable X the quantity:

X = xp X ( x )dx

Let us define as root mean square value of the random variable X the quantity:

2X = x 2 p X ( x ) dx

Let us define as variance of the random variable X the quantity:

( x x )

2X =

p X ( x ) dx

Developing the above equations:

2X = 2X 2X
The standard deviation X is the square root of the variance 2X . It defines the dispersion of X
around X . Let us define as coefficient of variation the nondimensional ratio:
VX =

X
X
3

It exists provided that the mean value is different from zero.

Let us define as skewness the non-dimensional quantity:

1 =

1
3X

(x X )

p X (x ) dx

If p X (x ) is symmetric, X lies on its symmetry axis and 1 = 0 .

Let us define as kurtosis the non-dimensional coefficient:

2 =

1
4X

(x X )

p X (x ) dx

If X is a normal random variable, 2 = 3 .

Normal distribution
A continuous random variable X has normal distribution if its density function has the form:

pX ( x ) =

1
X

1 x 2
X
exp

2 X
2

The distribution function, FX ( x ) = P [ X x ] , is given by the expression:

4

1
FX ( x ) =
2

1 u 2
exp 2 du

The following figure shows some qualitative diagrams of the normal distribution and its most
important properties.

The reduced or standard normal random variable is defined as:

Z=

X X
X

Thus the mean value and the standard deviation of Z are 0 and 1, respectively.

COUPLE OF RANDOM VARIABLES

Definitions
A couple of random variables X, Y is called a bi-variate random variable or a 2-component vector.

Joint distribution function

Let us consider a couple of random variables X, Y. The joint distribution function FXY (x , y ) is the
probability that X x, Y y:

FXY ( x, y ) = P ( X x, Y y )
It is always defined on the bi-dimensional space ( < x < + ) , ( < y < + ) .
The marginal distribution functions of X and Y, FX (x ) e FY (y ) , are the distribution functions of
each variable X and Y. In general, the knowledge of FX and FY does not allow to determine FXY .
Instead, given FXY , it is possible to derive FX e FY :

FX ( x ) = P ( X x, Y < + ) = FXY ( x, + )
FY ( y ) = P ( X < +, Y y ) = FXY ( +, y )
The joint distribution function has some relevant properties:

FXY ( , y ) = P ( X < , Y y ) = P ( 0 ) = 0

FXY ( x, ) = P ( X x, Y < ) = P ( 0 ) = 0

FXY ( , ) = P(X < , Y < ) = P (0) = 0

FXY (+ , + ) = P(X < + , Y < + ) = 1
FXY (x , y ) is a not decreasing function of x, y.

Joint density function

The joint density function p XY ( x, y ) , or simply the joint density of the random variables X and Y,
is given by:
2 FXY ( x, y )
p XY ( x, y ) =
xy
The application of the bi-dimensional form of the fundamental theorem of the integral calculus
gives rise to the expression:
FXY (x , y ) = p XY (, ) d d
x

By virtue of the above equation it results:

P ( x1 X x 2 , y1 Y y 2 ) =

x2

x1

y2

y1

p XY ( x, y ) dxdy

p XY ( x, y ) dxdy = 1

Moreover:
+

p X (x ) = p XY (x , y ) dy

p Y (y ) = p XY (x , y ) dx

The density functions p X (x ) and p Y (y ) of X and Y are called herein marginal density functions.

Independent random variables

Two random variables X and Y are defined as independent if the events {X x} and {Y y} are

independent for any x and y, i.e. if P ( X x, Y y ) = P ( X x ) P ( Y y ) . It follows:

FXY (x , y ) = FX (x ) FY (y )
p XY (x, y ) = p X (x ) p Y (y )
These equations are necessary and sufficient conditions of independence.

Indexes of a couple of random variables

The joint distribution function FXY (x , y ) and the joint density function p XY (x, y ) give a complete
probabilistic description of X and Y. The indexes are quantities that provide a synthetic information
on the couple of random variables. They can be framed into two classes:
(1) Indexes of X, Y dealt with separately They are the same indexes of each random variable.
(2) Indexes of X, Y dealt with together They express the probabilistic link between X and Y; they
comprehend the correlation, the covariance and the coefficient of correlation.
The correlation of X, Y is defined as:

R XY =

xyp XY ( x, y ) dxdy

The covariance of X, Y is defined as:

CXY =

( x )( y )p ( x, y ) dxdy

XY

Expanding the above equations it follows:

CXY = R XY X Y
If X = 0 or Y = 0 , then C XY = R XY . Moreover, if X = Y, then R XX = X2 , CXX = X2 .
The normalised covariance of X, Y, called also the coefficient of correlation, is defined as:
XY =

C XY
XY

It should be remembered that X, Y are independent if:

p XY (x, y ) = p X (x ) p Y (y )
These variables are not correlated if:

R XY = X Y
C XY = XY = 0
If X, Y are independent, they are also not correlated. In fact:

R XY =

xyp XY ( x, y ) dxdy = xp X ( x ) dx yp Y ( y ) dy = X Y

The inverse statement is generally not true: if X, Y are not correlated, not necessarily they are also
independent (this occurs only if X, Y are normal random variables). Thus, the condition of
independence is stronger that the condition of not correlation.
The normalised covariance XY expresses the degree of correlation between X and Y. We already
noted that XY = 0 if X and Y are not correlated. It is easy to demonstrate that:

1 XY 1
It is possible to demonstrate that:
XY = +1 Y = aX + b, a > 0
XY = 1 Y = aX + b, a < 0
This equations is equivalent to the position:
XY = 1

X X
Y Y
=Y

=
X
X
Y

The above equations explain the difference between independence and not correlation. The
statistical dependence involves any functional link between X and Y. The correlation involves a
functional link of the linear type. Thus, the correlation is a particular case of the independence.
The following figure shows some typical examples of the statistical link between X, Y. If XY = +1
(a) X and Y are proportional; if XY = 1 (b) X and Y are inversely proportional. For intermediate
values of between 0 and 1 (c), (d) X and Y tend to be roughly proportional or inversely
proportional. The scattering is complete for XY = 0 ; it is worth noting that XY may be equal to
zero also in the presence of a strong functional link of not linear type (f).

Bi-variate normal distribution

Two random variables X, Y have a bi-variate normal distribution if p XY (x, y ) has the form:
p XY ( x, y ) =

1
2X Y 1 2XY

2 ( x ) 2 2 ( x ) ( y ) + 2 ( y ) 2
Y

X
X Y XY
X
y
X
y
exp

2
2
2
2 X Y (1 XY )

where X and Y are the mean values of X and Y, 2X and 2Y are the variances of X and Y, XY
is the coefficient of correlation.

10

The above equations is often represented by the curves provided by the intersection of the joint
density function with the planes p XY (x, y ) = K parallel to the plane x, y:

(x

+ y 2 2 XY xy

(1 )
2
XY

)=K

For XY = 1 these curves degenerate into the linear relationships y = x .

It is relevant to remember that, if X, Y are statistically independent, then they are not correlated; the
inverse statement is in general not true. In the particular case that X, Y have a bi-variate normal
distribution, this statement is correct. Thus, if X, Y are not correlated, they are also independent.
Setting XY = 0 , the bi-variate normal density function becomes:
p XY ( x, y ) =

1 x 2 1
1 y y
x
exp
exp

2
2

2
2

p XY (x, y ) = p X (x ) p Y (y ) , p X (x ) , p Y (y ) being marginal normal distributions. In other words, in

this case, XY = 0 is a necessary and sufficient condition of statistical independence.

11

RANDOM VECTORS
Definitions
A vector that lists n random variables is a n-variate random vector. The probabilistic representation

of the n-variate random vector X = {x1x 2 .. x n } implies the knowledge of the joint distribution of
all the random variables X j ( j = 1,...n ) (n-th order distribution). If X is normal, its complete
probabilistic representation involves the knowledge of the joint distribution of all the possible
couples of random variables X i , X j ( i, j = 1,...n ) composing the vector (2nd order distributions).
T

Joint distribution function

The joint distribution function FX ( x ) = FX1X 2 ...Xn ( x1 , x 2 ,...x n ) of the vector X = {x1x 2 .. x n }T is the
probability that X1 x1 , X 2 x 2 ,...X n x n :

FX ( x ) = FX1X 2 ...Xn ( x1 , x 2 ,...x n ) = P ( X1 x1 , X 2 x 2 ,...X n x n )

The joint distribution function of some of the random variables that compose the vector can be
derived from the above equation setting as equal to infinite all the other variables. For instance:

FX1X2 ( x1 , x 2 ) = FX1X 2 ...Xn ( x1 , x 2 , +,... + )

Moreover, the following properties apply:
FX1X2 ...X n ( x1 , x 2 ,.., x j = ,.., x n ) = 0

for j = 1, ...n

FX1X2 ...Xn ( +, +, ... + ) = 1

FX1X2 ...Xn ( x1 , x 2 ,..x n ) is a not decreasing function of x 1 , x 2 , .. x n .

Joint density function

Generalising to the vector X the considerations already developed with reference to the couple of
random variables X,Y, the joint density function of X is given by the relationship:

p X1X 2 ...X n ( x1 , x 2 ,...x n ) =

n FX1X 2 ...Xn ( x1 , x 2 ,...x n )

x1x 2 ...x n

FX1X2 ...X n ( x1 , x 2 ,...x n ) =

x1

x2

... pX1X 2 ...Xn ( 1 , 2 ,... n ) d1d2 ...dn

xn

P (x 1a < X1 < x 1b , x 2a < X 2 < x 2 b , ... x na < X n < x nb ) =

=

x1b

x1a

x2b

x 2a

...

x nb

x na

p X1X2 ... Xn ( x1 , x 2 , ... x n ) dx1dx 2 ...dx n

It follows that:
12

... p X1X2 ... Xn ( x1 , x 2 ,...x n ) dx1dx 2 ...dx n = 1

The joint density function of some of the random variables composing the random vector can be
derived from p X1X2 ... X n (x 1 , x 2 , ... x n ) integrating between - and + with respect to all the other
variables. For instance:
p X1X2 (x 1 , x 2 ) = ... ...p X1X2X3 ... X n (x 1 , x 2 , x 3 ... x n ) dx 3 ... dx n
+

Independent random vectors

A random vector is composed by independent random variables if the following properties apply:
n

FX1X2 ...X n ( x1 , x 2 ,...x n ) = FX1 ( x1 ) FX2 ( x 2 ) ...FXn ( x n ) = FXi ( x i )

i =1

p X1X2 ...X n ( x1 , x 2 ,...x n ) = p X1 ( x1 ) p X2 ( x 2 ) ...p X n ( x n ) = p Xi (x i )

i =1

Indexes of a random vector

Using the indexes introduced for a couple of random variables, a random vector X = {x1x 2 .. x n }
is characterised, in synthetic form, by the vector of the mean values X and by the matrix of the
correlations R X or of the covariances C X .
T

The mean vector, called also the vector of the mean values, is defined as:
X = E [ X] = { X1 X 2 ... Xn }

Its components are the mean values of the random variables that compose the random vector X,
i.e. Xi = E [ X i ] ( i = 1,...n ) .
The correlation matrix is defined as:

RX = E X XT

R X1X1
R
X X
= 2 1

R X n X1

R X1X2
R X 2X 2

R X nX 2

R X1X n
.. R X2X n

.. R X n X n
..

The on-diagonal terms are the mean square values of each random variable ( R Xi Xi = 2Xi ) ; the offdiagonal terms are the correlations of all the possible couples of the random variables.

13

The covariance matrix is defined as:

C X = E (X X )(X - X )

C X1X1
C
X X
= 2 1

C X nX1

C X1X2
C X 2X 2

C XnX2

.. C X1X n
.. C X2X n

.. C X n X n

The on-diagonal terms are the variances of each random variable ( CXi Xi = 2Xi ) ; the off-diagonal
terms are the covariances of all the couples of random variables. Expanding the above equations it
follows that:
R X = C X + X TX

R X and C X are symmetric matrices. It is possible to show that they are also semi-positive defined.

R X is diagonal if all the couples of different random variables X i , X j ( i j) are orthogonal, i.e.
R Xi X j = 0 (i,j, ij). C X is diagonal of all the couples of different random variables X i , X j (i j)

are not correlated, i.e. CXi X j = 0 (i,j, ij).

It is demonstrated later that, if X is a normal vector, then the knowledge of X and R X or C X is
enough to determine the joint distributions of any order.

n-variate normal distribution

Let us consider a n-variate random vector X = { X1 X 2 ... X n } . Let X = X1 X 2 ... X n

T

}T

be

the mean vector and C X = E ( X X )( X X ) be the covariance matrix. X has a n-variate

normal distribution if its joint density function p X1 X 2 ... X n (x 1 x 2 ... x n ) has the form:
T

p X1 X2 ... Xn ( x1 , x 2 ,...x n ) =

( 2 )

n/2

CX

1/ 2

1
exp
2 C X

CX

jk

where CX is the modulus of the determinant of CX e CX

jk

(x

X j

)(x

Xk )

is the j,k-th co-factor of CX (i.e. the

determinant of the matrix obtained by cancelling the j-th row and the k-th column of CX , multiplied
by ( 1)

j+ k

pX ( x ) =

). In matrix form:

( 2 )

n/2

CX

1/ 2

T
1

exp ( X X ) CX1 ( X X )
2

Thus, the knowledge of the first order statistical mean ( X ) and of the second order statistical mean

( CX )

of X is enough to derive the joint density function of order n.

14