RANDOM VECTORS

The concept of random vector is a multidimensional generalization of the concept of random

variable.
Suppose that we conduct a probabilistic experiment and that the possible outcomes of the
experiment are described by a sample space . A random vector is a vector whose value
depends on the outcome of the experiment, as stated by the following:
Let Ω be a sample space. A random vector is a function that maps the sample space to
the set of n-dimensional real vector defined by

X : Ω → ℜn

Another definition of random vector is that it is a column vector of random variables; i.e the
elements of the vector are random variables. These random variables occur simultaneously;
i.e observations of these variables are taken on the same unit and usually they are correlated.
For example suppose we are interested in analyzing information on age, height, weight,
blood pressure and cholesterol level of a certain group of individuals at the same time; then
each of these measurements is a random variable and together they become components
of a random vector. Let X1 be age, X2 be height, X3 be weight, X4 be blood pressure and
X5 be cholesterol level; then the random vector is
 
X1
 
 X2 
 
X=  X 3


 
 4 
X
X5

It is a a random vector with five components hence it is a 5 × 1 column vector

in general if p random variables are defined jointly, then we have a p-dimensional random
vector  
X1
 
 X2 
X=
 .. 

 . 
Xp
This vector can be expressed as a row vector

X = [X1 , X2 , ..., Xp ]′

1
 JOINT PROBABILITY DISTRIBUTION FUNCTION FOR RANDOM
VECTOR
The probability distribution depends on whether the components of random vector are
discrete or continuous.

(a) If the components are discrete random variables, then joint probability distribution
function (j.p.d.f) is

Pr [X1 = x1 , X2 = x2 = .. = Xi = xi = ... = Xp = xp ] = f (x1 , x2 , .., xp )

which satisfies the conditions:

(i)
0 < Pr [X1 = x1 , X2 = x2 = .. = Xi = xi = ... = Xp = xp ] < 1

(ii)
X X X
... ... Pr [X1 = x1 , X2 = x2 = .. = Xi = xi = ... = Xp = xp ] = 1
xp xi x1
| {z }
p summations

For (ii) we are summing over all possible values that each random variable can take
on. We have a total of p summations.

(b) If the components are continuous random variables, then j.p.d.f is f (x1 , x2 , ..., xp )
and it satisfies the conditions:

(i)
0 < f (x1 , x2 , ..., xp ) < 1

(ii) Z Z Z
.... ..... f (x1 , x2 , ..., xp )δx1 ...δxi ..δxp = 1
xp xi x1
| {z }
p integrals

(iii)

Pr [a1 < X1 < b1 , a2 < X1 < b2 , ..., ap < X1 < bp ] =

Z bp Z b2 Z b1
.... f (x1 , x2 , ..., xp )δx1 δx2 ....δxp
ap a2 a1
| {z }
p integrals

2
 For (ii) and (iii) we have p integrals over all possible values the p variables can take
on.

MARGINAL AND CONDITIONAL DISTRIBUTIONS

Using the joint probability distribution function(j .p.d.f) we can obtain the marginal dis-
tributions and conditional distributions of the components of random vector.
Marginal distributions:
Suppose X is a p×1 random vector with j.p.d.f f (x1 , x2 , .., xp ); then we can define marginal
distributions for :

(i) Each component Xi ,i = 1, 2, .., p by summing over all possible values of the remaining
p − 1 random variables for discrete variables or integrating over all possible values of
remaining p − 1continuous random variables
Example: Let X = [X1 , X2 , X3 ]′ have j.p.d.f
(
1
24
(x1 + x2 + x3 ), 0 < x1 < 2, 1 < x2 < 4, 0 < x3 < 1
f (x1 , x2 , x3 ) =
0, elsewhere
Find the marginal distribution for X1 .
Solution
Let g(x1 ) be the marginal distribution of X1
Z 1Z 4
1
g(x1 ) = (x1 + x2 + x3 )δx2 δx3
1 24
Z 1 Z 4
0

1
= (x1 + x2 + x3 )δx2 δx3
0 1 24

Now
Z 4  4
1 1 x22
(x1 + x2 + x3 )δx2 = x1 x 2 + + x2 x3
24 24 2
1
  1
1 15
= 3x1 + + 3x3
24 2
The next integral is
Z 1    1
1 15 1 15x3 3x23
3x1 + + 3x3 δx3 = 3x1 x3 + +
0 24 2 24 2 2 0
1
= (x1 + 3)
8
Hence the marginal distribution is
1
g(x1 ) = (x1 + 3) 0 < x1 < 2
8
3
 (ii) Two components Xi and Xj ; by summing over all possible values of the remaining
p − 2 random variables for discrete variables or integrating over all possible values of
the remaining p − 2 continuous random variables.

(iii) For r components, r < p; the marginal distribution is obtained by either summing
over all possible values of the remaining p−r discrete random variables or integrating
over all possible values of the remaining p − r continuous random variables

Conditional distributions:
Let X be a p×1 random vector with j.p.d.f f (x1 , x2 , .., xp ). Further let X1 = [X1 , X2 , .., Xk ]′
and X2 = [Xk+1 , Xk+2 , .., Xp ]′ with respective marginal distributions g(x1 , x2 , .., xk ) and
h(xk+1 , xk+2 , ..xp ).
The conditional distribution of X1 given X2 is given by

f (x1 , x2 , .., xp )
f1 (X1 |X2 ) =
h(xk+1 , xk+2 , ..xp )

and the conditional distribution of X2 given X1 is given by

f (x1 , x2 , .., xp )
f2 (X2 |X1 ) =
g(x1 , x2 , .., xk )

Example: Let X = [X1 , X2 , X3 ]′ have j.p.d.f

(
1
3
(x1 + 2x2 + 3x3 ), 0 < x1 , x2 , x3 < 1
f (x1 , x2 , x3 ) =
0, elsewhere

Find

(i) the marginal distribution of X1 , X2

(ii) Pr X3 < 12 |X1 = 14 , X2 = 1
4

Solution

(i) let h(x1 , x2 ) be the marginal distribution of X1 , X2 .

Z 1
1
h(x1 , x2 ) = (x1 + 2x2 + 3x3 )δx3
0 3
 1
1 3x23
= x1 x3 + 2x2 x3 +
3 2 0
1
= (x1 + 2x2 )
3
4
 the marginal distribution is
(
1
3
(x1 + 2x2 ), 0 < x1 , x2 < 1
h(x1 , x2 ) =
0, elsewhere

(ii) Let the conditional distribution of X3 given X1 , X2 be g(X3 |X1 , X2 )

1
(x1 + 2x2 + 3x3 ) x1 + 2x2 + 3x3
g(X3 |X1 , X2 ) = 3
1 =
3
(x1 + 2x2 ) x1 + 2x2
1 1
For X1 = 4
and X2 = 4

x1 + 2x2 + 3x3
g(X3 |X1 , X2 ) = = 1 + 4x3
x1 + 2x2
Thus
  Z 1
1 1 1 2
Pr X3 < |X1 = , X2 = = 1 + 4x3 δx3
2 4 2 0
 1
= x3 + 2x23 02
3
=
4

MEAN VECTOR AND VARIANCE-COVARIANCE MATRIX OF RANDOM

VECTOR
Since each component of a random vector is a random variable; we can define the marginal
probability distribution of each of them and use these marginal distributions to obtain the
mean and variance of each random variable.
Let X be a p × 1 random vector with j.p.d.f f (x1 , x2 , .., xp ). For each Xi i = 1, 2, .., p,
using their marginal distribution

E[Xi ] = µi
Var[Xi ] = E[Xi − µi ]2 = σi2

Also since we can obtain the marginal distribution of combination of any two random
variables Xi and Xj ; similarly we can use these distributions to define covariance between
the two variables.

Cov(Xi , Xj ) = E [(Xi − µi )(Xj − µj )] = σij

We will use the mean, variance and covariance measures of components of random vector(i.e
random variables) to define the expected value and variance of the vector.

5
Mean vector :
The expected value of random vector X = [X1 , X2 , .., Xp ]′ is given by
     
X1 E[X1 ] µ1
     
 X2   E[X2 ]   µ2 
     
 ..   ..   .. 
 .   .   
E[X] = E  = = . =µ
 X   E[X ]   µ 
 i   i   i 
 .   ..   . 
 ..   .   . 
     . 
Xp E[Xp ] µp

µ is referred to as mean vector. Its elements are the expected values of random
variables.
Variance of random vector
The variance of random vector X = [X1 , X2 , .., Xp ]′ is given by

Var[X] = E [(X − µ)(X − µ)′ ]

It is the product of a column vector and its transpose which gives a p × p matrix:

 
σ12 σ12 . . . σ1j . . . σ1p
 
 σ21 σ22 . . . σ2j . . . σ2p 
 
 .. .. .. .. .. .. 
 . . . . . . 
Σ = (σij ) = 


 σi1 σi2 . . . σij . . . σip 
 .. .. .. .. .. .. 
 . . . . . . 
 
2
σp1 σp2 . . . σpj . . . σp

It is a symmetrical matrix with measures of variance along the diagonal and measures of
covariance off the diagonal. Σ is known as the variance-covariance matrix.
We use the measures of variance-covariance matrix to obtain correlation values:
σij
ρij = q
σi2 σj2

6
These measures can be used to obtain the correlation matrix
 
1 ρ12 . . . ρ1j . . . ρ1p
 
 ρ21 1 . . . ρ2j . . . ρ2p 
 
 .. .. .. .. .. .. 
 . . . . . . 
 
 ρ . . . ρij . . . ρip 
 i1 ρi2 
 . .. .. .. .. .. 
 .. . . . . . 
 
ρp1 ρp2 . . . ρpj . . . 1

It is a symmetrical matrix with the values 1 along the diagonal.

LINEAR COMBINATIONS OF RANDOM VECTOR
Recall when dealing with single random variable X , then one can define a new random
variable which is a function of X, for example suppose we define Y as Y = aX, a is a
constant. We can obtain the p.d.f of Y using p.d.f of X; more importantly even without
the p.d.f of X we know that

(i) E[Y]=a E[X]

(ii) Var(Y)= a2 Var(X)

Extending these properties of expectation to random vector; we can define the mean and
variance of linear functions of random vectors.
Let X be a random vector with p components. Further let µ and Σ be the mean vector
and variance-covariance matrix of vector.

(i) Define a new random vector Z such thatZ = a1 X1 + a2 X2 + .... + ap Xp = a′ X where

a = [a1 , a2 , .., ap ]′ is a vector of constants;

E[Z] = E[a′ X]
= a′ µ
Var[Z] = Var[a′ X]
= a′ Σa

(ii) Let A be a m × p matrix of constants. Define a new random vector Y, Y = AX; Y

7
 is a m × 1 column vector whose elements are m random variables.
    
Y1 a11 a12 . . . a1j . . . a1p X1
    
 Y2   a21 a22 . . . a2j . . . a2p   X2 
 
   
 ..   .. .. .. .. .. ..   .. 
 .   . . . . . .   
    . 
 Y = a aij . . . aip   Xi 
 
 j   i1 ai2 . . . 
 .   . .. .. .. .. ..   . 
 ..   .. . . . . .   . 
    . 
Ym am1 am2 . . . amj . . . amp Xp
The mean and variance of Y are given by

E[Y] = E[AX] = Aµ
Var[Y] = Var[AX] = AΣA′

Example: Let Y = [Y1 , Y2 , Y3 ]′ be a random vector having mean vector µ = [2, 3, 1]′ and
variance-covariance matrix
 
5 3 1
 
Σ= 3 7 2 
1 2 8
Find
(i) the mean and variance of Z = −2Y1 + Y2 + 3Y3

(ii) the correlation coefficient between X1 = Y1 + 2Y2 + 4Y3 and X2 = Y1 − Y2 − Y3

Solution
(i)
 
h i Y1
 
Z= −2 1 3 =  Y2 
Y3
hence the mean and variance of Z are
 
h i 2
 
E[Z] = −2 1 3 = 3 =2
1
and
 
h i 5 3 1
 
V ar[Z] = −2 1 3  3 7 2  = 53
1 2 8

8
(ii)
 
" # " # Y1
X1 1 2 4  
=  Y2 
X2 1 −1 −1
Y3

the variance-covariance matrix is

  
" # 5 3 1 1 1 " #
1 2 4    213 −47
Σ=  3 7 2   2 −1  =
1 −1 −1 −47 16
1 2 8 4 −1

The correlation matrix is

" # " #
1 √ −47 1 −0.805
213×16
=
√ −47 1 −0.805 1
213×16