Estimation for Wireless Communication –MIMO/OFDM on Cellular and Sensor

Networks
Professor A K Jagannatham
Department of Electrical Engineering
Indian Institute of Technology, Kanpur
Lecture Number 03
Properties of Maximum Likelihood (ML) Estimate- Mean and Unbiasedness
Hello, welcome to another module in this massive open online course on estimation for
wireless communication. So far we have looked at a simple model of a wireless sensor
network, we have formulated the likelihood function and based on the likelihood function we
have derived the maximum likelihood estimate of the unknown parameter.

So far what we have done is we have considered a simple wireless sensor network an
example of a wireless, a scenario rather of a wireless sensor network.

(Refer Slide Time: 00:57)

We have looked at N observations in this wireless sensor network, the observations are y 1
equals h + v 1, y 2 equals h + v 2, y N equals h + v N where these y 1, y 2, y N is are our
observations, right?

h is the unknown, h is the unknown parameter and these v 1, v 2, v N, these are IID
independent, so v 1, v 2, v N, these are IID that is Independent Identically Distributed
Gaussian noise samples with means 0 and variance Sigma Square. These are IID Gaussian
noise samples, all right.
So these are IID Gaussian noise samples with mean 0 and variance equal to Sigma Square.
These are IID Gaussian noise samples with mean 0 and variance equal to Sigma Square and
for this scenario we had derived the Likelihood Function. We had derived the Likelihood
Function for this scenario. And we have shown that the maximum likelihood estimate is
given by the sample mean.

(Refer Slide Time: 02:52)

The ML estimate, the Maximum Likelihood Estimate or the ML estimate that is h hat, the
Maximum Likelihood Estimate of the unknown parameter is basically one over N submission
k equal to 1 to N y of k. That is, this is the also known as the, the sample mean or simply
basically your average of observations y 1. This is the sample mean that is h hat which is the
Maximum Likelihood Estimate.

What is this; this is the ML estimate of the unknown parameter h, which is given by this
average that is, 1 over N submission k equals 1 to N y of k. That is the average of the
observations y 1, y 2, y N are also basically the sample mean. That is the sample is y 1, y 2, y
N. This is the mean or the average of the sample therefore, this is also known as the sample
mean.

This is the Maximum Likelihood Estimate for the simple scenario, simple wireless sensor
network scenario with N observations that we have considered so far of sensing an unknown
parameter h which we said can either be the parameters such as the temperature, pressure, the
humidity, et cetera yeah. So now let us examine the behavior or the properties of this
maximum likelihood estimate.
So in starting this module, let us explore the properties. Let us explore the properties of the
Maximum Likelihood Estimate.

(Refer Slide Time: 05:40)

Now remember we have h hat equals 1 over N submission k equals 1 to N y k. And recall
that each h k each y k observation y k is random in nature. In fact, each y k is random
Gaussian if you remember the very first module, we had said that these ys; y 1, y 2 1 y n are
noisy observations with mean given by the unknown parameter h and the variance given by
the variance Sigma Square of the noise V k.

Therefore each y k, that is it is important to stress this aspect each y k is random in nature, in
fact random Gaussian, which means h hat which is the sample mean or average of these y k is
also in nature. h hat in fact h hat equals the average of these different observations y k, your h
hat is the average of these different observations Y k, the Y is the observation, the noisy
observations are themselves random in nature, therefore h hat is random.
(Refer Slide Time: 07:02)

That is the estimate is random in nature, that is the first thing that we have to realise, that is
this estimate, the estimate h hat is random that is, it is not a deterministic quantity, it is
random in nature. Now since h hat is random in nature, it is not necessary that it is equal to
true parameter h. Therefore, h hat is a random quantity therefore it is not always equal to the
true parameter h.

Therefore, h hat is not necessarily equal to infact it is not necessarily equal to h.

(Refer Slide Time: 07:56)

Therefore, the estimate this is the estimate h hat is the estimate h is the true parameter, the
underlying the true parameter which is unknown, right. So we have the estimate h hat which
is derived from the observations and then we have the true parameter that is the original
parameter h which is unknown and which we are trying to measure.

And what we are saying is h hat is random, therefore it is not necessary that hat is actually
equal to h. And therefore we want to find how good how good is this estimate h hat. What is
the behaviour of this h hat, I mean what kind of properties does this h hat exhibit and how
close it is to the true parameter h. Since it is not exactly equal to h, can we at least hope in
some sense that it is close to the true parameter h.

So h hat is random, therefore what we want to do if we want to explore what is the behaviour
of h or what is the behaviour of h hat. And in some sense we want to explore, we want to
address this important question how close is h hat in what sense, how close is the estimate h
hat to the true, how close is this estimate h to the true parameter h. Now in that sense let us
start by computing the mean.

(Refer Slide Time: 10:24)

Let us start by computing the mean and variance of this estimate. So to address this quotient,
to address what is the behaviour of this estimate h hat and also how close is this estimate h
hat to the true value of the underlying parameter h. Let us start, let us start by computing the
mean and variance of this random quantity of this estimate h hat which is random in nature.
For that once again recall that h hat is the sample mean.

h hat equals 1 over n submission k equals 1 y k, this is the sample mean. Observe that this is
the linear combination of Gaussian random variables, linear combination of Gaussian random
variables y1, y 2 up to y n. Remember we started in the very first module itself. We said that
y k each y k is Gaussian in nature because the noise V k is Gaussian in nature; you are
shifting that by the constant mean that is h.

Therefore, each y k is also Gaussian in nature. And further what we have is this estimate h hat
which is the sample mean h hat is the sample mean, therefore h hat is the linear combination
of these observations y1, y 2 upto y k. Now since h hat is the linear combination of these
Gaussian random variables y1, y 2 y k, h hat itself is Gaussian in nature because the linear
combination of Gaussian random variables is Gaussian itself.

h hat is the linear combination of Gaussian random variables, h hat itself is Gaussian in
nature. So this h hat is Gaussian in nature, since it is a linear combination of Gaussian
random variables. So the first thing that we have established is that the maximum likelihood
estimate h hat itself is a Gaussian random, it is a random variable and in fact it is Gaussian
random variables, hence it is a linear combination of Gaussian random variables, okay.

(Refer Slide Time: 13:25)

So h hat is Gaussian nature, so h hat is Gaussian in nature. Now what is the mean of h hat,
now what is the main or the expected value of, what is the expected value of h hat? Now
recall, for this now recall each y k each y k is h + V k, yeah. And further we have remember
we assumed the noise to be 0 means. That is we have expected V k for each V k we have
expected V k 0, this is because the noise is 0 mean, all right.

So we are going to use this property, remember in each observation V k, in each noisy
observation in each our noisy observation y k, the noise V k has mean 0. Now let us look at h
hat, h hat is in fact 1 over n submission k equal to 1 to n y k.
(Refer Slide Time: 14:40)

Now substituting this expression for small y k, remember each y k is h + v k, so this is

submission k equal to 1 to n, h + v k which is equal to now divided by n.

(Refer Slide Time: 15:06)

I have submission h + V k which is n times h + submission k equals 1 to n V k which cannot

be simplified as h + submission 1 over n k equals 1 to n V k, this is my h hat. So I have
simplified this expression for h hat, h hat equals that is the estimate. Remember this is h hat
which is; let us not forget what these are. This is the estimate h hat, this is the true parameter
h and these V ks these are the noise, the observation noise.
Such that each expected V k equal to 0. Therefore, now if we look at the average value of the
estimate h hat or the expected value of the estimate h hat.

(Refer Slide Time: 16:30)

I have the expected value of h hat is given as the expected value of the expression above
which is the expected value of h + 1 over n submission k equal to 1 to n V k which is equal to
expected value of h + expected value of 1 over n submission k equal to 1 to n V k.

Now look at this, this is a constant, so the expected value of h is basically h, so this is equal to
your h + 1 over n submission k equal to 1 to n expected value of each V k. Remember we
said the noise is 0 mean, so expected value of each V k equals 0, therefore net what we have
is that basically your expected value of h hat, that is your the average value of the estimate
the expected value of h hat equals h and this is a very important property.

That is, even though what we are saying is even though your estimate h hat is random in
nature that is, h hat is not necessarily always equal to h. However, it exhibits a very important
property and the property that it exhibits is that on an average, that is if you look at expected
value of h hat, that is the average of h hat is basically equal to the true parameter h and this is
a very important property of the estimate.

That is it says that the mean of the estimate is equal to the true parameter h, such an estimator
is known such an estimate is known as in “Unbiased estimate”. This is an important property,
they says that basically the average value of the estimate. What does this signify; this
signifies that average value of estimate equals true value of the parameter h.
(Refer Slide Time: 19:14)

Therefore, such an estimator such an estimator is known as in is known as an Unbiased that

is, it has no bias that is, it is no preference, it is an unbiased estimate or an unbiased. It is
known as, such an estimator is basically known as basically known as an “Unbiased
Estimator” that is, it yields in unbiased estimate.

Which means even though it is random in nature, the average value of the estimate is
basically equal to the underlying, the true underlying parameter h. And therefore the ML
estimate that we have derived is unbiased in nature. Let us also write that down, let us also
note that down. Therefore, the ML estimate or in another words our sample mean that is, the
sample mean is unbiased in nature yeah.

(Refer Slide Time: 20:59)

Which means that what do we have, we have expected value of h hat is equal to h, what is
this, this is the average value of the estimate and what is this, this is the true value of the
underlying parameter. This is the true value of the underlying parameter and what we are
saying is, on an average, the average value of the estimate is equal to the true value of the
underlying parameter.

Therefore, the estimate or therefore this maximum likelihood estimate is unbiased in nature
which is a very important property of the estimate. In fact it is a very desirable property of the
estimate yeah. So this unbiased property is a very important and desirable property of the
estimate, which says that on an average, the average value or the mean of the estimate is
equal to the true value of the underlying parameter, all right.

So in this module what we have looked at is, we have started characterising the behaviour of
this ML estimate yeah, which is basically for a simple sensor network with noisy
observations of an underlying parameter, we have said that the maximum likelihood estimate
is the sample mean of the observation and we have also reduced that even though this is
random and, random Gaussian in nature.

That is the estimate h hat is a Gaussian random variable that is, it is not necessarily always
equal to the true underlying parameter h. On an average, that is the expected value of this
estimate or this estimator is equal to the true value of the underlying parameter h and
therefore this maximum likelihood estimate is an unbiased estimate which is an important
property.

So we will stop this module here and continue with other properties in the subsequent
modules, thank you very much.