Topic:

Professor : Hyung Yun Kong

Student : Pham Thanh Tuan

WIRELESS COMMUNICATION LAB

 Contents

1. Introduction
2. Estimation Theory
• Least squares estimation (LSE)
• Maximum likelihood estimation (MLE)
• Minimum mean square error (MMSE) estimation
• Maximum a posteriori (MAP) estimation
• Kalman filter
3. Detection Theory
• Neyman – Pearson detection
• Matched filter detection
• Energy detection
• Cyclostationary feature detection (CFD)
4. Application
2 WIRELESS COMMUNICATION LAB
 Introduction

• Infer value of unknown state of nature based on noisy observations

Mathematically, optimally
Noise

Transmission/
Nature Processing
measurement

phenomenon model of observation decision rule

experiment or transmission estimation function
the “truth” process
mapping from
observation space
model of hypothesis ෡
𝑃𝑌|𝐻 (𝑦|ℎ) Y to detection/ 𝐻
H
estimates

3 WIRELESS COMMUNICATION LAB

 Introduction

• Detection example 1: Digital communications

4 WIRELESS COMMUNICATION LAB

 Introduction

• Detection example 2: Radar communications

5 WIRELESS COMMUNICATION LAB

 Introduction

• Estimation example: Radar communications

6 WIRELESS COMMUNICATION LAB

 Least squares estimation

Why use LSE?

Main benefit: do NOT need a statistical model!
But we need a signal model

s[n;𝜃] strue[n;𝜃]
signal x[n] = strue[n;𝜃] + w[n]
∑ ∑
model + + ≜ s[n;𝜃] + e[n]
+ +

𝛿𝑛 w n noise Model &

model (measurement measurement error
error error)

7 WIRELESS COMMUNICATION LAB

 Least squares estimation

What is the LSE?

• Suppose we have data x = [𝑥 0 , 𝑥 1 , … , 𝑥(𝑁 − 1)] 𝑇 , signal model

s 𝜃 = 𝑠 0, 𝜃 , 𝑠 1, 𝜃 , … , 𝑠(𝑁 − 1, 𝜃) 𝑇 .

The least squares estimate (LSE) of 𝜃 is:

𝑁−1
2 𝑇
𝜃መ𝐿𝑆𝐸 = 𝑎𝑟𝑔min 𝐽(𝜃) = 𝑎𝑟𝑔min ෍ 𝑥 𝑛 − 𝑠 𝑛, 𝜃 = x−s 𝜃 x−s 𝜃
𝜃 𝜃
𝑛=0

• The LSE minimizes the distance or energy between the data and the signal
model with the estimated value of 𝜃.

8 WIRELESS COMMUNICATION LAB

 Maximum likelihood estimation

• Minimum variance unbiased estimators are often difficult to find, in which

case we can resort to sub-optimal, but tractable estimators such as the
Maximum Likelihood Estimator (MLE).
• Consider estimating 𝜃 from a set of observations x. The likelihood function
p(x; θ) is a function of 𝜃 for a given, fixed x.
• The MLE is the value of 𝜃 that maximizes the likelihood function, that is

Maximum Likelihood Estimator:

𝜃መ𝑀𝐿𝐸 = 𝑎𝑟𝑔max 𝑝 𝑥|𝜃
𝜃

9 WIRELESS COMMUNICATION LAB

 Maximum likelihood estimation

Properties of the MLE:

• The MLE is asymptotically unbiased (i.e. consistent: converges in
probability to the true parameter value).
• The MLE is asymptotically efficient.
• The MLE is asymptotically distributed as 𝑁 𝜃, 𝐼−1 𝜃 , where I(𝜃) is the
Fisher information.

10 WIRELESS COMMUNICATION LAB

 Minimum mean square error (MMSE)

The MMSE estimator is determined with minimize the Bayesian risk under the
quadratic cost function. We saw that

𝜃መ𝑀𝑀𝑆𝐸 = න 𝜃𝑝 𝜃|𝑥 𝑑𝜃 = 𝐸 𝜃|x

What is the MMSE?

The MMSE estimator is the mean of the posterior density 𝑝 𝜃|𝑥 .
Unlike in classical MSE estimator theory, this estimator does not depend on
the value of 𝜃, which has been averaged out.
Classical MSE estimation MMSE estimation
2 2 2
𝜃መ𝑀𝑆𝐸 = න 𝜃መ − 𝜃 𝑝 𝑥|𝜃 𝑑𝑥 𝜃መ𝑀𝑀𝑆𝐸 = 𝐸 𝜃መ − 𝜃 = න න 𝜃መ − 𝜃 𝑝 𝑥|𝜃 𝑑𝑥𝑑𝜃

Depends on 𝜃! Dependence on 𝜃 is averaged out!

11 WIRELESS COMMUNICATION LAB

 Minimum mean square error (MMSE)

Properties of the MMSE:

• The MMSE estimator for the Bayesian model becomes the MVUE
estimator for the classical linear model as the prior distribution becomes
uninformative.

• The MMSE commutes over affine transformations.

• When x = x1 , x2 T and 𝜃 are jointly Gaussian, the MMSE is additive for

independent data set x1,x2.
θ෠ MMSE = E θ + Cθx1 Cx−1
1 x1
x1 − E x1 + Cθx2 Cx−1
2 x2
x2 − E x2

12 WIRELESS COMMUNICATION LAB

 Maximum a posteriori (MAP) estimation

Maximum a posteriori (MAP) estimator is the mode (max) of the posterior

distribution p(𝜃|x), and may be easier to compute that say the MMSE as no
integration is needed, only maximization.

The MAP estimator, it its various forms, is given by:

𝜃መ𝑀𝐴𝑃 = 𝑎𝑟𝑔max p(𝜃|𝑥)

𝜃

𝑝 𝑥|𝜃 𝑝 𝜃
= 𝑎𝑟𝑔max
𝜃 𝑝 𝑥

= 𝑎𝑟𝑔max 𝑝 𝑥|𝜃 𝑝 𝜃
𝜃

= 𝑎𝑟𝑔max ln 𝑝 𝑥|𝜃 + ln 𝑝 𝜃
𝜃

13 WIRELESS COMMUNICATION LAB

 Maximum a posteriori (MAP) estimation

Properties of the MAP:

Some interesting properties of the MAP estimator:

• As N → ∞ the MAP becomes the Maximum Likelihood Estimator.

• If x and 𝜃 are jointly Gaussian then the MAP = MMSE estimator.

• The MAP commutes over invertible linear transformations, but in contrast

to Maximum Likelihood Estimator, does not commute over non-linear
function in general.

14 WIRELESS COMMUNICATION LAB

 Kalman filter

The Kalman filter tries to estimate the state x of a discrete time controlled
process that is governed by the equation:
𝑥𝑘 = 𝐴𝑥𝑘−1 + 𝐵𝑢𝑘−1 + 𝑤𝑘−1
with measurement 𝑧𝑘
𝑧𝑘 = 𝐻𝑧𝑘 + 𝑣𝑘
Where:
• Process noise 𝑝(𝑤)~𝑁(0, 𝑄), process noise covariance Q.
• Measurement noise 𝑝(𝑣)~𝑁(0, 𝑅), measurement noise covariance R
• Matrix A relates the state at time k to the state at previous time k-1, in
absence of noise.
• Matrix B relates the optional control input to the state x.
• Matrix H relates the state to the measurement.

15 WIRELESS COMMUNICATION LAB

 Kalman filter

• Goal: Find a posteriori state estimate 𝑥ො𝑘 as a linear combination of an a

priori estimate 𝑥ො𝑘− and a weighted difference between the actual
measurement zk and a measurement prediction 𝐻𝑥ො𝑘−

xˆk  xˆk  K ( z k  Hxˆk )

• The difference (𝑧𝑘 −𝐻𝑥ො𝑘− ) is called innovation or residual.

• K is the gain or blending factor that minimizes the a posteriori error

covariance.

16 WIRELESS COMMUNICATION LAB

 Kalman filter

Time update equations (predict) Measurement update equations

(correct)
Update expected value of x
K k  Pk H T ( HPk H T  R) 1
xˆk  Axˆk 1  Buk
xˆk  xˆk  K k ( z k  Hxˆk )
Update error covariance matrix P

P  APk 1 A  Q
T Pk  ( I  K k H ) Pk
k
• Compute Kalman gain Kk
• Project state and covariance
estimates forward in time • Compute a posteriori estimate xො k
• Compute a posteriori error
covariance estimate Pk

Initial estimates for

𝑥ො𝑘−1 and 𝑃𝑘−1

17 WIRELESS COMMUNICATION LAB

 Neyman-Pearson detection

Decide between two hypothesis: ℋ0 or ℋ1 .

To do so we find a decision rule which maps the observation x into either ℋ0
or ℋ1 . Because the observation process is modelled probabilistically, the
following errors may be made:
• P(ℋ0 ;ℋ1 ) = prob(decide ℋ0 when ℋ0 is true) = prob of correct non-
detection.
• P(ℋ0 ;ℋ1 ) = prob(decide ℋ0 when ℋ1 is true) = prob of missed detection
:=PM.
• P(ℋ0 ;ℋ1 ) = prob(decide ℋ1 when ℋ0 is true) = prob of false alarm :=PFA.
• P(ℋ0 ;ℋ1 ) = prob(decide ℋ1 when ℋ1 is true) = prob of detection :=PD.
More generally P(ℋ𝑖 ;ℋ𝑗 ) is probability of deciding ℋ𝑖 when hypothesis ℋ𝑗 is
true.

18 WIRELESS COMMUNICATION LAB

 Neyman-Pearson detection

To maximize PD for a given 𝑃𝐹𝐴 = 𝛼, decide ℋ1 if

𝑝 x; ℋ1
𝐿 𝑥 ≔ >𝛾
𝑝 x; ℋ0

Where the threshold 𝛾 is found from

𝑃𝐹𝐴 = න 𝑝 x; ℋ0 𝑑x = 𝛼
x:𝐿 x >𝛾

L(x) is the likelihood ratio, and comparing L(x) to a threshold is termed the
likelihood ratio test.

19 WIRELESS COMMUNICATION LAB

 Matched filter detection

We consider detecting the presence of a known signal s[n] in Gaussian noise.

This mean, the received signal x[n] is
ℋ0 : 𝑥 𝑛 = 𝑤 𝑛
ℋ1 : 𝑥 𝑛 = 𝑠 𝑛 + 𝑤 𝑛
Where 𝑤[𝑛] is for now assumed to be white noise with variance 𝜎 2 .
Matched filter model: 𝑁−1

𝑇 𝑥 = ෍ x 𝑛 𝑠[𝑛]
𝑛=0

“Correlator”

“Matched filter”

20 WIRELESS COMMUNICATION LAB

 Matched filter detection

The matched filter implementation of the Neyman-Pearson detector weights

the samples with more energy (larger values) more heavily than those of small
energy.
The matched filter has the interesting property that it maximizes the SNR at
the output of an FIR filter.
Furthermore, its performance (PD) can be derived explicitly as a function of
PFA as

ℇ
𝑃𝐷 = 𝑄 𝑄−1 𝑃𝐹𝐴 −
𝜎2

Where ℇ is the energy in the signal, ℇ = ∑𝑁−1 2

𝑛=0 𝑠 𝑛 .

21 WIRELESS COMMUNICATION LAB

 Matched filter detection

Performance of Matched filter:

The test statistic 𝑇 𝑥 = x 𝑇 𝑠 has pdf 𝒩 0, 𝜎 2 ℇ under ℋ0 and pdf 𝒩 ℇ, 𝜎 2 ℇ

under ℋ1 . We can obtain PFA and PD as follows:

𝑃𝐹𝐴 = 𝑃𝑟 𝑇 > 𝛾 ′ |ℋ0 ⇒ 𝛾 ′ = 𝜎 2 ℇ𝑄−1 𝑃𝐹𝐴

𝛾′ − ℇ ℇ
𝑃𝐷 = 𝑃𝑟 𝑇 > 𝛾 ′ ; ℋ1 = 𝑄 = 𝑄 𝑄−1 𝑃𝐹𝐴 −
𝜎 2ℇ 𝜎2

22 WIRELESS COMMUNICATION LAB

 Energy detection

The decision static for energy detector:

𝑁−1

𝑇(𝑥) ≜ ෍ 𝑥 𝑛 2

𝑛=0

Where the test statistic T(x), the received signal x[n], N: the number of
samples.
Energy detection compares the energy of the received signal x[n] in a certain
frequency band to a threshold value (𝛾):
– T(x) > 𝛾 :
• H1: Exist signal in channel
– T(x) < 𝛾 :
• Ho: Not exist signal in channel

23 WIRELESS COMMUNICATION LAB

 Energy detection

Performance of Energy detection:

• PD, PFA is important for the evaluation of detection performance.
𝛾 − 𝑁𝜎𝑛2 − 𝑁𝑝𝑠
𝑃𝐷 = 𝑃𝑟 𝑇 > 𝛾|ℋ1 = 𝑄 ×
𝜎𝑛 2𝑁𝜎𝑛2 + 4𝑁𝑝𝑠
𝛾 − 𝑁𝜎𝑛2
𝑃𝐹𝐴 = 𝑃𝑟 𝑇 > 𝛾|ℋ0 = 𝑄 ×
𝜎𝑛2 2𝑁
Where:
• PS: the average primary signal power
• Q: the tail probability of a zero-mean unit-variance Gaussian random
variable
1 +∞ −𝑢2ൗ
𝑄 𝑥 = න 𝑒 2 𝑑𝑢
2𝜋 𝑥
• In fact, difficult to reach the PD = 1 (depend on these changes in its
environment)  Establish the threshold based on probability PFA

24 WIRELESS COMMUNICATION LAB

 Cyclostationary feature detection (CFD)

Spectral correlation function (SCF) of y(n):

H0 : S y ( f )  S w ( f )
H1 : S y ( f )  S x ( f )  S w ( f )
𝛼 𝑓 is the SCF of the noise, 𝑆 𝛼 𝑓 is the SCF of the primary user
Where 𝑆𝑤 𝑥
𝛼 𝑓 = 0 for cycle
signal. Since noise is not a cyclostationary process, 𝑆𝑤
frequency α ≠ 0.

For fixed number of samples N compute estimate of spectral correlation

function as follows:
~
1 1 N  * 

Sy ( f )  
N T n 0
YT ( n, f 
2
)YT ( n, f -
2
)
n T / 2


 j 2fu
YT (n, f )  y (u ) e du
n T / 2

25 WIRELESS COMMUNICATION LAB

 Cyclostationary feature detection (CFD)

Performance of Cylostationary feature detection:

Performance of the detector is measured in terms of output SNR, as PD and PFA

are mathematically intractable to compute.
E ( zsc | H1 ) - E(zsc| H 0 )
Output SNR is related to deflection coefficient: d 
Var( zsc | H 0 )
d 0(0) SNRin N
Energy detector: d (0) ~ 1/ 2
3
1   N (1  N ) 

d 0 ( )    S x ( f ) df 
2

2
  
d  SNRin N
Feature detector: d ( ) ~ 0
1  N

When noise variance perfectly known (ρN=0), detectors perform comparably

When noise variance unknown (ρN≠0), feature outperforms energy detector

26 WIRELESS COMMUNICATION LAB

 Application

Detection and estimation techniques form the bedrock of modern signal

processing and communication systems.

Radar communications

Sonar

Image processing

Biomedicine

Control

Seismology

27 WIRELESS COMMUNICATION LAB

28
WIRELESS COMMUNICATION LAB
 Reference
[1] Harry L. Van Trees, Detection, Estimation, and Modulation Theory, Part I, II, III, IV.

[2] H. Vincent Poor, Introduction to Signal Detection and Estimation.

[3] Louis L. Scharf and Cedric Demeure, Statistical Signal Processing: Detection, Estimation, and Time Series
Analysis.

[4] Steven M. Kay, Prentice Hall, Fundamentals of Statistical Signal Processing, Volume 1: Estimation Theory.

[5] Steven M. Kay, Prentice Hall, Fundamentals of Statistical Signal Processing, Volume 2: Detection Theory.

29 WIRELESS COMMUNICATION LAB