Scribd
Upload
374 views61 pages

Wiener Filter - Last22

Linear estimation and prediction techniques are important problems in signal processing. Optimal digital filters like the Wiener filter and Kalman filter can produce the best estimates of signals from noisy or distorted observations. [1] The Wiener filter design aims to minimize the mean square error between the estimated and desired signals. [2] It provides a framework for problems like filtering, smoothing, prediction, and deconvolution. [3] The Wiener-Hopf equations give the filter coefficients that solve the estimation problem for both FIR and IIR filters.

Uploaded by

abu
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
374 views61 pages

Wiener Filter - Last22

Linear estimation and prediction techniques are important problems in signal processing. Optimal digital filters like the Wiener filter and Kalman filter can produce the best estimates of signals from noisy or distorted observations. [1] The Wiener filter design aims to minimize the mean square error between the estimated and desired signals. [2] It provides a framework for problems like filtering, smoothing, prediction, and deconvolution. [3] The Wiener-Hopf equations give the filter coefficients that solve the estimation problem for both FIR and IIR filters.

Uploaded by

abu
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
You are on page 1/ 61

Linear Estimation

&
Prediction
Introduction
• Estimation of one signal from another is one of the most
important problems in signal processing
• In many applications, the desired signal is not available or
observed directly.
– Speech, Radar, EEG etc
• Desired signal may be noisy and highly distorted
• In very simple and idealized environments, it may be possible to
design classical filters such as LP,HP or BP, to restore the
desired signal from the measured data.
Introduction
• The signal estimation problem can be stated as follows:

• The objective is to estimate a random signal xn on the basis of


available observations of a related signal yn .
• The resulting estimate xˆn will be a function of the observations yn
• Linear estimation utilizes linear mean square (LMS) criterion
which requires the estimate to be a linear function of the
observations:
Interval of estimation
Introduction

 classical filters shall rarely be optimal in the sense of producing the “best”
estimate of the signal.
 Class of filters called OPTIMUM DIGITAL FILTERS are popularly used for
signal estimations.
 OPTIMUM DIGITAL FILTERS are two types:
 Digital Wiener Filter
 Discrete Kalman Filter
Wiener Filter
• In the 1940’s, Norbert Wiener pioneered research in the problem of designing
a filter that would produce the optimum estimate of a signal from a noisy
measurement or observation.
Wiener Filter
• The Wiener filtering problem, is to design a filter to recover a
signal d[n] from noisy measurement

• Assuming that both d[n] and v[n] are wide-sense stationary


random process, Wiener considered the problem of designing
the filter that would produces the minimum mean square
estimate of d[n] (by using x[n])
Wiener Filter
• Thus the error terms are (Mean Square Error)

• Problem is to find the filter (filter coefficients, FIR or IIR) that


minimizes ξ (minimum mean square error).
Wiener Filter
• Depending on how the signals x[n] and d[n] are
related to each other, a number of different and
important problems may be cast into Wiener
filtering framework.
These problems are
• Filtering
• Smoothing
• Prediction
• De-convolution
Wiener Filter
Wiener Filter
The FIR Wiener Filter
• Design of an FIR Wiener Filter
– That will produce the minimum mean-square
estimate of a given process d[n] by filtering a set
of observations of statistically related process x[n]
• It is assumed that x[n] and d[n] are jointly
wide-sense stationary with known
autocorrelations rx[k] and rd[k], and known
cross-correlation rdx[k]
The FIR Wiener Filter
• Denoting the unit sample response of the
Wiener filter by w[n], and assuming (p-1)st
order filter, the system function W(z) is

• With x[n] as input to this filter, the output is


(using DT convolution)
The FIR Wiener Filter
• Wiener filter design problem requires that we
find the filter coefficients w[k], that minimize
the mean-square error:

• Using optimization steps, for k=0,1,…,p-1


Error

Taking Partial
Derivative for
kth value

Orthogonality Principle

After putting e[n] back


The FIR Wiener Filter
• Since x[n] and d[n] are jointly WSS

• Set of ‘p’ linear equations in the ‘p’ unknowns


w[k] for k=0,1,….,p-1
The FIR Wiener Filter
• In matrix form using the fact that
autocorrelation sequence is conjugate
symmetric rx[k]=r*x[-k]

Wiener-Hopf Equations
The FIR Wiener Filter
• Wiener-Hopf Equations
The FIR Wiener Filter
• The minimum mean square error in the
estimate of d[n] is

Equal to zero bz of Orthognality


The FIR Wiener Filter
• After taking expected values

In Vector Notation
The FIR Wiener Filter
Filtering
• In the filtering problem, a signal d[n] is to be
estimated from a noise (v[n]) corrupted
observation x[n]

• Assuming that noise has a zero mean and it is


uncorrelated with d[n]
Filtering
• The cross-correlation between d[n] and x[n]
becomes
Filtering
• With v[n] and d[n] uncorrelated, it follows

• To simplify these equations, specific information about the


statistic of the signal and noise are required
Example
• Let d(n) be a process with an autocorrelation sequence
Example
Linear Prediction
• With noise-free observations, linear prediction
is concerned with the estimation (prediction) of
x[n+1] in terms of linear combination of the
current and previous values of x[n]
Linear Prediction
• An FIR linear predictor of order ‘p-1’ has the
form

– where w[k] for k=0,1,…,p-1 are the coefficients of


the prediction filter.
• Linear predictor may be cast into Wiener
filtering problem by setting d[n]=x[n+1]
Linear Prediction
• Setting up the Wiener-Hopf equations
Example
• find the optimum linear predictor for x(n) that has an
autocorrelation sequence given by:
Example
Linear Prediction in noise
• With noise present, a more realistic model for
linear prediction is the one in which the signal
to be predicted is measured in the presence
of noise.
Linear Prediction in noise
• Input to Wiener filter is given by

• Goal is to design a filter that will estimate


x[n+1] in terms of linear combination of ‘p’
previous values of y[n]
Linear Prediction in noise
• The Wiener-Hopf equations are

• If the noise is uncorrelated with signal x[n],


then Ry, the autocorrelation matrix for y[n] is
Linear Prediction in noise
• The only difference between linear prediction
with and without noise is in the
autocorrelation matrix for the input signal.
• In the case of noise that is uncorrelated with
x[n],
Example
• Let the measurement of x(n) in the previous example is noisy and
suppose that:
Example
Multi-Step Prediction
• In one-step linear prediction, x[n+1] is
predicted in terms of linear combination of the
current and previous values of x[n]
• In multi-step prediction, x[n+δ] is predicted in
terms of linear combination of the ‘p’ values
x[n],x[n-1],…,x[n-p+1]
Multi-Step Prediction
• In multi-step prediction

• In multi-step prediction, since


Positive Integer
Multi-Step Prediction
• Wiener-Hopf equations are
Example
• Let us consider the problem of multistep prediction for a random
process having an autocorrelation sequence of the form:
Example
Noise Cancellation
• The goal of noise canceller is to estimate a signal d[n] from a
noise corrupted observation, that is recorded by primary
sensor.

• Unlike the filtering problem, which requires that the


autocorrelation of the noise be known, with noise canceller
this information is obtained from a secondary sensor that is
placed within the noise field.
Noise Cancellation

Primary sensor

Secondary sensor
Noise Cancellation
• Although the noise measured by secondary
sensor, v2[n], will be correlated with the noise in
the primary sensor v1[n], the two will not be
same.
• Reasons for being not same:
– Difference in sensor characteristics
– Difference in the propagation path from noise source
to the two sensors.
• Since v1[n]≠v2[n], it is not possible to estimate
d[n] by simply subtracting v2[n] from x[n]
Noise Cancellation
• Noise canceller consists of Wiener filter that is
designed to estimate the noise v1[n] from the
signal received by the secondary sensor
• This estimate is then subtracted from the
primary signal x[n] to form an estimate of d[n]
which is given by
Noise Cancellation
• With v2[n] as the input to Wiener filter, that is
used to estimate the noise v1[n], the Wiener-
Hopf equations are

• Rv is the autocorrelation matrix of v2[n]


2

• rv1,v2 is the vector of cross-correlations


between desired signal v1[n] and Wiener filter
input v2[n]
Noise Cancellation
• The cross-correlation between v1[n] and v2[n]
is

• If we assume v2[n] is uncorrelated with d[n],


then second term is zero, hence
Example
• Suppose that the desired signal d(n) is a sinusoid:
IIR Wiener Filter
• For IIR Wiener filter, there are an infinite number of unknowns, i.e., the values of
h (n) for all n.
• For a non-causal (unconstrained) IIR Wiener filter, the problem is to find the unit
sample response, h(n), of the IIR filter:
IIR Wiener Filter

 Using the previous equations and rearranging the terms:

Hence,

 the solution to the above equations is straightforward if we write the left side of the
equation as the convolution of h(k) with rx(k),
IIR Wiener Filter

 Using Parseval's theorem this error may be expressed in the frequency domain as follows:
IIR Wiener Filter
The Causal IIR Wiener Filter
• For a causal filter, the unit sample response is zero for n < 0 and the
estimate of d (n) takes the form:

 To find the filter coefficients that minimize the mean-square error, we proceed in exactly
the same way that we did for the non-causal Wiener filter.
The Causal IIR Wiener Filter
the previous equation reduced to:
The Causal IIR Wiener Filter
 If x(n) is filtered with a filter having a system function of the form shown below:

 Let H (z) be the causal Wiener filter that produces the minimum mean-square
estimate of d(n) from x(n), and suppose that x(n) is filtered with a cascade of three
filters, F(z), , , and H(z) as shown in Fig:
The Causal IIR Wiener Filter
The Causal IIR Wiener Filter
The Causal IIR Wiener Filter

In the frequency domain, this error may be written as :


Causal Wiener Filtering
Let’s consider the problem of causal Wiener filtering for estimating a process d(n)
from the noisy observations:

You might also like