Scribd
Upload
185 views

6-A Prediction Problem

The document discusses linear prediction problems in signal processing. It explains that linear prediction aims to predict future sample values of a stationary process using a linear function of past sample values. Both forward and backward prediction are considered. Forward prediction predicts the next sample, while backward prediction predicts a previous sample. The optimal linear prediction coefficients are determined by minimizing the prediction error, which leads to normal equations also known as the Wiener-Hopf or Yule-Walker equations. The Levinson-Durbin algorithm provides an efficient recursive method for solving these normal equations. Lattice predictors provide an alternative implementation of linear prediction using an all-pass filter structure.

Uploaded by

Prabhjot Khurana
Copyright
© © All Rights Reserved
Available Formats
Download as PPT, PDF, TXT or read online on Scribd
185 views

6-A Prediction Problem

The document discusses linear prediction problems in signal processing. It explains that linear prediction aims to predict future sample values of a stationary process using a linear function of past sample values. Both forward and backward prediction are considered. Forward prediction predicts the next sample, while backward prediction predicts a previous sample. The optimal linear prediction coefficients are determined by minimizing the prediction error, which leads to normal equations also known as the Wiener-Hopf or Yule-Walker equations. The Levinson-Durbin algorithm provides an efficient recursive method for solving these normal equations. Lattice predictors provide an alternative implementation of linear prediction using an all-pass filter structure.

Uploaded by

Prabhjot Khurana
Copyright
© © All Rights Reserved
Available Formats
Download as PPT, PDF, TXT or read online on Scribd
You are on page 1/ 31

AG

A Prediction Problem

DS
P

Problem: Given a sample set of a stationary


processes

{x[n], x[n 1], x[n 2],..., x[n M ]}

to predict the value of the process some time


into the future as

x[n m] f ( x[n], x[n 1], x[n 2],..., x[n M ])

The function may be linear or non-linear. We


concentrate only on linear prediction functions

Professor A G

AG
C

A Prediction Problem

DS
P

Linear Prediction dates back to Gauss in


the 18th century.
Extensively used in DSP theory and
applications (spectrum analysis, speech
processing, radar, sonar, seismology,
mobile telephony, financial systems etc)
The difference between the predicted
and actual value at a specific point in
time is caleed the prediction error.
Professor A G

AG
C

A Prediction Problem

DS
P

The objective of prediction is: given


the data, to select a linear function
that minimises the prediction error.
The Wiener approach examined
earlier may be cast into a
predictive form in which the desired
signal to follow is the next sample
of the given process
Professor A G

Forward & Backward


Prediction

AG
C
DS
P

If the prediction is written as

x[n] f ( x[n 1], x[n 2],..., x[n M ])

Then we have a one-step forward


prediction
If the prediction is written as

x[n M ] f ( x[n], x[n 1], x[n 2],..., x[n M 1])

Then we have a one-step backward


prediction
Professor A G

Forward Prediction
Problem

AG
C
DS
P

The forward prediction error is then


e f [n] x[n] x[n]
Write the prediction equation as
M

x[n] w[k ]x[n k ]


k 1

And as in the Wiener case we


minimise the second order norm of
the prediction error
Professor A G

Forward Prediction
Problem

AG
C
DS
P

Thus the solution accrues from


2
2
J min E{(e f [n]) } min E{( x[n] x[n]) }

Expanding we have
2
2
J min E{( x[n]) } 2 E{( x[n]x[n]) E{( x[n]) }

Differentiating with resoect to the


weight vector we obtain
J
x[n]
x[n]
2 E{( x[n]
) 2 E{x[n]
}
wi
wi
wi

Professor A G

Forward Prediction
Problem

AG
C
DS
P

However x[n] x[n i ]


wi
And hence

J
2 E{( x[n]x[n i ]) 2 E{x[n]x[n i ]}
wi

or

M
J
2 E{( x[n]x[n i ]) 2 E{ w[k ]x[n k ]x[n i ]}
wi
k 1

Professor A G

Forward Prediction
Problem

AG
C
DS
P

On substituting with the


correspending correlation
sequences
we have
M
J
2r[i ] 2 w[k ]rxx [i k ]
wi
k 1

Set this expression to zero for


M
minimisation
to yield
w[k ]rxx [i k ] rxx [i ] i 1,2,3,..., M
k 1

Professor A G

Forward Prediction
Problem

AG
C
DS
P

These are the Normal Equations, or


Wiener-Hopf , or Yule-Walker equations
structured for the one-step forward
predictor

In this specific case it is clear that we


need only know the autocorrelation
propertities of the given process to
determine the predictor coefficients
Professor A G

AG
C

Forward Prediction Filter

DS
P

1
m0
aM [ m] w[m] m 1,.., M
0
mM

Set

And rewrite earlier expression as

rxx [0]
k 0
aM [m]rxx [m k ]
0 k 1,2,..., M
m 0
M

These equations are sometimes known as


the augmented forward prediction normal
equations
Professor A G

10

AG
C

Forward Prediction Filter

DS
P

The prediction error is then given


M
as
e f [n] aM [k ]x[n k ]
m 0

This is a FIR filter known as the


1
2
M
prediction-error
filter
A f ( z ) 1 a1[1]z aM [2]z ... aM [ M ]z

Professor A G

11

Backward Prediction
Problem

AG
C
DS
P

In a similar manner for the backward


prediction case we write

eb [n] x[n M ] x[n M ]

And
M

~[k ]x[n k 1]
x[n M ] w
k 1

Where we assume that the backward


predictor filter weights are different
from the forward case
Professor A G

12

Backward Prediction
Problem

AG
C
DS
P

Thus on comparing the the forward and


backward formulations with the Wiener
least squares conditions we see that the
desirable signal is now

x[n M ]

Hence the normal equations for the


backward case can be written as
M

w~[m]rxx [m k ] rxx [ M k 1]

m 1

Professor A G

k 1,2,3,..., M
13

Backward Prediction
Problem

AG
C
DS
P

This can be slightly adjusted as

w~[ M m 1]rxx [k m] rxx [k ]

m 1

k 1,2,3,..., M

On comparing this equation with the


corresponding forward case it is seen that
the two have the same mathematical form
and
~

w[m] w[ M m 1]

Or equivalently
~

w[m] w[ M m 1]
Professor A G

m 1,2,..., M

m 1,2,..., M
14

AG
C

Backward Prediction Filter

DS
P

Ie backward prediction filter has the same


weights as the forward case but reversed.

Ab ( z ) aM [ M ] aM [ M 1]z 1 aM [ M 2]z 2 ... z M

This result is significant from which many


properties of efficient predictors ensue.
Observe that the ratio of the backward

prediction error filter to the forward


prediction error filter is allpass.
This yields the lattice predictor structures.
More on this later
Professor A G

15

AG
C

Levinson-Durbin

DS
P

Solution of the Normal Equations


The Durbin algorithm solves the following

R m w m rm

Where the right hand side is a column R


of
as in the normal equations.
Assume we have a solution for

R k w k rk
1 k m
T
Where
rk [r1 , r2 , r3 ,..., rk ]
Professor A G

16

AG
C

Levinson-Durbin

DS
P

For the next iteration the normal equations


can be written as
*

Where

Set

Rk
T
rk J k

rk
rk 1

rk 1
zk
w k 1
k

J k rk
w k 1 rk 1
r0

Professor A G

Jk

Is the k-order
counteridentit
y

17

AG
C

Levinson-Durbin

DS
P

Multiply out to yield


1
k

1
*
k k k

z k R (rk k J r ) w k k R J r

*
k k

Note that
Hence

R k1J k J k R k1

zk wk k Jk w

*
k

Ie the first k elements ofw k 1 are


adjusted versions of the previous solution
Professor A G

18

AG
C

Levinson-Durbin

DS
P

The last element follows from the


second equation of

Ie

Rk
T
rk J k

J k rk*

r0

w k rk
r
k k 1

1
k (rk 1 rkT J k z k )
r0
Professor A G

19

AG
C

Levinson-Durbin

DS
P

k
The parameters
are known as
the reflection coefficients.
These are crucial from the signal
processing point of view.

Professor A G

20

AG
C

Levinson-Durbin

DS
P

The Levinson algorithm solves the


problem
R my b

In the same way as for Durbin we


keep track of the solutions to the
problems R k y k b k
Professor A G

21

AG
C

Levinson-Durbin

DS
P

Thus assumingw k y, k
to be
known at the k step, we solve at
the next step the problem
Rk
T
rk J k

*
k k

J r
r0

vk bk
b
k k 1

Professor A G

22

AG
C

Levinson-Durbin

DS
P

vk
y k 1

Where

Thus
1
*
*
v k R k (b k k J k rk ) y k k J k y k

bk 1 r J y
k
r0 r y

T
k k k
T *
k k

Professor A G

23

AG
C

Lattice Predictors

DS
P

Return to the lattice case.


We write

Ab ( z )
TM ( z )
Af ( z)

or

aM [ M ] aM [ M 1]z 1 aM [ M 2]z 2 ... z M


TM ( z )
1 a1[1]z 1 aM [2]z 2 ... aM [ M ]z M
Professor A G

24

AG
C

Lattice Predictors

DS
P

The above transfer function is allpass of order


M.
It can be thought of as the reflection coeffient
of a cascade of lossless transmission lines, or
acoustic tubes.
In this sense it can furnish a simple algorithm
for the estimation of the reflection coefficients.
We strat with the observation that the transfer
function can be written in terms of another
allpass filter embedded in a first order allpass
structure

Professor A G

25

AG
C

Lattice Predictors

DS
P

This takes the form


1 z 1TM 1 ( z )
TM ( z )
1 1 z 1TM 1 ( z )
TM 1 ( z )
Where 1 is to be chosen to make
of degree (M-1) .
From the above we have
TM ( z ) 1
TM 1 ( z ) 1
z (1 1TM ( z ))
Professor A G

26

AG
C

Lattice Predictors

DS
P

And hence

(aM 1[ M ] aM 1[ M 1]z 1 ... z M


TM ( z ) 1
z (1 aM 1[1]z 1 aM 1[2]z 2 ... aM 1[ M ]z M )
Where
aM [r ] 1aM [ M r ]
aM 1[r ]
1 1aM [ M ]

Thus for a reduction in the order the


constant term in the numerator, which is
also equal to the highest term in the
denominator, must be zero.
Professor A G

27

AG
C

Lattice Predictors

DS
P

1 aM [ M ]
This requirement yields
The realisation structure is

1
TM (z )
z

TM 1 ( z )

Professor A G

28

AG
C

Lattice Predictors

DS
P

There are many rearrangemnets that can be


made of this structure, through the use of
Signal Flow Graphs.
One such rearrangement would be to
reverse the direction of signal flow for the
lower path. This would yield the standard
Lattice Structure as found in several
textbooks (viz. Inverse Lattice)
The lattice structure and the above
development are intimately related to the
Levinson-Durbin Algorithm
Professor A G

29

AG
C

Lattice Predictors

DS
P

The form of lattice presented is not the


usual approach to the Levinson algorithm in
that we have developed the inverse filter.
Since the denominator of the allpass is also
the denominator of the AR process the
procedure can be seen as an AR coefficient
to lattice structure mapping.
For lattice to AR coefficient mapping we
follow the opposite route, ie we contruct the
allpass and read off its denominator.
Professor A G

30

AG
C

PSD Estimation

DS
P

It is evident that if the PSD of the


prediction error is white then the
prediction transfer function multiplied
by the input PSD yields a constant.
Therefore the input PSD is determined.
Moreover the inverse prediction filter
gives us a means to generate the
process as the output from the filter
when the input is white noise.
Professor A G

31

You might also like