See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/52008168

Adaptive Filtering: Algorithms and Practical

Implementation

Chapter · January 2008

DOI: 10.1007/978-1-4614-4106-9

CITATIONS READS

706 4,803

1 author:

Paulo S. R. Diniz
Federal University of Rio de Janeiro
353 PUBLICATIONS   4,481 CITATIONS   

SEE PROFILE

Some of the authors of this publication are also working on these related projects:

CNPq-Universal Sparse Adaptive Signal Processing for Communications View project

On Data-Selective Learning View project

All content following this page was uploaded by Paulo S. R. Diniz on 26 May 2014.

The user has requested enhancement of the downloaded file.

Paulo S.R. Diniz

Adaptive Filtering
Algorithms and Practical
Implementation

Third Edition

Springer
 CONTENTS

PREFACE "X

1 INTRODUCTION TO ADAPTIVE FILTERING 1

1.1 Introduction 1
1.2 Adaptive Signal Processing 2
1.3 Introduction to Adaptive Algorithms 4
1.4 Applications 7
1.5 References 11

FUNDAMENTALS OF ADAPTIVE FILTERING 13

2.1 Introduction 13
2.2 Signal Representation 14
2.2.1 Deterministic Signals 14
2.2.2 Random Signals 15
2.2.3 Ergodicity 21
2.3 The Correlation Matrix 23
2.4 Wiener Filter 34
2.5 Linearly Constrained Wiener Filter 39
2.5.1 The Generalized Sidelobe Canceller 43
2.6 Mean-Square Error Surface 44
2.7 Bias and Consistency 47
2.8 Newton Algorithm 48
2.9 Steepest-Descent Algorithm 49
2.10 Applications Revisited 54
2.10.1 System Identification 54
2.10.2 Signal Enhancement 55
2.10.3 Signal Prediction 56
2.10.4 Channel Equalization 57
2.10.5 Digital Communication System 65
2.11 Concluding Remarks 67
2.12 References 68
2.13 Problems 70
xviii Contents

THE LEAST-MEAN-SQUARE (LMS) ALGORITHM 77

3.1 Introduction 77
3.2 The LMS Algorithm 77
3.3 Some Properties of the LMS Algorithm 79
3.3.1 Gradient Behavior - . 79
3.3.2 Convergence Behavior of the Coefficient Vector 80
3.3.3 Coefficient-Error-Vector Covariance Matrix 82
3.3.4 Behavior of the Error Signal 85
3.3.5 Minimum Mean-Square Error 85
3.3.6 Excess Mean-Square Error and Misadjustment 87
3.3.7 Transient Behavior 89
3.4 LMS Algorithm Behavior in Nonstationary Environments 90
3.5 Complex LMS Algorithm 94
3.6 Examples 95
3.6.1 Analytical Examples 95
3.6.2 System Identification Simulations 107
3.6.3 Channel Equalization Simulations 113
3.6.4 Fast Adaptation Simulations 114
3.6.5 The Linearly Constrained LMS Algorithm 118
3.7 Concluding Remarks 121
3.8 References 124
3.9 Problems 126

LMS-BASED ALGORITHMS 131

4.1 Introduction 131
4.2 Quantized-Error Algorithms 132
4.2.1 Sign-Error Algorithm 133
4.2.2 Dual-Sign Algorithm 140
4.2.3 Power-of-Two Error Algorithm 141
4.2.4 Sign-Data Algorithm 141
4.3 The LMS-Newton Algorithm 143
4.4 The Normalized LMS Algorithm 145
4.5 The Transform-Domain LMS Algorithm 147
4.6 The Affine Projection Algorithm 156,
4.6.1 Misadjustment in the Affine Projection Algorithm 161
4.6.2 Behavior in Nonstationary Environments 169
4.6.3 Transient Behavior 171
4.6.4 Complex Affine Projection Algorithm 173
Contents xix

4.7 Simulation Examples 174

4.7.1 Signal Enhancement Simulation 178
4.7.2 Signal Prediction Simulation 180
4.8 Concluding Remarks 183
4.9 References 186
4.10 Problems 189

CONVENTIONAL R L S ADAPTIVE FILTER 195

5.1 Introduction 195
5.2 The Recursive Least-Squares Algorithm 195
5.3 Properties of the Least-Squares Solution 200
5.3.1 Orthogonality Principle 200
5.3.2 Relation Between Least-Squares and Wiener Solutions 201
5.3.3 Influence of the Deterministic Autocorrelation Initialization 203
5.3.4 Steady-State Behavior of the Coefficient Vector 203
5.3.5 Coefficient-Error-Vector Covariance Matrix 205
5.3.6 Behavior of the Error Signal 207
5.3.7 Excess Mean-Square Error and Misadjustment 210
5.4 Behavior in Nonstationary Environments 215
5.5 Complex RLS Algorithm 219
5.6 Simulation Examples 221
5.7 Concluding Remarks ~ 223
5.8 References 227
5.9 Problems 227

DATA-SELECTIVE ADAPTIVE FILTERING 231

6.1 Introduction . 231
6.2 Set-Membership Filtering 232
6.3 Set-Membership Normalized LMS Algorithm 234
6.4 Set-Membership Affine Projection Algorithm 237
6.4.1 A Trivial Choice for Vector 7(fc) 241
6.4.2 A Simple Vector 7 (fc) 242
6.4.3 Reducing the Complexity in the Simplified SM-AP Algorithm 243
6.5 Set-Membership Binormalized LMS Algorithms 245
6.5.1 SM-BNLMS Algorithm 1 247
6.5.2 SM-BNLMS Algorithm 2 249
6.6 Computational Complexity 251
6.7 Time-Varying 7 252
6.8 Partial-Update Adaptive Filtering 254
6.8.1 Set-Membership Partial-Update NLMS Algorithm 256
xx Contents

6.9 Simulation Examples 260

6.9.1 Echo Cancellation Environment 264
6.9.2 Wireless Channel Environment 271
6.10 Concluding Remarks 280
6.11 References 281
6.12 Problems ' 283

7 ADAPTIVE LATTICE-BASED RLS ALGORITHMS 289

7.1 Introduction 289
7.2 Recursive Least-Squares Prediction - 290
7.2.1 Forward Prediction Problem 290
7.2.2 Backward Prediction Problem 293
7.3 Order-Updating Equations 295
7.3.1 A New Parameter 6(k, i) 295
7.3.2 Order Updating of £jfm.n (fc, i) and wb(fc, i) 297
7.3.3 Order Updating of ^ m i n (k, i) and w/ (k, i) 298
7.3.4 Order Updating of Prediction Errors 298
7.4 Time-Updating Equations 300
7.4.1 Time Updating for Prediction Coefficients 300
7.4.2 Time Updating for 5(k, i) 302
7.4.3 Order Updating for j(k, i) 304
7.5 Joint-Process Estimation 307
7.6 Time Recursions of the Least-Squares Error 311
7.7 Normalized Lattice RLS Algorithm 313
7.7.1 Basic Order Recursions 313
7.7.2 Feedforward Filtering 315
7.8 Error-Feedback Lattice RLS Algorithm 318
7.8.1 Recursive Formulas for the Reflection Coefficients 318
7.9 Lattice RLS Algorithm Based on A Priori Errors 319
7.10 Quantization Effects 321
7.11 Concluding Remarks 327
7.12 References 328
7.13 Problems 329

8 FAST TRANSVERSAL RLS ALGORITHMS 333

8.1 Introduction 333
8.2 Recursive Least-Squares Prediction 334
8.2.1 Forward Prediction Relations 334
8.2.2 Backward Prediction Relations 335
8.3 Joint-Process Estimation 337
Contents xxi

8.4 Stabilized Fast Transversal RLS Algorithm 339

8.5 Concluding Remarks 345
8.6 References 346
8.7 Problems 347

9 QR-DECOMPOSITION-BASED RLS FILTERS 351

9.1 Introduction 351
9.2 Triangularization Using QR-Decomposition 351
9.2.1 Initialization Process 353
9.2.2 Input Data Matrix Triangularization 353
9.2.3 QR-Decomposition RLS Algorithm 360
9.3 Systolic Array Implementation 365
9.4 Some Implementation Issues 372
9.5 Fast QR-RLS Algorithm 373
9.5.1 Backward Prediction Problem 376
9.5.2 Forward Prediction Problem 378
9.6 Conclusions and Further Reading 384
9.7 References 387
9.8 Problems 389

10 ADAPTIVE IIR FILTERS 395

10.1 Introduction 395
10.2 Output-Error IIR Filters 396
10.3 General Derivative Implementation 400
10.4 Adaptive Algorithms 402
10.4.1 Recursive Least-Squares Algorithm 402
10.4.2 The Gauss-Newton Algorithm 404
10.4.3 Gradient-Based Algorithm 407
10.5 Alternative Adaptive Filter Structures 407
10.5.1 Cascade Form 407
10.5.2 Lattice Structure 409
10.5.3 Parallel Form 416
10.5.4 Frequency-Domain Parallel Structure 417
10.6 Mean-Square Error Surface 426
10.7 Influence of the Filter Structure on the MSE Surface 433
10.8 Alternative Error Formulations 435
10.8.1 Equation Error Formulation 435
10.8.2 The Steiglitz-McBride Method 439
10.9 Conclusion 442
lO.lOReferences 443
10.11 Problems 446
xxii Contents

11 NONLINEAR ADAPTIVE FILTERING 451

11.1 Introduction 451
11.2 The Volterra Series Algorithm 452
11.2.1 LMS Volterra Filter 454
11.2.2 RLS Volterra Filter 457
11.3 Adaptive Bilinear Filters 464
11.4 Multilayer Perceptron Algorithm 469
11.5 Radial Basis Function Algorithm 473
11.6 Conclusion 480
11.7 References 482
11.8 Problems 484

12 SUBBAND ADAPTIVE FILTERS 485

12.1 Introduction 485
12.2 Multirate Systems 486
12.2.1 Decimation and Interpolation 486
12.3 Filter Banks 488
12.3.1 Two-Band Perfect Reconstruction Filter Banks 493
12.3.2 Analysis of Two-Band Filter Banks 494
12.3.3 Analysis of M-Band Filter Banks 494
12.3.4 Hierarchical M-Band Filter Banks 495
12.3.5 Cosine-Modulated Filter Banks 495
12.3.6 Block Representation 497
12.4 Subband Adaptive Filters 497
12.4.1 Subband Identification 501
12.4.2 Two-Band Identification 502
12.4.3 Closed-Loop Structure 502
12.5 Cross-Filters Elimination 508
12.5.1 Fractional Delays "* 510
12.6 Delayless Subband Adaptive Filtering 515
12.6.1 Computational Complexity 517
12.7 Frequency-Domain Adaptive Filtering 521
12.8 Conclusion 530
12.9 References 531
12.10Problems 533

13 BLIND ADAPTIVE FILTERING 537

13.1 Introduction 537
Contents xxiii

13.2 Constant-Modulus Algorithm 538

13.2.1 Godard Algorithm 539
13.2.2 Constant-Modulus Algorithm 540
13.2.3 Sato Algorithm 540
13.2.4 Error Surface of CMA 542
13.3 Affine Projection CM Algorithm 549
13.4 Blind SIMO Equalizers 555
13.4.1 Identification Conditions 557
13.5 SIMO-CMA Equalizer 558
13.6 Concluding Remarks 564
13.7 References 565
13.8 Problems 567

A COMPLEX DIFFERENTIATION 571

A.I Introduction 571
A.2 The Complex Wiener Solution 571
A.3 Derivation of the Complex LMS Algorithm 574
A.4 Useful Results 575

B QUANTIZATION EFFECTS IN THE LMS ALGORITHM 577

B.I Introduction 577
B.2 Error Description 577
B.3 Error Models for Fixed-Point Arithmetic 579
B.4 Coefficient- Error- Vector Covariance Matrix 580
B.5 Algorithm Stop 582
B.6 Mean-Square Error 582
B.7 Floating-Point Arithmetic Implementation 584
B.8 Floating-Point Quantization Errors in LMS Algorithm 585

C QUANTIZATION EFFECTS IN THE RLS ALGORITHM 589

C.I Introduction 589
C.2 Error Description 589
C.3 Error Models for Fixed-Point Arithmetic 591
C.4 Coefficient-Error-Vector Covariance Matrix 592
C.5 Algorithm Stop 595
C.6 Mean-Square Error 596
C.7 Fixed-Point Implementation Issues 597
C.8 Floating-Point Arithmetic Implementation 597
C.9 Floating-Point Quantization errors in RLS Algorithm 600
xxiv Contents

D KALMAN FILTERS 605

D.I Introduction 605
D.2 State-Space Model 605
D.2.1 Simple Example 606
D.3 Kalman Filtering 608
D.4 Kalman Filter and RLS 614
D.5 References 615

INDEX 617

View publication