Kernel adaptive filtering: A comprehensive introduction

KERNEL ADAPTIVE FILTERING......Page 5
CONTENTS......Page 9
PREFACE......Page 13
ACKNOWLEDGMENTS......Page 17
NOTATION......Page 19
ABBREVIATIONS AND SYMBOLS......Page 21
1.1 Supervised, Sequential, and Active Learning......Page 25
1.2 Linear Adaptive Filters......Page 27
1.3 Nonlinear Adaptive Filters......Page 34
1.4 Reproducing Kernel Hilbert Spaces......Page 36
1.5 Kernel Adaptive Filters......Page 40
1.6 Summarizing Remarks......Page 44
Endnotes......Page 45
2 KERNEL LEAST-MEAN-SQUARE ALGORITHM......Page 51
2.1 Least-Mean-Square Algorithm......Page 52
2.2 Kernel Least-Mean-Square Algorithm......Page 55
2.3 Kernel and Parameter Selection......Page 58
2.4 Step-Size Parameter......Page 61
2.5 Novelty Criterion......Page 62
2.6 Self-Regularization Property of KLMS......Page 64
2.8 Normalized Kernel Least-Mean-Square Algorithm......Page 72
2.9 Kernel ADALINE......Page 73
2.10 Resource Allocating Networks......Page 77
2.11 Computer Experiments......Page 79
2.12 Conclusion......Page 87
Endnotes......Page 89
3 KERNEL AFFINE PROJECTION ALGORITHMS......Page 93
3.1 Affine Projection Algorithms......Page 94
3.2 Kernel Affine Projection Algorithms......Page 96
3.3 Error Reusing......Page 101
3.5 Taxonomy for Related Algorithms......Page 102
3.6 Computer Experiments......Page 104
3.7 Conclusion......Page 113
Endnotes......Page 115
4.1 Recursive Least-Squares Algorithm......Page 118
4.2 Exponentially Weighted Recursive Least-Squares Algorithm......Page 121
4.3 Kernel Recursive Least-Squares Algorithm......Page 122
4.4 Approximate Linear Dependency......Page 126
4.5 Exponentially Weighted Kernel Recursive Least-Squares Algorithm......Page 127
4.6 Gaussian Processes for Linear Regression......Page 129
4.7 Gaussian Processes for Nonlinear Regression......Page 132
4.8 Bayesian Model Selection......Page 135
4.9 Computer Experiments......Page 138
4.10 Conclusion......Page 143
Endnotes......Page 144
5 EXTENDED KERNEL RECURSIVE LEAST-SQUARES ALGORITHM......Page 148
5.1 Extended Recursive Least Squares Algorithm......Page 149
5.2 Exponentially Weighted Extended Recursive Least Squares Algorithm......Page 152
5.3 Extended Kernel Recursive Least Squares Algorithm......Page 153
5.4 EX-KRLS for Tracking Models......Page 155
5.5 EX-KRLS with Finite Rank Assumption......Page 161
5.6 Computer Experiments......Page 165
5.7 Conclusion......Page 174
Endnotes......Page 175
6.1 Definition of Surprise......Page 176
6.2 A Review of Gaussian Process Regression......Page 178
6.3 Computing Surprise......Page 180
6.4 Kernel Recursive Least Squares with Surprise Criterion......Page 183
6.5 Kernel Least Mean Square with Surprise Criterion......Page 184
6.6 Kernel Affine Projection Algorithms with Surprise Criterion......Page 185
6.7 Computer Experiments......Page 186
6.8 Conclusion......Page 197
Endnotes......Page 198
EPILOGUE......Page 199
A.1 Singular Value Decomposition......Page 201
A.3 Eigenvalue Decomposition......Page 203
A.5 Block Matrix Inverse......Page 205
A.7 Joint, Marginal, and Conditional Probability......Page 206
A.8 Normal Distribution......Page 207
A.10 Newton’s Method......Page 208
B APPROXIMATE LINEAR DEPENDENCY AND SYSTEM STABILITY......Page 210
REFERENCES......Page 217
INDEX......Page 228