Complex Valued Nonlinear Adaptive Filters: Noncircularity, Widely Linear and Neural Models

Contents......Page 3
Preface......Page 9
Magic of Complex Numbers......Page 12
History of Complex Numbers......Page 13
History of Mathematical Notation......Page 19
Development of Complex Valued Adaptive Signal Processing......Page 20
Some Examples of Complex Valued Signal Processing......Page 23
is Not Only Convenient But Also Natural......Page 29
Why Complex Modelling of Real Valued Processes?......Page 30
Exploiting the Phase Information......Page 33
Other Applications of Complex Domain Processing of Real Valued Signals......Page 36
Additional Benefits of Complex Domain Processing......Page 39
Adaptive Filtering Architectures......Page 42
Linear and Nonlinear Stochastic Models......Page 43
Linear and Nonlinear Adaptive Filtering Architectures......Page 44
State Space Representation and Canonical Forms......Page 48
Properties of Complex Functions......Page 51
Universal Function Approximation......Page 54
Nonlinear Activation Functions for Complex Neural Networks......Page 56
Generalised Splitting Activation Functions (GSAF)......Page 61
Summary: Choice of the Complex Activation Function......Page 62
Elements of CR Calculus......Page 63
The Cauchy–Riemann Equations......Page 64
Generalised Derivatives of Functions of Complex Variable......Page 65
-derivatives of Cost Functions......Page 70
Complex Valued Adaptive Filters......Page 76
Adaptive Filtering Configurations......Page 77
The Complex Least Mean Square Algorithm......Page 80
Nonlinear Feedforward Complex Adaptive Filters......Page 87
Normalisation of Learning Algorithms......Page 92
Performance of Feedforward Nonlinear Adaptive Filters......Page 94
Summary: Choice of a Nonlinear Adaptive Filter......Page 96
Adaptive Filters with Feedback......Page 98
Training of IIR Adaptive Filters......Page 99
Nonlinear Adaptive IIR Filters: Recurrent Perceptron......Page 104
Training of Recurrent Neural Networks......Page 106
Simulation Examples......Page 109
Filters with Adaptive Stepsize......Page 113
Benveniste Type Variable Stepsize Algorithms......Page 114
Complex Valued GNGD Algorithms......Page 116
Simulation Examples......Page 119
Dynamical Range Reduction......Page 124
FIR Adaptive Filters with an Adaptive Nonlinearity......Page 126
Recurrent Neural Networks with Trainable Amplitude of Activation Functions......Page 127
Simulation Results......Page 129
The Data-reusing Complex Valued Least Mean Square (DRCLMS) Algorithm......Page 133
Data-reusing Complex Nonlinear Adaptive Filters......Page 135
Data-reusing Algorithms for Complex RNNs......Page 138
Matrix Representation of a Complex Number......Page 141
Möbius Transformation......Page 144
Activation Functions & Möbius Transformations......Page 146
All-Pass Systems as Möbius Transformations......Page 150
Fractional Delay Filters......Page 151
Augmented Complex Statistics......Page 154
Complex Random Variables (CRV)......Page 155
Complex Circular Random Variables......Page 161
Complex Signals......Page 162
Second-order Characterisation of Complex Signals......Page 164
Minimum Mean Square Error (MMSE) Estimation in......Page 172
Complex White Noise......Page 175
Autoregressive Modelling in......Page 176
The Augmented Complex LMS (ACLMS) Algorithm......Page 178
Adaptive Prediction Based on ACLMS......Page 181
Duality btw Complex & Real Valued Filters......Page 185
A Dual Channel Real Valued Adaptive Filter......Page 186
Duality Between Real and Complex Valued Filters......Page 188
Simulations......Page 190
Widely Linear Filters with Feedback......Page 193
Widely Linear Adaptive Filters with Feedback......Page 194
The Augmented Complex Valued RTRL (ACRTRL) Algorithm......Page 199
The Augmented Kalman Filter Algorithm for RNNs......Page 200
Augmented Complex Unscented Kalman Filter (ACUKF)......Page 202
Simulation Examples......Page 205
Parametric Signal Modality Characterisation......Page 208
Standard Hybrid Filtering in......Page 210
Tracking the Linear/Nonlinear Nature of Complex Valued Signals......Page 211
Split vs Fully Complex Signal Natures......Page 215
Online Assessment of the Nature of Wind Signal......Page 217
Collaborative Filters for General Complex Signals......Page 218
Adaptive Filtering based on EMD......Page 222
The Empirical Mode Decomposition Algorithm......Page 223
Complex Extensions of Empirical Mode Decomposition......Page 227
Applications of Complex Extensions of EMD......Page 231
Validation of Complex Representations......Page 234
Signal Modality Characterisation in......Page 235
Testing for the Validity of Complex Representation......Page 240
Quantifying Benefits of Complex Valued Representation......Page 244
Distinctive Properties of Calculus in C......Page 246
Liouville Theorem......Page 252
C.1 Definitions of Algebraic Notions of Group, Ring and Field......Page 253
C.3 Higher Dimension Algebras......Page 254
C.4 The Algebra of Quaternions......Page 255
C.5 Clifford Algebras......Page 256
D.1 Logistic Sigmoid Activation Function......Page 257
D.2 Hyperbolic Tangent Activation Function......Page 258
Elementary Transcendental Functions (ETF)......Page 259
F. 2 Standard Vector and Matrix Differentiation......Page 263
Notions from Learning Theory......Page 265
G.2 The Bias–Variance Dilemma......Page 266
G.4 Transformation of Input Data......Page 267
Notions from Approximation Theory......Page 269
Terminology used in Neural Networks......Page 272
J.1 The Complex RTRL Algorithm (CRTRL) for CPRNN......Page 274
Gradient Adaptive Step Size (GASS) Algorithms in......Page 278
K.1 Gradient Adaptive Stepsize Algorithms Based on......Page 279
K.2 Variable Stepsize Algorithms Based on......Page 280
)......Page 282
)......Page 283
)......Page 285
A Posteriori Learning......Page 286
Strategies in Adaptive Learning......Page 287
Notions from Stability Theory......Page 289
O.1 Vector and Matrix Norms......Page 291
O.2 Relaxation in Linear Systems......Page 292
Contraction Mappings, Fixed Point Iteration & Fractals......Page 297
P. 1 Historical Perspective......Page 301
P. 2 More on Convergence: Modified Contraction Mapping......Page 303
P. 3 Fractals and Mandelbrot Set......Page 306
Refs......Page 307
Index......Page 319