Efficient Online Learning Algorithms for Total Least Square Problems (Engineering Applications of Computational Methods, 21)

Preface
	To Whom This Book Is Addressed
	Prerequisites
Acknowledgments
Contents
Notations
	Abbreviations
	Mathematical Notations
List of Figures
List of Tables
1 Introduction
	1.1 An Overview of Total Least Square
		1.1.1 Total Least Square Problems
		1.1.2 Total Least Square Methods
		1.1.3 Related Monographs for Total Least Square
	1.2 Aim and Main Features of This Book
		1.2.1 Aim of This Book
		1.2.2 Main Features of This Book
	1.3 Organization of This Book
	References
2 Total Least Square Problems
	2.1 Preliminaries
		2.1.1 Overdetermined Linear Equations and SVD of Matrix
		2.1.2 Ordinary Least Squares (OLS) Problems
	2.2 Classical TLS Problem
		2.2.1 Basic TLS Problems
		2.2.2 OLS and TLS Geometric Considerations
		2.2.3 Multidimensional TLS Problem
		2.2.4 Non-generic Unidimensional TLS Problem
	2.3 Extension of TLS Problem
		2.3.1 Mixed OLS-TLS Problem
		2.3.2 Statistical Properties and Validity
		2.3.3 Basic Data Least Squares Problem
		2.3.4 Generalized TLS Problem
		2.3.5 Weighted TLS Problem
		2.3.6 Constrained TLS Problem
		2.3.7 Structured TLS Problem
		2.3.8 TLS in Nonlinear ElV Models
		2.3.9 TLS Under Non-gaussian Noises
	2.4 Summary
	References
3 Total Least Square Methods
	3.1 TLS Solution
	3.2 Partial TLS Algorithm
	3.3 Iterative Computation Methods
		3.3.1 Direct Versus Iterative Computation Methods
		3.3.2 Inverse Iteration
		3.3.3 Chebyshev Iteration
		3.3.4 Lanczos Methods
		3.3.5 Rayleigh Quotient Iteration
	3.4 Rayleigh Quotient Minimization Non-neural Methods
		3.4.1 Davila’s Recursive TLS Algorithm
		3.4.2 Feng’s Fast Recursive TLS Algorithm
		3.4.3 Feng’s Fast Approximate Inverse Power Iterative TLS Algorithm
	3.5 Neural Networks Methods for TLS
		3.5.1 Neural Networks Methods for MCA
		3.5.2 Neural Networks Methods for SVD
		3.5.3 Neural Networks that Iterate Only in the TLS Hyperplane
	3.6 Other Methods of Extended TLS Problems
		3.6.1 Weighted TLS Algorithms
		3.6.2 Constrained TLS Algorithm
		3.6.3 Structured TLS Solution
		3.6.4 TLS in Nonlinear ElV Models
		3.6.5 TLS Under Non-Gaussian Noises
	3.7 Conclusions
	References
4 Fast Recursive TLS Algorithms
	4.1 Introduction
	4.2 Review of Davila’s RTLS Algorithm for FIR Adaptive Filtering
		4.2.1 RLS Filter Bias and Mean Squared Error for IR Estimation
		4.2.2 TLS for Adaptive FIR Filters
		4.2.3 Recursive TLS Algorithm
	4.3 A Fast Recursive TLS Algorithm for Adaptive FIR Filtering
		4.3.1 TLS Problems in Signal Processing
		4.3.2 Landscape of Criterion
		4.3.3 New RTLS Algorithm
		4.3.4 Convergence Analysis
		4.3.5 Simulations and Conclusions
	4.4 A Fast Recursive TLS Algorithm for Adaptive IIR Filtering
		4.4.1 TLS Problems in Adaptive IIR Filtering and N-RTLS Algorithm
		4.4.2 Algorithm Convergence
		4.4.3 Simulations and Conclusions
	4.5 Summary
	References
5 Approximate Inverse Power Iteration TLS Algorithm
	5.1 Introduction
	5.2 Review of Inverse Power Iteration
	5.3 AIP Iteration Algorithm for Adaptive TLS FIR Filtering
		5.3.1 Preliminaries
		5.3.2 AIP Iteration Algorithm
		5.3.3 Algorithm Convergence
		5.3.4 Simulations Examples and Conclusion
	5.4 An AIP Algorithm for Adaptive Extraction of Minor Subspace
		5.4.1 Proposed Adaptive Tracking Algorithm of MS
		5.4.2 Theoretical Analysis of Algorithm
	5.5 Summary
	References
6 Neural-Based MCA Algorithms for Adaptive TLS
	6.1 Introduction
	6.2 Review of Neural-Network-Based MCA Algorithms
		6.2.1 Oja’s MCA Algorithms
		6.2.2 Self-stabilizing MCA Algorithms
		6.2.3 Orthogonal Oja’s Algorithms
		6.2.4 Coupled MCA Algorithms
	6.3 Novel Parallel Multiple MCs Extraction Algorithm by Diagonal Matrix
		6.3.1 Preliminary and Novel Algorithm
		6.3.2 Fixed Points Analysis
		6.3.3 Stability on the Manifold
		6.3.4 Simulation Experiments and Conclusion
	6.4 A Weighted Information Criterion and Multiple MC Algorithm
		6.4.1 Preliminaries
		6.4.2 A Weighted Information Criterion and Its Landscape
		6.4.3 Adaptive Multiple MCs Extraction Algorithms
		6.4.4 Convergence Analysis
		6.4.5 Simulations Experiments and Conclusions
	6.5 A Coupled Minor Component Analysis Algorithm
		6.5.1 Coupled Dynamical System
		6.5.2 Coupled MCA Learning Algorithms
		6.5.3 Analysis of Convergence and Self-stabilizing Property
		6.5.4 Simulation and Conclusions
	6.6 Summary
	References
7 Neural-Based SVD Algorithms for Adaptive TLS
	7.1 Introduction
	7.2 Review of Neural-Based SVD Algorithms
		7.2.1 Parallel SVD Learning Algorithms on Double Stiefel Manifold
		7.2.2 Cross-Associative Neural Network for SVD (CANN)
		7.2.3 Coupled SVD of a Cross-Covariance Matrix
	7.3 A Neural Network for SVD of Non-squared Data Matrix
		7.3.1 A Novel Recurrent Neural Network
		7.3.2 Stability Analysis of Algorithm
		7.3.3 Simulation Experiments and Conclusions
	7.4 A Fast and Effective Neural Network Algorithm to Perform SVD
		7.4.1 Preliminary
		7.4.2 Novel Information Criterion and Algorithm
		7.4.3 Convergence Analysis
		7.4.4 Experiments and Conclusions
	7.5 Coupled Neural Network Algorithm for PST Extraction
		7.5.1 Novel Information Criterion and Its Coupled System
		7.5.2 Experiments and Conclusions
	7.6 Summary
	References
8 Neural-Based TLS Algorithms
	8.1 Introduction
	8.2 Review of Neural-Based TLS Algorithms
		8.2.1 The Hopfield-Like Neural Network of Luo, Li, and He
		8.2.2 The Linear Neuron of Gao, Ahmad, and Swamy
		8.2.3 The Linear Neurons of Cichocki and Unbehauen
		8.2.4 The TLS EXIN Neural Network and Its Modified Version
		8.2.5 The LS-TLS Neural Algorithm of Bruce and Williamson
	8.3 A Self-stabilizing Neural Algorithm for TLS Filtering
		8.3.1 The TLS Linear Neuron with a Self-stabilizing Algorithm
		8.3.2 Self-stabilizing and Stability
		8.3.3 Simulation of Algorithm
	8.4 Conclusion
	References
9 TLS Algorithm Under Non-Gaussian Noises
	9.1 Introduction
	9.2 Review of the LS/TLS Method Under Non-Gaussian Noises
		9.2.1 The LS Method Under Non-Gaussian Noise
		9.2.2 TLS Algorithm with Input and Output of the Same Noise Intensity
	9.3 Mixed LS-TLS Algorithm Under Non-Gaussian Noises
		9.3.1 Partial Input and Output Contaminated with Non-Gaussian Noises
		9.3.2 Disparate Input and Output Non-Gaussian Noises
		9.3.3 Input Non-Gaussian Noises Only
	9.4 Simulation Experiments and Analysis
		9.4.1 Experiment of NG-TLS Algorithm
		9.4.2 Experiment of NG-LS-TLS Algorithm
	9.5 Summary
	References
10 Performance Analysis Methods of TLS Algorithms
	10.1 Introduction
	10.2 Review of the Analysis Methods of MCA/TLS Algorithms
		10.2.1 Deterministic Continuous-Time System Method
		10.2.2 Stochastic Discrete-Time System Method
		10.2.3 Lyapunov Function Approach
		10.2.4 Fixed Pointed Method
		10.2.5 Deterministic Discrete-Time System Method
	10.3 Convergence of Novel Generalized MCA Algorithm via DDT
		10.3.1 Algorithm Presentation
		10.3.2 Self-stabilizing Analysis
		10.3.3 Dynamic Characteristic of the GChen Algorithm
		10.3.4 Numerical Simulations and Conclusions
	10.4 Summary
	References