Deep Learning in Time Series Analysis

Cover
Title Page
Copyright Page
Dedication
Foreword
Preface
Table of Contents
Contributors
Part I Fundamentals of Learning
	1. Introduction to Learning
		1.1 Artificial Intelligence
		1.2 Data and Signal Definition
		1.3 Data Versus Signal
		1.4 Signal Models
		1.5 Noise and Interference
		1.6 Time Series Definition
		1.7 Time Series Analysis
		1.8 Deep Learning and Time Series Analysis
		1.9 Organisation of the Book
	2. Learning Theory
		2.1 Learning and Adaptation
		2.2 Learning in a Practical Example
		2.3 Mathematical View to Learning
			2.3.1 Training and Validation Data
			2.3.2 Training Method
			2.3.3 Training Parameters
			2.3.4 Hyperparameters
		2.4 Learning Phases
		2.5 Training, Validation, and Test
		2.6 Learning Schemes
			2.6.1 Supervised-Static Learning
			2.6.2 Supervised-Dynamic Learning
			2.6.3 Unsupervised-Static Learning
			2.6.4 Unsupervised-Dynamic Learning
		2.7 Training Criteria
		2.8 Optimization, Training, and Learning
		2.9 Evaluation of Learning Performance
			2.9.1 Structural Risk
			2.9.2 Empirical Risk
			2.9.3 Overfitting and Underfitting Risk
			2.9.4 Learning Capacity
		2.10 Validation
			2.10.1 Repeated Random Sub Sampling (RRSS)
			2.10.2 K-Fold Validation
			2.10.3 A-Test Validation
		2.11 Privileges of A-Test Method
			2.11.1 A-Test and Structural Risk
			2.11.2 A-Test and Leaning Capacity
			2.11.3 A-Test vs other Methods
		2.12 Large and Small Training Data
	3. Pre-processing and Visualisation
		3.1 Dimension Reduction
			3.1.1 Feature Selection
				3.1.1.1 Hill-Climbing Algorithm
				3.1.1.2 Linear Discriminant Analysis (LDA)
				3.1.1.3 Fisher Method
			3.1.2 Linear Transformation
				3.1.2.1 Principal Component Analysis (PCA)
				3.1.2.2 PCA-Fisher Method
		3.2 Supervised Mapping
			3.2.1 K-Nearest Neighbours (KNN)
			3.2.2 Perceptron Neural Network
			3.2.3 Multi-layer Perceptron Neural Networks (MLP)
		3.3 Unsupervised Mapping
			3.3.1 K-Means Clustering
			3.3.2 Self-Organizing Map (SOM)
			3.3.3 Hierarchical Clustering
Part II Essentials of Time Series Analysis
	4. Basics of Time Series
		4.1 Introduction to Time Series Analysis
		4.2 Deterministic, Chaotic and Stochastic
		4.3 Stochastic Behaviors of Time Series
			4.3.1 Cyclic Time Series
				4.3.1.1 Sector Definition
				4.3.1.2 Uniform Sectors
				4.3.1.3 Growing-Time Sectors
			4.3.2 Partially Cyclic Time Series
		4.4 Time Series Prediction
		4.5 Time Series Classification
	5. Multi-Layer Perceptron (MLP) Neural Networks for Time Series Classification
		5.1 Time-Delayed Neural Network (TDNN)
		5.2 Time-Growing Neural Network (TGNN)
		5.3 Forward, Backward and Bilateral Time-Growing Window
		5.4 Privileges of Time-Growing Neural Network
			5.4.1 TGNN includes MLP in its architecture
			5.4.2 TGNN can include TDNN in its structure
			5.4.3 TGNN is optimal in learning the first window
	6. Dynamic Models for Sequential Data Analysis
		6.1 Dynamic Time Warping (Structural Classification)
		6.2 Hidden Markov Model (Statistical Classification)
			6.2.1 Model-based analysis
			6.2.2 Essentials of Hidden Markov Model (HMM)
			6.2.3 Problem statement and implementation
			6.2.4 Time series analysis and HMM
		6.3 Recurrent Neural Network
Part III Deep Learning Approaches to Time Series Classification
	7. Clustering for Learning at Deep Level
		7.1 Clustering as a Tool for Deep Learning
		7.2 Modified K-Means Method
		7.3 Modified Fuzzy C-Means
		7.4 Discriminant Analysis
		7.5 Cluster-Based vs Discriminant Analysis Methods
		7.6 Combined Methods
	8. Deep Time Growing Neural Network
		8.1 Basic Architecture
		8.2 Learning at the Deep Level
			8.2.1 Learning the growing centre
			8.2.2 Learning the deep elements
		8.3 Surface Learning
	9. Deep Learning of Cyclic Time Series
		9.1 Time Growing Neural Network
		9.2 Growing-Time Support Vector Machine
		9.3 Distance-Based Learning
		9.4 Optimization
	10. Hybrid Method for Cyclic Time Series
		10.1 Learning Deep Contents
		10.2 Cyclic Learning
		10.3 Classification
	11. Recurrent Neural Networks (RNN)
		11.1 Introduction
		11.2 Structure of Recurrent Neural Networks
		11.3 Unfolding the Network in Time
		11.4 Backpropagation Through Time
		11.5 The Challenge of Long-term Dependencies
		11.6 Long-Short Term Memory (LSTM)
		11.7 Other Recurrent Networks
			11.7.1 Unfolding outputs at all steps
			11.7.2 Gated recurrent networks
			11.7.3 Echo state networks
	12. Convolutional Neural Networks (CNN)
		12.1 Introduction
		12.2 Architecture Overview
		12.3 Convolutional Layer
		12.4 Pooling Layer
		12.5 Learning of CNN
		12.6 Recurrent CNN
Bibliography
Index