Process Neural Networks: Theory and Applications

Cover......Page 1
ADVANCED TOPICS\\rIN SCIENCE AND TECHNOLOGY IN CHINA......Page 2
Title page......Page 4
Copyright Page......Page 5
Preface......Page 6
Table of Contents......Page 8
1.1 Development of Artificial Intelligence......Page 14
1.2 Characteristics of Artificial Intelligent System......Page 18
1.3.1 Fuzzy Computing......Page 22
1.3.3 Evolutionary Computing......Page 25
1.3.4 Combination of the Three Branches......Page 28
1.4 Process Neural Networks......Page 29
References......Page 30
2 Artificial Neural Networks......Page 33
2.1 Biological Neuron......Page 34
2.2 Mathematical Model of a Neuron......Page 35
2.3.1 Feedforward/Feedback Neural Network Model......Page 36
2.3.2 Function Approximation Capability of Feedforward Neural Networks......Page 38
2.3.3 Computing Capability of Feedforward Neural Networks......Page 40
2.3.5 Generalization Problem for Feedforward Neural Networks......Page 41
2.3.6 Applications of Feedforward Neural Networks......Page 43
2.4.1 Fuzzy Neurons......Page 45
2.4.2 Fuzzy Neural Networks......Page 46
2.5.2 Maximum (or Minimum) Aggregation Artificial Neural Networks......Page 48
2.5.3 Other Nonlinear Aggregation Artificial Neural Networks......Page 49
2.6 Spatio-temporal Aggregation and Process Neural Networks......Page 50
2.7 Classification of Artificial Neural Networks......Page 52
References......Page 53
3.1 Revelation of Biological Neurons......Page 56
3.2 Definition of Process Neurons......Page 57
3.3 Process Neurons and Functionals......Page 60
3.4 Fuzzy Process Neurons......Page 61
3.4.1 Process Neuron Fuzziness......Page 62
3.4.2 Fuzzy Process Neurons Constructed using Fuzzy Weighted Reasoning Rule......Page 63
3.5 Process Neurons and Compound Functions......Page 64
References......Page 65
4.1 Simple Model of a Feedforward Process Neural Network......Page 66
4.2 A General Model of a Feedforward Process Neural Network......Page 68
4.3 A Process Neural Network Model Based on Weight Function Basis Expansion......Page 69
4.4 Basic Theorems of Feedforward Process Neural Networks......Page 71
4.4.1 Existence of Solutions......Page 72
4.4.2 Continuity......Page 75
4.4.3 Functional Approximation Property......Page 77
4.5 Structural Formula Feedforward Process Neural Networks......Page 80
4.5.1 Structural Formula Process Neurons......Page 81
4.5.2 Structural Formula Process Neural Network Model......Page 82
4.6.1 Network Structure......Page 84
4.6.2 Continuity and Approximation Capability of the Model......Page 86
4.7 Continuous Process Neural Networks......Page 88
4.7.1 Continuous Process Neurons......Page 89
4.7.2 Continuous Process Neural NetworkModel......Page 90
4.7.3 Continuity, Approximation Capability, and Computing Capability of the Model......Page 91
4.8 Functional Neural Network......Page 96
4.8.1 Functional Neuron......Page 97
4.8.2 Feedforward Functional Neural Network Model......Page 98
4.9 Epilogue......Page 99
References......Page 100
5 Learning Algorithms for Process Neural Networks......Page 101
5.1.1 A General Learning Algorithm Based on Gradient Descent......Page 102
5.1.2 Learning Algorithm Based on Gradient-Newton Combination......Page 104
5.2 Learning Algorithm Based on Orthogonal Basis Expansion......Page 106
5.2.1 Orthogonal Basis Expansion of Input Functions......Page 107
5.2.2 Learning Algorithm Derivation......Page 108
5.2.3 Algorithm Description and Complexity Analysis......Page 109
5.3.1 FourierOrthogonal Basis Expansion of the Function in L2[0, 2rr]......Page 110
5.3.2 Learning Algorithm Derivation......Page 112
5.4.1 Learning Algorithm Based on Discrete Walsh Function Transformation......Page 114
5.4.2 Learning Algorithm Based on Continuous Walsh Function Transformation......Page 118
5.5.1 Spline Function......Page 121
5.5.2 Learning Algorithm Derivation......Page 122
5.5.3 Analysis of the Adaptability and Complexity of a Learning Algorithm......Page 124
5.6.1 Learning Algorithm Based on Rational Square Approximation......Page 125
5.6.2 Learning Algorithm Based on Optimal Piecewise Approximation......Page 132
References......Page 139
6 Feedback Process Neural Networks......Page 141
6.1.1 Network Structure......Page 142
6.1.2 Learning Algorithm......Page 143
6.1.3 Stability Analysis......Page 145
6.2.1 Feedback Process Neural Network with Time-varying Functions as Inputs and Outputs......Page 148
6.2.2 Feedback Process Neural Network for Pattern Classification......Page 149
6.2.3 Feedback Process Neural Network for Associative Memory Storage......Page 150
6.3 Application Examples......Page 151
References......Page 155
7.1 Multi-aggregation Process Neuron......Page 156
7.2.1 A General Model of Multi-aggregation Process Neural Network......Page 158
7.2.2 Multi-aggregation Process Neural Network Model with Multivariate Process Functions as Inputs and Outputs......Page 160
7.3.1 Learning Algorithm of General Models of Multi-aggregation Process Neural Networks......Page 161
7.3.2 Learning Algorithm of Multi-aggregation Process Neural Networks with Multivariate Functions as Inputs and Outputs......Page 165
7.4 Application Examples......Page 168
7.5 Epilogue......Page 172
References......Page 173
8.1 Process Neural Networks with Double Hidden Layers......Page 174
8.1.1 Network Structure......Page 175
8.1.2 Learning Algorithm......Page 176
8.1.3 Application Examples......Page 178
8.2 Discrete Process Neural Network......Page 179
8.2.1 Discrete Process Neuron......Page 180
8.2.2 Discrete Process Neural Network......Page 181
8.2.3 Learning Algorithm......Page 182
8.2.4 Application Examples......Page 183
8.3 Cascade Process Neural Network......Page 185
8.3.1 Network Structure......Page 186
8.3.2 Learning Algorithm......Page 188
8.3.3 Application Examples......Page 189
8.4.1 NetworkStructure......Page 191
8.4.2 Learning Algorithm......Page 192
8.4.3 Application Examples......Page 195
8.5 Counter Propagation Process Neural Network......Page 197
8.5.2 Learning Algorithm......Page 198
8.5.3 Determination of the Number of Pattern Classifications......Page 199
8.5.4 Application Examples......Page 200
8.6.1 Radial-Basis Process Neuron......Page 201
8.6.2 Network Structure......Page 202
8.6.3 Learning Algorithm......Page 203
8.6.4 Application Examples......Page 205
References......Page 206
9.1 Application in Process Modeling......Page 208
9.2 Application in Nonlinear System Identification......Page 211
9.2.1 Principle of Nonlinear System Identification......Page 212
9.2.2 Process Neural Network for System Identification......Page 213
9.2.3 Nonlinear System Identification Process......Page 214
9.3 Application in Process Control......Page 216
9.3.2 Designing and Solving of the Process Controller......Page 217
9.3.3 Simulation Experiment......Page 221
9.4 Application in Clustering and Classification......Page 223
9.5 Application in Process Optimization......Page 228
9.6 Applications in Forecast and Prediction......Page 229
9.7 Application in Evaluation and Decision......Page 237
9.8 Application in Macro Control......Page 239
9.9 Other Applications......Page 240
References......Page 244
Postscript......Page 246
Index......Page 251