Hebbian Learning and Negative Feedback Networks (Advanced Information and Knowledge Processing)

Contents......Page 3
Acronyms......Page 11
Preface......Page 12
Introduction......Page 14
Artificial Neural Networks......Page 15
The Organisation of this Book......Page 16
--- Single Stream Networks......Page 19
Hebbian Learning......Page 22
Quantification of Information......Page 24
Principal Component Analysis......Page 27
Weight Decay in Hebbian Learning......Page 29
ANNs and PCA......Page 32
Anti-Hebbian Learning......Page 34
Independent Component Analysis......Page 36
Conclusion......Page 40
Introduction......Page 41
Model......Page 51
Using Distance Differences......Page 56
Minor Components Analysis......Page 58
Conclusion......Page 64
Peer-Inhibitory Neurons......Page 67
Analysis of Differential Learning Rates......Page 69
Differential Activation Functions......Page 80
Emergent Properties of the Peer-Inhibition Network......Page 92
Conclusion......Page 93
Multiple Cause Data......Page 95
Non-negative Weights......Page 98
Factor Analysis......Page 102
Conclusion......Page 118
Exploratory Data Analysis......Page 120
Exploratory Projection Pursuit......Page 121
The Data and Sphering......Page 122
The Projection Pursuit Network......Page 123
Other Indices......Page 132
Using Exploratory Projection Pursuit......Page 136
Independent Component Analysis......Page 142
Conclusion......Page 145
Background......Page 146
The Classification Network......Page 149
The Scale Invariant Map......Page 152
The Subspace Map......Page 158
The Negative Feedback Coding Network......Page 167
Conclusion......Page 176
The Negative Feedback Network and Cost Functions......Page 178
-Insensitive Hebbian Learning......Page 180
The Maximum Likelihood EPP Algorithm......Page 185
A Combined Algorithm......Page 190
Conclusion......Page 195
--- Dual Stream Networks......Page 196
Statistical Canonical Correlation Analysis......Page 199
The First Canonical Correlation Network......Page 200
Experimental Results......Page 202
A Second Neural Implementation of CCA......Page 210
Simulations......Page 212
Linear Discriminant Analysis......Page 214
Discussion......Page 215
A Probabilistic Perspective......Page 217
Robust CCA......Page 219
A Model Derived from Becker’s Model 1......Page 220
Discussion......Page 223
Nonlinear Correlations......Page 225
The Search for Independence......Page 229
Kernel Canonical Correlation Analysis......Page 233
Relevance Vector Regression......Page 242
Appearance-Based Object Recognition......Page 245
Mixtures of Linear Correlations......Page 248
Exploratory Correlation Analysis......Page 255
Experiments......Page 259
Connection to CCA......Page 261
FastECA......Page 262
Local Filter Formation From Natural Stereo Images......Page 264
Twinned Maximum Likelihood Learning......Page 274
Unmixing of Sound Signals......Page 278
Conclusion......Page 279
Multicollinearity & Partial Least Squares......Page 282
Application to CCA......Page 283
Extracting Multiple Canonical Correlations......Page 287
Experiments on Multicollinear Data......Page 288
A Neural Implementation of Partial Least Squares......Page 291
Conclusion......Page 295
Twinned Principal Curves......Page 297
Properties of Twinned Principal Curves......Page 299
Twinned Self-Organising Maps......Page 311
Discussion......Page 313
Review......Page 314
Current and Future Work......Page 317
A.1 The Interneuron Model......Page 319
A.2 Other Models......Page 321
A.3 Related Biological Models......Page 324
B.1 F¨oldi´ak’s Sixth Model......Page 326
B.2 Competitive Hebbian Learning......Page 329
B.3 Multiple Cause Models......Page 330
B.4 Predictability Minimisation......Page 333
B.5 Mixtures of Experts......Page 335
B.6 Probabilistic Models......Page 337
C.1 Jutten and Herault......Page 344
C.2 Nonlinear PCA......Page 347
C.3 Information Maximisation......Page 348
C.4 Penalised Minimum Reconstruction Error......Page 353
C.5 FastICA......Page 354
D.1 The I-Max Model......Page 356
D.2 Stone’s Model......Page 358
D.3 Kay’s Neural Models......Page 359
D.4 Borga’s Algorithm......Page 361
E.1 Artificial Data Sets......Page 366
E.2 Real Data Sets......Page 369
Refs......Page 373
Index......Page 382