Physical Models of Neural Networks

PREFACE v
1 ASSOCIATIVE MEMORY AND ATTRACTORS 1
2 THE NEURON AND ITS MODEL 4
2.1 A quick look at the neuron 4
2.2 Modelling the neuron 8
2.3 Feedback dynamical network 10
2.4 Connectivism 13
3 HEBBIAN LEARNING 15
4 THE HOPFIELD MODEL 19
4.1 Definition of the model and basic facts about it 19
4.2 Noise analysis 24
4.3 Mean-field theory of the Hopfield net: derivation
of the Amit-Gutfreund-Sompolinsky equations . . 30
4.4 Mean-field theory: solution of the equations 36
5 DESCENDANTS OF THE HOPFIELD MODEL 40
5.1 The Hopfield model is robust against 40
5.2 The projection rule 43
5.3 Low activity and biased networks 45
5.4 Hierarchical storage 50
5.5 Time sequences 54
5.6 Invariant pattern recognition 58
5.7 Learning within bounds and short-term memory . . . . 60
6 PATIENT LEARNING 64
6.1 Setting the aim 64
6.2 Iterative learning algorithms 66
6.3 The Perceptron Convergence Theorem 70
6.4 The Gardner capacity of a network 72
7 DYNAMICS OF RETRIEVAL 76
7.1 The asymmetrically diluted model 78
7.2 General asymmetric models 83
8 FEED-FORWARD NETWORKS 87
8.1 The simple perceptron 87
8.2 Layered feed-forward networks 91
9 THE BOLTZMANN MACHINE 98
10 SELF-ORGANIZED FEATURE MAPS:
THE KOHONEN MODEL 101
10.1 Preliminaries 101
10.2 Topological ordering: local and global 103
10.3 The Kohonen algorithm 105
10.4 Question marks on theory Ill
11 OUTLOOK 113
APPENDIX A:
THE REPLICA TRICK WAY TO MEAN FIELD 115
APPENDIX B:
DYNAMICS OF A TWO-PATTERN NETWORK 125
REFERENCES 133
SUBJECT INDEX 141