Foundations of the Theory of Learning Systems

Cover......Page 1
Series......Page 2
Title page......Page 3
Copyright page......Page 4
Dedication......Page 5
Contents......Page 7
Preface to the Russian Edition......Page 11
Acknowledgments......Page 13
1.1 Introduction......Page 15
1.2 Concept of the Goal of Learning......Page 16
1.3 Complex Goals of Learning......Page 17
1.5 Types of Learning......Page 19
1.6 Discussion......Page 20
Comments......Page 21
References......Page 22
2.2 Algorithmic Approach......Page 23
2.3 Algorithms of Learning......Page 25
2.4 Convergence of the Algorithms......Page 27
2.5 Criterion of Convergence of the Algorithms......Page 28
2.6 Modified Algorithms......Page 29
2.7 General Algorithms of Learning......Page 31
2.8 Special Cases......Page 33
2.9 Algorithms of Learning in the Presence of Constraints......Page 35
2.10 Special Cases......Page 38
Comments......Page 40
References......Page 42
3.1 Introduction......Page 45
3.2 Performance Indices of Learning......Page 46
3.3 Generalized Performance Indices of Learning......Page 47
3.4 Discrete Algorithms of Quasi-Optimal Learning......Page 48
3.5 Linear Discrete Algorithms of Optimal Learning: I......Page 50
3.6 Linear Discrete Algorithms of Optimal Learning: II......Page 52
3.8 The Simplest Linear Discrete Algorithms of Optimal Learning......Page 54
3.10 Continuous and Hybrid Algorithms of Optimal Learning......Page 57
3.11 Special Cases......Page 60
3.12 More on Continuous Algorithms of Optimal Learning......Page 61
3.13 The Limiting Case......Page 62
3.14 Discussion......Page 64
3.15 Algorithms of Learning with Repetition......Page 65
3.16 Conclusion......Page 67
Comments......Page 68
References......Page 70
4.1 Introduction......Page 72
4.2 Average Risk......Page 73
4.3 Conditions for the Minimum of the Average Risk......Page 74
4.4 Binary Case......Page 76
4.5 Classical Bayes Approach......Page 77
4.7 Maximum A Posteriori Probability Rule......Page 79
4.8 Mixed Decision Rule......Page 80
4.9 Neyman-Pearson Rule......Page 81
4.10 Min-Max Decision Rule......Page 83
4.11 General Decision Rule......Page 84
4.12 Discussion......Page 85
Comments......Page 87
References......Page 88
5.1 Introduction......Page 90
5.3 Binary Case......Page 91
5.4 Traditional Adaptive Approach......Page 92
5.5 Adaptive Bayes Approach: I......Page 94
5.6 Adaptive Bayes Approach: II......Page 97
5.7 Learning to Apply the Siegert-Kotelnikov Rule......Page 99
5.8 Learning to Apply the Mixed Decision Rule......Page 101
5.9 Learning to Apply the Neyman-Pearson Rule......Page 102
5.10 Is It Necessary to Learn the Min-Max Decision Rule?......Page 103
5.11 Learning to Apply the General Decision Rule......Page 104
5.13 Conclusion......Page 106
Comments......Page 108
References......Page 109
6.1 Introduction......Page 112
6.2 Goal of Self-Learning......Page 113
6.3 Binary Case......Page 114
6.4 Algorithms of Self-Learning......Page 115
6.5 Algorithms of Optimal Self-Learning......Page 116
6.6 Adaptive Bayes Approach......Page 120
6.7 Self-Learning When the Number of Regions Is Known......Page 122
6.8 Self-Learning When the Number of Regions Is Unknown: I......Page 123
6.9 Self-Learning When the Number of Regions Is Unknown: II......Page 126
6.10 Discussion......Page 127
Comments......Page 128
References......Page 129
7.1 Introduction......Page 131
7.2 Description of the System......Page 132
7.3 Structure of the Model......Page 133
7.5 Algorithms of Learning......Page 134
7.6 Linear Learning Model......Page 137
7.7 Optimal Learning Linear Model......Page 139
7.8 Nonlinear Learning Model: I......Page 140
7.9 Nonlinear Learning Model: II......Page 142
7.11 Influence of Noise......Page 143
7.12 Removing the Influence of Noise......Page 145
Comments......Page 146
References......Page 147
8.1 Introduction......Page 149
8.2 Statement of the Problem......Page 150
8.3 Structure of the Filter......Page 151
8.4 Optimal Wiener Filter......Page 153
8.5 Learning Wiener Filter......Page 154
8.6 Learning Wiener Filter with Known A Priori Information about Noise......Page 155
8.7 Learning Wiener Filter with Known A Priori Information about the Signal......Page 157
8.8 A Generalization......Page 158
8.9 Optimal Learning Wiener Filters......Page 159
8.10 Learning Filters of Another Type......Page 160
8.11 Optimal Learning Filter......Page 162
Comments......Page 163
References......Page 164
9.1 Introduction......Page 166
9.2 Perceptron......Page 167
9.3 Adaline......Page 168
9.4 Learning Receiver: I......Page 170
9.5 Learning Receiver: II......Page 173
9.6 Self-Learning Classifier......Page 176
9.7 Learning Filters......Page 179
9.8 Learning Antenna System......Page 184
9.9 Learning Communication System......Page 187
9.10 Learning Coding Device......Page 188
9.11 Self-Learning Sampler......Page 191
9.12 Learning Control System......Page 194
9.13 Learning Diagnostic Systems......Page 197
9.14 Establishment of Parametric Sequences......Page 199
Comments......Page 204
References......Page 207
Epilogue......Page 210
Author Index......Page 213
Subject Index......Page 217