Neural networks and fuzzy systems_A Dynamical Systems Approach to Machine Intelligence

Kosko B. Neural networks and fuzzy systems_A Dynamical Systems Approach to Machine Intelligence ......Page 3
Copyright ......Page 4
CONTENTS ......Page 6
FOREWORD by Lotfi A. Zadeh  xvii ......Page 15
FOREWORD by James A. Anderson xix ......Page 17
Preface xxv ......Page 22
1 NEURAL NETWORKS AND FUZZY SYSTEMS 1......Page 25
Neural Pre-Attentive and Attentive Processing, 2 ......Page 26
Fuzziness as Multivalence 3 ......Page 27
Bivalent Paradoxes as Fuzzy Midpoints, 4 ......Page 28
Fuzziness in the Twentieth Century, 5 ......Page 29
Sets as Points in Cubes, 7 ......Page 31
Subsethood and Probability, 9 ......Page 33
The Dynamical-Systems Approach to Machine Intelligence: The Brain as a Dynamical System 12 ......Page 36
Neural and Fuzzy Systems as Function Estimators, 13 ......Page 37
Neural Networks as Trainable Dynamical Systems, 14 ......Page 38
Fuzzy Systems and Applications, 18 ......Page 42
Intelligent Behavior as Adaptive Model-Free Estimation 19 ......Page 43
Generalization and Creativity, 20 ......Page 44
Learning as Change, 22 ......Page 46
Expert-System Knowledge as Rule Trees, 24 ......Page 48
Symbolic vs. Numeric Processing, 25 ......Page 49
Fuzzy Systems as Structured Numerical Estimators, 26 ......Page 50
Generating Fuzzy Rules with Product-Space Clustering, 28 ......Page 52
Fuzzy Systems as Parallel Associators, 29 ......Page 53
Fuzzy Systems as Principle-Based Systems, 32 ......Page 56
References 34 ......Page 58
Problems 36 ......Page 60
Neural Network Theory 38 ......Page 62
Neurons as Functions 39 ......Page 63
Signal Monotonicity 40 ......Page 64
Biological Activations and Signals 41 ......Page 65
Competitive Neuronal Signals, 43 ......Page 67
Neuronal Dynamical Systems 44 ......Page 68
Neuronal State Spaces, 45 ......Page 69
Signal State Spaces as Hypercubes, 46 ......Page 70
Neuronal Activations as Short-Term Memory, 47 ......Page 71
Common Signal Functions 48 ......Page 72
Pulse-Coded Signal Functions 50 ......Page 74
Velocity-Difference Property of Pulse-Coded Signals, 51 ......Page 75
References 52 ......Page 76
Problems 53......Page 77
Neuronal Dynamical Systems 55 ......Page 79
Passive Membrane Decay, 56 ......Page 80
Membrane Resting Potentials, 57 ......Page 81
Additive External Input, 58 ......Page 82
Synaptic Connection Matrices, 59 ......Page 83
Bidirectional and Unidirectional Connection Topologies, 60 ......Page 84
Additive Activation Models 61 ......Page 85
Bivalent Additive BAM, 63 ......Page 87
Bidirectional Stability, 68 ......Page 92
Lyapunov Functions, 69 ......Page 93
Bivalent BAM Theorem, 73 ......Page 97
BAM Connection Matrices 79 ......Page 103
Optimal Linear Associative Memory Matrices, 81 ......Page 105
Autoassociative OLAM Filtering, 83 ......Page 107
BAM Correlation Encoding Example, 85 ......Page 109
Memory Capacity: Dimensionality Limits Capacity, 91 ......Page 115
The Hopfield Model, 92 ......Page 116
Additive Dynamics and the Noise-Saturation Dilemma 94 ......Page 118
Grossberg's Saturation Theorem, 95 ......Page 119
General Neuronal Activations: Cohen-Grossberg and Multiplicative Models 99 ......Page 123
References 103 ......Page 127
Problems 106 ......Page 130
Part I: Discrete Additive Bidirectional Associative Memory (BAM), 108 ......Page 132
Part II, 109 ......Page 133
Learning as Encoding, Change, and Quantization 111 ......Page 135
Supervised and Unsupervised Learning in Neural Networks, 113 ......Page 137
Four Unsupervised Learning Laws 115 ......Page 139
Four Deterministic Unsupervised Learning Laws, 116 ......Page 140
Brownian Motion and White Noise, 118 ......Page 142
Measurability and Sigma-Algebras, 119 ......Page 143
Probability Measures and Density Functions, 122 ......Page 146
Gaussian White Noise as a Brownian Pseudoderivative Process, 127 ......Page 151
Stochastic Unsupervised Learning and Stochastic Equilibrium 131 ......Page 155
Stochastic Equilibrium, 133 ......Page 157
Asymptotic Correlation Encoding, 138 ......Page 162
Hebbian Correlation Decoding, 140 ......Page 164
Competitive Learning 145 ......Page 169
Competition as Indication, 146 ......Page 170
Competition as Correlation Detection, 147 ......Page 171
Asymptotic Centroid Estimation, 148 ......Page 172
Competitive Covariance Estimation, 149 ......Page 173
Fuzzy Cognitive Maps, 152 ......Page 176
Adaptive Causal Inference, 158 ......Page 182
Klopf s Drive Reinforcement Model, 159 ......Page 183
Concomitant Variation as Statistical Covariance, 161 ......Page 185
Pulse-Coded Differential Hebbian Learning, 163 ......Page 187
Differential Competitive Learning 166 ......Page 190
Differential Competitive Learning as Delta Modulation, 168 ......Page 192
References 170 ......Page 194
Problems 173 ......Page 197
Part I: Competitive Learning, 175 ......Page 199
Part II: Differential Competitive Learning, 176 ......Page 200
5 SYNAPTIC DYNAMICS II: SUPERVISED LEARNING 179......Page 203
Supervised Function Estimation 180 ......Page 204
Supervised Learning as Operant Conditioning 181 ......Page 205
Supervised Learning as Stochastic Pattern Learning with Known Class Memberships 183 ......Page 207
Supervised Learning as Stochastic Approximation 185 ......Page 209
The Perceptron: Learn Only If Misclassify, 187 ......Page 211
The LMS Algorithm: Linear Stochastic Approximation, 190 ......Page 214
History of the Backpropagation Algorithm, 196 ......Page 220
Feedforward Sigmoidal Representation Theorems, 199 ......Page 223
Multilayer Feedforward Network Architectures, 201 ......Page 225
Backpropagation Algorithm and Derivation, 203 ......Page 227
Backpropagation as Stochastic Approximation, 210 ......Page 234
Robust Backpropagation, 211 ......Page 235
Other Supervised Learning Algorithms, 212 ......Page 236
References 213 ......Page 237
Problems 215 ......Page 239
Part I: Exclusive-OR (XOR), 218 ......Page 242
Part II: Sine Function, 219 ......Page 243
Part III: Training Set versus Test Set, 220 ......Page 244
Neural Networks as Stochastic Gradient Systems 221 ......Page 245
Global Equilibria: Convergence and Stability 223 ......Page 247
Competitive AVQ Stochastic Differential Equations, 225 ......Page 249
Unsupervised Competitive Learning (UCL), 227 ......Page 251
Stochastic Equilibrium and Convergence, 228 ......Page 252
Global Stability of Feedback Neural Networks 232 ......Page 256
ABAMs and the Stability-Convergence Dilemma, 233 ......Page 257
Stability-Convergence Dilemma, 235 ......Page 259
The ABAM Theorem, 236 ......Page 260
Higher-Order ABAMs, 239 ......Page 263
Adaptive Resonance ABAMs, 240 ......Page 264
Differential Hebbian ABAMS, 241 ......Page 265
Structural Stability of Unsupervised Learning 242 ......Page 266
Random Adaptive Bidirectional Associative Memories 243 ......Page 267
Noise-Saturation Dilemma and the RABAM  Noise-Suppression Theorem, 247 ......Page 271
RABAM Noise-Suppression Theorem, 248 ......Page 272
RABAM Annealing, 253 ......Page 277
References 255 ......Page 279
Problems 257 ......Page 281
Part I: Random Adaptive Bidirectional Associative Memory (RABAM), 258 ......Page 282
Part II: Binary Adaptive Resonance Theory (ART-1), 259 ......Page 283
Adaptive Fuzzy Systems 262......Page 286
Fuzzy Sets and Systems 263 ......Page 287
Fuzziness in a Probabilistic World 264 ......Page 288
Randomness vs. Ambiguity: Whether vs. How Much 265 ......Page 289
The Universe as a Fuzzy Set 268 ......Page 292
The Geometry of Fuzzy Sets: Sets as Points 269 ......Page 293
Paradox at the Midpoint, 273 ......Page 297
Counting with Fuzzy Sets, 274 ......Page 298
The Fuzzy Entropy Theorem 275 ......Page 299
The Subsethood Theorem 278 ......Page 302
Bayesian Polemics, 289 ......Page 313
The Entropy-Subsethood Theorem 293 ......Page 317
References 294 ......Page 318
Problems 296 ......Page 320
Fuzzy Systems as Between-Cube Mappings 299 ......Page 323
Fuzzy and Neural Function Estimators 302 ......Page 326
Neural vs. Fuzzy Representation of Structured Knowledge, 304 ......Page 328
FAMs as Mappings, 306 ......Page 330
Fuzzy Vector-Matrix Multiplication: Max-Min Composition, 307 ......Page 331
Fuzzy Hebb FAMs 308 ......Page 332
The Bidirectional FAM Theorem for Correlation-Minimum Encoding, 310 ......Page 334
Correlation-Product Encoding, 311 ......Page 335
Superimposing FAM Rules, 313 ......Page 337
Recalled Outputs and “Defuzzification”, 314 ......Page 338
FAM System Architecture, 316 ......Page 340
Binary Input-Output FAMs: Inverted-Pendulum Example, 317 ......Page 341
Multiantecedent FAM Rules: Decompositional Inference, 322 ......Page 346
Adaptive Decompositional Inference, 326 ......Page 350
Adaptive FAMs: Product-Space Clustering in FAM Cells 327 ......Page 351
Adaptive FAM-Rule Generation, 328 ......Page 352
Adaptive BIOFAM Clustering, 329 ......Page 353
Adaptive BIOFAM Example: Inverted Pendulum, 333 ......Page 357
References 335 ......Page 359
Problems 336 ......Page 360
Software Problems 337 ......Page 361
Fuzzy and Neural Control Systems 339 ......Page 363
Fuzzy Truck Backer-Upper System, 340 ......Page 364
Neural Truck Backer-Upper System, 345 ......Page 369
Comparison of Fuzzy and Neural Systems, 346 ......Page 370
Sensitivity Analysis, 347 ......Page 371
Adaptive Fuzzy Truck Backer-Upper, 348 ......Page 372
Fuzzy Truck-and-Trailer Controller, 352 ......Page 376
AFAM Truck-and-Trailer Control Systems, 356 ......Page 380
Conclusion, 360 ......Page 384
References 361 ......Page 385
Transform Image Coding with Adaptive Fuzzy Systems 363 ......Page 387
Adaptive Cosine Transform Coding of Images, 365 ......Page 389
Adaptive FAM systems for Transform Coding 366 ......Page 390
Selection of Quantizing Fuzzy-Set Values, 367 ......Page 391
Product-Space Clustering to Estimate FAM Rules, 368 ......Page 392
Differential Competitive Learning, 370 ......Page 394
Simulation, 373 ......Page 397
Conclusion, 374 ......Page 398
References 377 ......Page 401
Problems 378 ......Page 402
Fuzzy and Math-Model Controllers 379 ......Page 403
Real-Time Target Tracking 381 ......Page 405
Fuzzy Controller 382 ......Page 406
Fuzzy-Centroid Computation, 386 ......Page 410
Fuzzy-Controller Implementation, 390 ......Page 414
Kalman-Filter Controller 392 ......Page 416
Fuzzy and Kalman-Filter Control Surfaces, 394 ......Page 418
Simulation Results 396 ......Page 420
Sensitivity Analysis, 399 ......Page 423
Adaptive FAM (AFAM), 402 ......Page 426
References 406 ......Page 430
APPENDIX: NEURAL AND FUZZY SOFTWARE INSTRUCTIONS 407 ......Page 431
General, 408 ......Page 432
ART, 409 ......Page 433
BAM, 411 ......Page 435
BKP, 413 ......Page 437
CL, 414 ......Page 438
RABAM, 416 ......Page 440
Fuzzy Truck Backer-Upper Control System, 418 ......Page 442
Fuzzy Target-Tracking Demonstration, 419 ......Page 443
Adaptive Fuzzy Control of Inverted Pendulum, 421 ......Page 445
INDEX 425......Page 449
cover......Page 1