Adaptive Brain

The Adaptive Brain I: Cognition, Learning, Reinforcement, and Rhythm......Page 4
Copyright Page......Page 5
TABLE OF CONTENTS......Page 11
CHAPTER 1: A PSYCHOPHYSIOLOGICAL THEORY OF REINFORCEMENT, DRIVE, MOTIVATION, AND ATTENTION 1......Page 22
1. Introduction......Page 25
2. Buffer the Learning Cells......Page 26
3. A Digression on Classical Conditioning......Page 27
5. A Thought Experiment: The Synchronization Problem of Classical Conditioning......Page 28
6. Some Experimental Connections......Page 31
7. Conditioned Arousal......Page 32
8. Secondary Reinforcers......Page 34
9. Minimal Network Realization of Conditioned Nonspecific Arousal......Page 35
10. Secondary Conditioning: A Principle of Equivalence for the Anatomy of CS and UCS......Page 37
11. Are Drives Energizers or Sources of Information?......Page 38
13. Internal Facts versus External Demands: Existence of Homeostatic, or Drive, Inputs......Page 39
14. Conditioned Reinforcers, Drives, Incentive Motivation, and Habits 21......Page 42
15. Comparison with Hullian Concepts......Page 44
16. Data on Conditioned Reinforcers and Drives......Page 45
18. Data on Self-Stimulation......Page 46
19. Reinforcement Without Drive Reduction......Page 47
20. Go Mechanism, Amplifiers, Now Print......Page 48
21. Data on Incentive Motivation......Page 50
23. Late Nonspecific Potential Shifts......Page 51
24. An Emergent Neocortical Analog......Page 52
25. Motivational Set: Is Incentive Motivation Conditionable?......Page 57
26. Distinct Cortical Recurrent Loops for STM and LTM......Page 58
27. Motivation-Dependent Responses to Sensory Cues: Multiple Sensory Representations or Developmental Competition for Synaptic Sites?......Page 60
28. Sensory-Drive Heterarchy: Competitive Decisions After Drives and Conditioned Reinforcers Interact......Page 61
29. Differential Effects of Drive and Reinforcement on Learning Rate versus Performance Speed......Page 67
31. Suppression by Punishment......Page 69
32. Antagonistic Rebound and Learned Avoidance......Page 71
33. Slowly Accumulating Transmitter Gates in Tonically Aroused Competing Channels Cause Antagonistic Rebound......Page 73
34. Dipole Fields in Motivational Processing by the Hippocampal-Hypothalamic Axis......Page 76
35. Some Pharmacological and Physiologiral Correlates 58 of Motivational Dipole Fields......Page 79
36. Competition, Normalization, and STM Among 60 Sensory Representations......Page 81
37. Attention and the Persistence Paradox of 62 Parallel Processing......Page 83
38. Sensory Incentive versus Motor Incentive: The Hippocampus as a Cognitive Map......Page 85
39. Expectancy Matching and Attentional Reset: Unblocking and Dishabituation......Page 87
40. Concluding Remarks......Page 92
References......Page 95
CHAPTER 2: SOME PSYCHOPHYSIOLOGICAL AND PHARMACOLOGICAL CORRELATES OF A DEVELOPMENTAL, COGNITIVE, AND MOTIVATIONAL THEORY......Page 103
1. Introduction: Self-Organizing Internal Representations......Page 104
4. Hypothesis Testing and Error Correction in a Fluctuating Environment......Page 106
5. Attention Shifts in Pavlovian Experiments......Page 107
6. Transient Effects: Causality, STM Reset, and P300......Page 109
8. The Problem of Stimulus Equivalence......Page 111
9. Categorical Perception, Bayesian Statistics, and Temporally I'nstable Feedforward Coding......Page 112
10. Unconscious Inferences: Why Do Learned Feedback Expectancies Exist?......Page 114
11. Processing Negativity and Match Detection......Page 115
14. The Noise-Saturation Dilemma: Noise Suppression and Pattern Matching in Shunting On-Center Off-Surround Networks......Page 118
15. Disinhibition of Mismatch Negativity by Unexpected Events......Page 122
16. Attentional versus Orienting Subsystems: Error Perseveration, P300, and the Hippocampus......Page 123
17. Parallel Hypothesis Testing in Real Time: STM Reset and Renormalization......Page 125
18. Contrast Enhancement and STM Normalization in Competitive Feedback Networks......Page 126
19. Limited Capacity STM System: Automatic versus Controlled Processing......Page 127
20. Sigmoid Signal Functions and Noise Suppression......Page 129
22. Gated Dipoles: Antagonistic Rebound Due to Slow Transmitter Gating in Competing Channels......Page 130
23. Tolerance Implies Withdrawal: Rebound Insomnia and a Way Out......Page 134
25. An Arousal Test of the Gating Transmitter......Page 135
26. P300, Catecholamine Rebound, CNV Rebound, Critical Period Termination......Page 137
28. Adaptive Resonance: A Solution to the Stability-Plasticity Dilemma......Page 138
29. The Dipole Field......Page 139
30. Drives, Conditioned Reinforcers, Incentive Motivation, and CNV......Page 142
31. Extinction, Conditioned Emotional Responses, Conditioned Avoidance Responses, and Secondary Conditioning......Page 144
32. Cholinergic-Catecholaminergic Interactions in Drinking versus the Brain as a Chemical Bath......Page 145
33. Intragastric versus Normal Drinking......Page 146
35. Self-Stimulation and Kindling......Page 148
36. Critical Period Reversal, P300 Suppression, and a Test of LTM Encoding by Cholinergic Pathways......Page 151
38. Motivational Switching and Hysteresis Without a Mythical Cusp Catastrophe......Page 153
39. Formal Symptoms of Underaroused Depression......Page 154
42. Inverted U in P300 and CNV......Page 155
44. Juvenile Hyperactivity......Page 156
46. Analgesia: Endorphins versus Loud Noise......Page 157
47. The Hyperphagic Syndrome and the Mythical Set-Point......Page 158
49. A Normal versus Hyperphagic Feeding Cycle......Page 159
51. Adaptive Resonance Between Dipole Fields......Page 164
53. Theta, CNV, and Motor Potential Correlates of a Hippocampal Model......Page 166
54. A Sensory Dipole Field: The Synchronization Problem and DC Potential Shifts......Page 168
56. Valenstein Effect: Nonspecific Drive Representations or Nonspecific Conditioned Reinforcers and Conditioned Incentives?......Page 170
57. Discrimination and Overshadowing Due to Conditioned Incentive Motivational Feedback: CNV Correlates......Page 171
59. Cortical Reset Triggers Hippocampal Reset: Two Distinct P300’s......Page 173
62. Concluding Remarks......Page 174
63. Transmitters as Gates......Page 175
64. Intracellular Adaptation and Habituation......Page 176
65. A Gated Dipole......Page 177
66. Rebound Due to Phasic Cue Offset......Page 178
68. Inverted U in Dipole Output......Page 181
70. Paradoxical On-Reaction to Unexpected Events and Differential Enhancement of Overshadowed Cues......Page 182
71. Paradoxical Lack of Rebound to Phasic Decrement: Ordering of Reinforcement Magnitude......Page 183
72. Inhibiting Excitatory Resistance versus Exciting Inhibitory Conductance in Disinhibitory Incentive Motivational Pathways......Page 184
73. Intracellular Dipoles......Page 186
75. Paradoxical Inhibitory Action of Excitatory Transmitter on Tonically Aroused Cells References......Page 187
References......Page 189
CHAPTER 3: PROCESSING OF EXPECTED AND UNEXPECTED EVENTS DURING CONDITIONING AND ATTENTION: A PSYCHOPHYSIOLOGICAL THEORY......Page 201
3. Some Internal Paradoxes......Page 203
5. Causality Violation on the Behaviorally Observable Level......Page 205
7. Formal versus Physical Concepts: A Second Type of LTM?......Page 206
8. Overexpectation, STM Signaling, LTM Gating, and Drive Representations......Page 207
10. An Alternative Processing Framework: Complementary Attentional and Orienting Subsystems......Page 209
11. The Stability-Plasticity Dilemma and Evoked Potential Correlates......Page 210
12. Gated Dipoles......Page 211
13. Antagonistic Rebound to Cue Offset......Page 213
15. What Is An Expectation?......Page 214
17. STM Reset versus STM Resonance......Page 217
19. The Code of Adaptive Resonances......Page 219
20. The Noise-Saturation Dilemma......Page 220
21. STM Contrast Enhancement and Normalization: Hypothesis Testing and Overshadowing in a Limited-Capacity System......Page 221
22. Overshadowing and Sigmoid Signal Functions......Page 225
24. Noise Suppression and Pattern Matching......Page 226
26. Gated Feedback Dipoles......Page 227
28. Extinction, Conditioned Emotional Responses, Conditioned Avoidance Responses, and Secondary Conditioning......Page 231
29. Motivational Baseline, Switching, and Hysteresis......Page 233
31. A Motivational Dipole Field: Drive-Reinforcer Matching and Motivational Competition......Page 235
32. A Sensory Dipole Field: The Synchronization Problem and DC Potential Shifts......Page 237
34. Unblocking, Context, and Habituation......Page 240
35. Double Shock Experiments: Initial STM Advantage of Surprising Events......Page 242
37. Incentive Motivation Feedback Influences STM Competition......Page 243
40. The Tone Is More Unexpected After Two Shocks......Page 244
43. Modulation of US and Drive Input Effects by Expectancies, Conditioned Reinforcers, and Transmitter Habituation......Page 245
44. Latent Inhibition......Page 246
46. Overshadowing During Key Pecking......Page 247
48. Two Distinct P300 Evoked Potentials in Cortex and Hippocampus......Page 249
50. Some Comparisons With the Formal Models......Page 250
51. Schedule Interactions and Behavioral Contrast......Page 251
References......Page 252
CHAPTER 4: NEURAL DYNAMICS OF CATEGORY LEARNING AND RECOGNITION: ATTENTION, MEMORY CONSOLIDATION, AND AMNESIA......Page 259
1. Introduction: Self-organization of Recognition Categories......Page 261
2. Bottom-Up Adaptive Filtering and ContrastEnhancement in Short Term Memory......Page 266
3. Top-Down Template Matching and Stabilization of Code Learning......Page 269
4. Interactions Between Attentional and Orienting Subsystems: STM Reset and Search......Page 270
5. Attentional Gain Control and Attentional Priming......Page 271
6. Matching: The 2/3 Rule......Page 272
7. Direct Access to Subsets and Supersets......Page 275
8. Weber Law Rule and Associative Decay Rule for Long Term Memory......Page 277
9. Fast Learning and Slow Learning: The Direct Access Rule......Page 278
10. Stable Choices in Short Term Memory......Page 279
11. Order of Search and the Subset Recoding Property......Page 280
12. Example of Code Instability......Page 281
13. Search of Subsets, Supersets, and Mixed Sets......Page 284
14. The Nature of Categorical Invariance During Learning......Page 288
15. Vigilance, Orienting, and Reset......Page 290
16. Distinguishing Signal from Noise in Patterns of Variable Complexity: Weighing the Evidence......Page 292
17. Vigilance Level Tunes Categorical Coarseness: Environmental Feedback......Page 295
18. Universal Recognition Design Across Modalities......Page 296
19. Interdisciplinary Relationships: Word Recognition, Evoked Potentials, and Medial Temporal Amnesia......Page 299
Appendix: Network Equations......Page 302
References......Page 305
CHAPTER 5: ABSOLUTE STAILITY OF GLOBAL PATTERN FORMATION AND PARALLEL MEMORY STORAGE BY COMPETITIVE NEURAL NETWORKS 287......Page 308
I. Introduction: Absolute Stability of Global Pattern Formation in Self-Organizing Networks......Page 310
II. Some Sources of Sustained Oscillations......Page 312
III. A Global Liapunov Function......Page 314
IV. Application of the LaSalle Invariance Principle......Page 315
V. Decomposition of Equilibria into Suprathreshold and Subthreshold Variables......Page 319
VI. Almost All Suprathreshold Equilibrium Sets Are Countable......Page 320
VII. All w-Limit Points Are Equilibria......Page 321
VIII. Neural Networks with Finitely Many Suprathreshold Equilibrium Points......Page 322
IX. Concluding Remarks......Page 327
References......Page 328
CHAPTER 6: A NEURAL THEORY OF CIRCADIAN RHYTHMS: THE GATED PACEMAKER......Page 330
B. Multiple Oscillators from Similar Mechanisms......Page 333
D. Comparison with other Pacemaker Models......Page 334
2. The Gated Pacemaker......Page 335
3. Qualitative Basis of Oscillations......Page 338
4. Parameter Estimation......Page 341
5. A Typical Oscillation in the Dark......Page 342
6. Phase Response Curves in Diurnal and Nocturnal Gated Pacemakers......Page 344
7. Parametric Structure of Oscillations: Threshold-Linear Signal Function......Page 348
9. Dependence of Solution Types on Signal Function: The Sigmoid Case......Page 351
10. Parametric Structure of Oscillations: Sigmoid Signal Function......Page 353
11. Mittens, Oyster Shells, Sequence Clusters, and Chaos......Page 355
12. Singular Perturbation Analysis......Page 362
13. Concluding Remarks: Interdisciplinary Applications......Page 368
References......Page 370
CHAPTER 7: A NEURAL THEORY OF CIRCADIAN RHYTHMS: ASCHOFF’S RULE IN DIURNAL AND NOCTURNAL MAMMALS......Page 373
2. Aschoff’s Rule, The Circadian Rule, and Exceptions......Page 375
3. Asymmetry of Fatigue and Light in Diurnal and Nocturnal Models......Page 376
5. Testing Existence of Fatigue Signal......Page 18
6. Gated Pacemaker Model......Page 379
7. Signal Functions, Activity Thresholds, and Attenuation of Light Input During Sleep......Page 381
8. Aschoff's Rule and its Exceptions: Numerical Studies......Page 383
10. Light Attenuation and Self-Selected Light-Dark Cycles in Diurnal Mammals: A Prediction......Page 387
12. Analysis of Aschoff's Rule......Page 389
13. Analysis of Joint Action of Fatigue and Light Attenuation on Circadian Period......Page 394
14. Analysis of Pacemaker Without Fatigue......Page 398
15. Comparison With Other Models......Page 401
16. Switching Between Diurnal and Nocturnal Properties......Page 404
References......Page 405
CHAPTER 8: A NEURAL THEORY OF CIRCADIAN RHYTHMS: SPLIT RHYTHMS, AFTER-EFFECTS, AND MOTIVATIONAL INTERACTIONS......Page 407
1. Introdurtion: A Neural Model of the Circadian System in the Mammalian Suprachiasmatic Nuclei......Page 409
2. Homeostatic and Nonhomeostatic Modulators of the Circadian Pacemaker......Page 410
3. Long-Term After-Effects: Parametric and Nonparametric Experiments......Page 411
4. Split Rhythms: Influences of Light, Hormones, and SCN Ablation......Page 418
5. The Gated Pacemaker Model......Page 427
6. Signal Functions, Activity Thresholds, and Attenuation of Light Input During Sleep......Page 430
7. Long-Term After-Effects: Slow Gain Control and Associative Conditioning......Page 431
8. Analysis of After-Effects......Page 440
9. Alternative Slow Gain Control Processes......Page 443
10. Split Rhythms and Inconsistent After-Effects on Period......Page 446
11. Split Rhythms: Slow Gain Control Processes......Page 447
12. Analysis of Split Rhythms: Depression of Gain-by LL -......Page 449
13. Analvsis of Split Rhvthms: Interaction of Slow Gain, Pacemaker Gate, and Fatigue......Page 450
15. Fatigue as an Internal Zeitgeber......Page 451
17. A Formal Connection Between After-Effects and Aschoff's Rule......Page 452
18. Regulation of Motivated Behavior by Hierarchical Networks: The Homology Between Pacemaker Circuits and Motivational Circuits......Page 454
20. The Underaroused and Overaroused Depressive Syndromes......Page 457
21. Anticipatory Wheel Turning, Ultradian Feeding Cvcles, and After-Effects of a Single Pulse of Light......Page 458
23. Comparison With Other Models Appendix: Choice of Parameters......Page 459
References......Page 464
CHAPTER 9: ASSOCIATIVE AND COMPETITIVE PRINCIPLES OF LEARNING AND DEVELOPMENT: THE TEMPORAL UNFOLDING AND STABILITY OF STM AND LTM PATTERNS......Page 469
2. From List Learning to Neural Networks: The Self-organization of Individual Behavior......Page 470
3. Unitized Nodes, Short Term Memory, and Automatic Activation......Page 471
5. The Inadequacy of Rehearsal as an Explanatory Concept......Page 472
7. Network versus Computer Parsing: Distinct Error Gradients at Different List Positions......Page 474
8. Graded STM and LTM Patterns: Multiplicative Sampling and Slow Decay by LTM Traces......Page 475
9. Binary versus Continuous Associative Laws......Page 477
11. Behavioral Choices and Competitive Feedback......Page 478
12. Skewing of the Bow: Symmetry-Breaking Between the Future and the Past......Page 479
13. Evolutionary Invariants of Associative Learning: Absolute Stability of Parallel Pattern Learning......Page 482
14. Local Symmetry and Self-similarity in Pattern Learning and Developmental Invariance......Page 484
15. The Unit of LTM is a Spatial Pattern: Global Constraints on Local Network Design......Page 485
17. The Primacy of Shunting Competitive Networks over Additive Networks......Page 489
18. The Noise-Saturation Dilemma and Absolute Stability of Competitive Decision-Making......Page 490
19. The Babel of Code Development Models......Page 491
21. Outstars and Instars......Page 492
22. Adaptive Filtering of Spatial Pattern Sequences......Page 495
23. Synaptic Conservation, Code Invariance, and Code Instability......Page 497
24. Critical Period Termination, the StabilityPlasticity Dilemma, and Adaptive Resonance......Page 498
25. Stable Coding of Pattern Sequences......Page 499
References......Page 502
Author Index......Page 507
Subject Index......Page 513