Computational Neuroscience Models of the Basal Ganglia

Acknowledgements
Contents
1 Introduction
	Abstract
	References
2 The Molecular, Cellular, and Systems-Level Structure of the Basal Ganglia
	Abstract
	2.1 Anatomical Structure of Basal Ganglia
		2.1.1 Systems-Level
			2.1.1.1 Multiple Cortico-BG Loops
		2.1.2 Cellular Level
			2.1.2.1 Striatum—The Major Input Gateway
			2.1.2.2 The Oscillator Network of BG
			2.1.2.3 The Output Ports of BG (GPi and SNr)
			2.1.2.4 Dopaminergic System (SNc)
			2.1.2.5 Ventral Tegmental Area (VTA)
	References
3 The Motor, Cognitive, Affective, and Autonomic Functions of the Basal Ganglia
	Abstract
	3.1 Motor Processes of the Basal Ganglia
		3.1.1 Hand—Reaching, Handwriting, Precision Grip
		3.1.2 Gait
		3.1.3 Saccades
		3.1.4 Speech and Language
	3.2 Cognitive Processes of the Basal Ganglia
		3.2.1 Action Selection/Decision Making
		3.2.2 Attention
		3.2.3 Working Memory
		3.2.4 Sequence Learning
		3.2.5 Sleep Regulation
	3.3 Mood and Emotional Processes of the Basal Ganglia
		3.3.1 Negative and Positive Affect
	3.4 Autonomic Processes of the Basal Ganglia
	References
4 Classical Computational Approaches to Modeling the Basal Ganglia
	Abstract
	4.1 Dimensionality Reduction Models
	4.2 Action Selection Models
	4.3 Go/NoGo Models
	4.4 RL Models of Basal Ganglia
	4.5 Conclusions
	References
5 The Basal Ganglia System as an Engine for Exploration
	Abstract
	5.1 Introduction
		5.1.1 The Indirect Pathway and Exploration
	5.2 The Basic Model
		5.2.1 Striatum
		5.2.2 Modeling the STN–GPe System
			5.2.2.1 Modeling STN–GPe Neuron Pair
			5.2.2.2 Network Model of STN–GPe System
		5.2.3 GPi
		5.2.4 Action Selection in Thalamus
	5.3 Simulation Experiments
		5.3.1 Binary Action Selection
		5.3.2 Modeling the N-Armed Bandit Problem
			5.3.2.1 Computations in the Striatum
			5.3.2.2 Computations in STN–GPe System
			5.3.2.3 Computations in GPi
			5.3.2.4 Reward and Learning
			5.3.2.5 N-Armed Bandit—A Simulation Study
		5.3.3 Climbing Value Gradient Using δV
	5.4 Discussion
	Appendix
	References
6 Synchronization and Exploration 	in Basal Ganglia—A Spiking Network Model
	Abstract
	6.1 Introduction
	6.2 Methods
		6.2.1 Spiking Neuron Model of the Basal Ganglia
		6.2.2 Binary Action Selection Task
		6.2.3 The N-Armed Bandit Task
			6.2.3.1 Behavioral Model (Adapted from Bourdaud et al. (2008))
			6.2.3.2 Strategy for Slot Machine Selection
		6.2.4 Measures
			6.2.4.1 Synchronization
		6.2.5 Action Selection Using the Race Model
	6.3 Results
		6.3.1 Neural Dynamics
		6.3.2 Decision Making
	6.4 Discussion
	References
7 A Basal Ganglia Model of Freezing of Gait in Parkinson’s Disease
	Abstract
	7.1 Introduction
	7.2 Motivation, Objective, and Scope
	7.3 Methods and Results
		7.3.1 The Influence of Doorways on FOG
		7.3.2 The Role of Cognition in FOG
		7.3.3 Influence of Turning on FOG
		7.3.4 Freezing in Other Modalities
	7.4 Conclusions
	References
8 Modeling Precision Grip Force in Controls and Parkinson’s Disease Patients
	Abstract
	8.1 Precision Grip Force Neural Control
		8.1.1 Role of BG in PGLT
	8.2 Computational Models of Precision Grip
		8.2.1 Kim and Inooka (1994)
		8.2.2 de Gruijl, van der Smagt, and De Zeeuw (2009)
		8.2.3 Ulloa, Bullock and Rhodes (2003)
		8.2.4 Fagergren, Ekeberg, and Forssberg (2000)
		8.2.5 Fagergren, Ekeberg, and Forssberg (2003)
		8.2.6 Kim, Nakazawa, and Inooka (2002)
		8.2.7 Grip Force During Transient Friction Change (Gupta et al., 2013a, 2013b)
		8.2.8 Utility-Based Decision-Making Model of Grip Force Generation in Parkinson’s Patients (Gupta, Balasubramani, & Chakravarthy, 2013c)
	References
9 Go-Explore-NoGo (GEN) Paradigm in Decision Making—A Multimodel Approach
	Abstract
	9.1 Introduction
	9.2 Methods
		9.2.1 Spiking Izhikevich Two-Variable Neuron Model
		9.2.2 Hybrid Biophysical Model
			9.2.2.1 STN Neurons
			9.2.2.2 GPe Neurons
			9.2.2.3 GPi Neurons
			9.2.2.4 Synaptic Currents
			9.2.2.5 Dopaminergic Modulation
		9.2.3 Tasks
			9.2.3.1 Binary Action Selection Task
			9.2.3.2 n-Arm Bandit Task
	9.3 Results
		9.3.1 Binary Action Selection
		9.3.2 N-Arm Bandit Task
	9.4 Discussion
	References
10 A Cortico-Basal Ganglia Model to Understand the Neural Dynamics of Targeted Reaching in Normal and Parkinson’s Conditions
	Abstract
	10.1 Introduction
	10.2 Methods
		10.2.1 Arm Model
		10.2.2 The Sensory-Motor Cortical Loop
		10.2.3 Training the Cortical Loop
		10.2.4 The Basal Ganglia
		10.2.5 Prefrontal Cortex—Information of Goal Position
		10.2.6 Timescales of Motor Movement in the Cortex and the BG
		10.2.7 Simulating Pathology—Parkinsonian Condition
	10.3 Results
		10.3.1 Mapping of the Joint Configurations in the PC and MC
		10.3.2 Reaching Movements of the Arm
		10.3.3 Velocity Profiles of Controls and PD Patients
		10.3.4 Model Performance on the Pursuit Task
		10.3.5 Motor Initiation with the Cortico-BG Loop
		10.3.6 PD Symptoms
	10.4 Discussion
		10.4.1 Cortico-Basal Ganglia Loop as an Attractor Network
		10.4.2 Indirect Pathway for Exploration and Emergence of PD Symptoms
		10.4.3 Effect of Dopamine on Motor Performance
		10.4.4 Limitations and Future Directions
	Conflict of Interest
	References
11 Studying the Effect of Dopaminergic Medication and STN–DBS on Cognitive Function Using a Spiking Basal Ganglia Model
	Abstract
	11.1 Introduction
	11.2 Materials and Methods
		11.2.1 Spiking Neuron Model of the Basal Ganglia
		11.2.2 Behavioral Tasks
			11.2.2.1 Iowa Gambling Task (IGT)
			11.2.2.2 Probabilistic Learning Task (PLT)
		11.2.3 Simulating Tasks Using Spiking Neuron Network Model
			11.2.3.1 Cortico-Striatal Weight Update and Temporal Difference Error
			11.2.3.2 Simulating Untreated PD and Medically Treated 	PD Conditions
			11.2.3.3 DBS Current
		11.2.4 Performance Measures
			11.2.4.1 IGT Score
			11.2.4.2 PLT-Learning
				PLT-Testing Accuracy and Difference in Reward Expectation (DRE)
				PLT-Choice/Avoidance Accuracy
	11.3 Results
		11.3.1 De-synchronization by DBS Current
	11.4 Discussion
	References
12 Modeling Serotonin’s Contributions to Basal Ganglia Dynamics
	Abstract
	12.1 Introduction
	12.2 Methods
		12.2.1 Modeling the Joint Functions of DA and 5-HT in the BG: An Abstract Model (Model I)
		12.2.2 Modeling the Joint Functions of DA and 5-HT in the BG: A Network Model (Model II)
	12.3 Results
		12.3.1 Reward–Punishment Sensitivity
		12.3.2 Serotonin and Timescale of Reward Prediction
		12.3.3 Serotonin and Risk Sensitivity
	12.4 Discussion
		12.4.1 Significance of Sign(Qt)
		12.4.2 5-HT-DA Interaction in the ‘Risk’ Component of Decision Making
		12.4.3 Main Finding of the DA-5-HT-Based BG Network Model for Utility-Based Decision Making
		12.4.4 Striatal DA and 5-HT
		12.4.5 The Co-expressing D1R–D2R MSNs
	12.5 Future Work
	References
13 Modeling Serotonin’s Contributions to Basal Ganglia Dynamics in Parkinson’s Disease with Impulse Control Disorders
	Abstract
	13.1 Introduction
	13.2 Probabilistic Learning, Parkinson’s Disease, and Impulse Control Disorder
		13.2.1 Experiment Summary
		13.2.2 Simulation
		13.2.3 Results
	13.3 Applying the Network Model of BG to Probabilistic Learning Task
		13.3.1 Results
	13.4 Analyzing the Reaction Times and Impulsivity
		13.4.1 Modeling Results
	13.5 Discussion
	References
14 An Oscillatory Neural Network Model for Birdsong Learning and Generation: Implications for the Role of Dopamine in Song Learning
	Abstract
	14.1 Introduction
		14.1.1 Birdsong Learning
		14.1.2 Neuroanatomy of Birdsong
		14.1.3 Dopamine in Learning
		14.1.4 Modeling Bird Song Learning
		14.1.5 Objective
	14.2 Model Description
		14.2.1 The Motor Pathway Model
		14.2.2 The Anterior Forebrain Pathway in the Model
		14.2.3 The Respiratory System and Syrinx Model
		14.2.4 The Vocal Filter Model
		14.2.5 Training Algorithm
	14.3 Results
		14.3.1 Lesion Studies
			14.3.1.1 LMAN Lesion
			14.3.1.2 Area X Lesion
		14.3.2 Dopamine Depletion Studies
	14.4 Discussion
	References
15 The Basal Ganglia: Summary and Future Modeling Research
	Abstract
	15.1 Applying the BG Model to Various Behavioral Processes
	15.2 Clinical Applications
	References