The probabilistic foundations of rational learning

Contents......Page 8
List of Figures......Page 11
 2.1 Two-armed bandit problem.......Page 50
 3.1 Taking Turns game.......Page 73
 3.3 Markov Exchangeability......Page 77
 3.4 Cycles......Page 79
 3.5 Markov Reinforcement Learning......Page 83
Preface and Acknowledgments......Page 12
Abstract Models of Learning......Page 16
1 Consistency and Symmetry......Page 24
 1.1 Probability......Page 25
 1.2 Pragmatic Approaches......Page 27
 1.3 Epistemic Approaches......Page 30
 1.4 Conditioning and Dynamic Consistency......Page 34
 1.5 Symmetry and Inductive Inference......Page 39
 1.6 Summary and Outlook......Page 44
2 Bounded Rationality......Page 47
 2.1 Fictitious Play......Page 48
 2.2 Bandit Problems......Page 49
 2.3 Payoff-Based Learning Procedures......Page 52
 2.4 The Basic Model of Reinforcement Learning......Page 56
 2.5 Luce’s Choice Axiom......Page 58
 2.6 Commutative Learning Operators......Page 62
 2.7 A Minimal Model......Page 65
 2.8 Rationality and Learning......Page 67
 3.1 Taking Turns......Page 71
 3.2 Markov Fictitious Play......Page 74
 3.6 Markov Learning Operators......Page 87
 3.7 The Complexity of Learning......Page 88
4 Large Worlds......Page 92
 4.1 It’s a Large World (After All)......Page 93
 4.2 Small World Rationality......Page 97
 4.3 Learning the Unknown......Page 100
 4.4 Exchangeable Random Partitions......Page 103
 4.5 Predicting the Unpredictable......Page 105
 4.6 Generalizing Fictitious Play......Page 109
 4.7 Generalizing Reinforcement Learning......Page 111
 4.8 Learning in Large Worlds with Luce’s Choice Axiom......Page 113
5 Radical Probabilism......Page 117
 5.1 Prior Probabilities......Page 118
 5.2 Probability Kinematics......Page 121
 5.3 Radical Probabilism......Page 126
 5.4 Dynamically Consistent Models......Page 129
 5.5 Martingales......Page 134
 5.6 Conditional Probability and Conditional Expectation......Page 136
 5.7 Predicting Choices......Page 139
 6.1 Probabilities of Future Probabilities......Page 141
 6.2 Dynamic Consistency......Page 144
 6.3 Expected Accuracy......Page 146
 6.4 Best Estimates......Page 148
 6.5 General Distance Measures......Page 152
 6.6 The Value of Knowledge......Page 154
 6.7 Genuine Learning......Page 156
 6.8 Massaging Degrees of Belief......Page 158
 6.9 Countable Additivity......Page 161
7 Disagreement......Page 164
 7.1 Agreeing to Disagree......Page 165
 7.2 Diverging Opinions......Page 166
 7.3 Learning from Others......Page 169
 7.4 Averaging and Inductive Logic......Page 171
 7.5 Generalizations......Page 176
 7.6 Global Updates......Page 178
 7.7 Alternatives......Page 179
 7.8 Conclusion......Page 180
8 Consensus......Page 182
 8.1 Convergence to the Truth......Page 183
 8.2 Merging of Opinions......Page 186
 8.3 Nash Equilibrium......Page 188
 8.4 Merging and Probability Kinematics......Page 192
 8.5 Divergence and Probability Kinematics......Page 196
 8.6 Alternative Approaches......Page 199
 8.7 Rational Disagreement......Page 200
 A.1 The Johnson–Carnap Continuum of Inductive Methods......Page 204
 A.2 De Finetti Representation......Page 206
 A.3 Bandit Problems......Page 207
 B.1 Partial Exchangeability......Page 211
 B.2 Representations of Partially Exchangeable Arrays......Page 212
 B.3 Average Reinforcement Learning......Page 213
 B.4 Regret Learning......Page 214
 C.1 Abstract Families......Page 216
 C.2 Marley’s Theorem......Page 217
 C.3 The Basic Model......Page 219
Bibliography......Page 221
Index......Page 235