Philosophy of Statistics

Philosophy of Statistics......Page 4
Copyright Page
......Page 5
Dedication......Page 6
General
Preface......Page 8
Preface......Page 10
Contributors......Page 14
Contents......Page 18
1 Philosophy, Statistics, and Philosophy of Statistics......Page 22
2 Four Statistical Paradigms......Page 24
3 The Likelihood Principle......Page 39
4 The Curve-Fitting Problem, Problem of Induction, and Role of Simplicity in Inference......Page 40
5 Recent Advances in Model Selection......Page 45
6 Causal Inference in Observational Studies......Page 46
7 Specific Topics of Interest......Page 49
8 An Application of Statistics to Climate Science......Page 63
9 Historical Topics in Probability and Statistics......Page 64
10 Plans Behind Arranging Major Sections/Chapters......Page 66
11
CODA......Page 67
Acknowledgements......Page 69
Bibliography......Page 70
Part I: Probability & Statistics......Page 72
1
Introduction......Page 74
3
From Probablity to Statistics and a Road-Map for the Rest of the Chapter......Page 83
4
Data Represented and Described......Page 85
5
Random Variables and Probability Distributions......Page 92
6
Statistical Inference......Page 103
7
Conclusion......Page 112
Bibliography......Page 115
Part II: Philosophical Controversies about Conditional
Probability......Page 118
1
Introduction......Page 120
2
Mathematical Theory......Page 121
3
Philosophical Applications......Page 126
4
Problems with the Ratio Analysis of Conditional Probability......Page 132
5
Kolmorogov's Refinement: Conditional Probability as a Random Variable......Page 136
6
Conditional Probability as Primitive......Page 139
7
Conditional Probabilities and Updating Rules......Page 142
8
Some Paradoxes and Puzzles Involving Conditional Probability and Conditionalization......Page 144
9
Probabilities of Conditionals and Conditional Probabilities......Page 148
Bibliography......Page 153
1
Pluralism About Conditional Probability......Page 158
2
Conditional Probability vs. Probability Given the Background......Page 166
Bibliography......Page 169
Part III: Four Paradigms of
Statistics......Page 170
Classical Statistics Paradigm......Page 172
1
What is Error Statistics?......Page 174
2
A Philosophy for Error Statistics......Page 186
3
Error Statistics vs. The Likelihood Principle......Page 204
4
Error Statistics is Self-Correcting: Testing Statistical Model Assumptions......Page 210
Bibliography......Page 216
1
Introduction......Page 220
2
The Development and Logic of Significance Tests......Page 221
3
(Mis)Interpreting Significance Tests......Page 236
4
Objections to Significance Tests......Page 240
5
Conclusion......Page 245
Bibliography......Page 246
Bayesian Paradigm......Page 252
1
Bayesianism in Inference and Decision......Page 254
2
Decision Problems......Page 257
3
Degrees of Belief and Desire......Page 262
4
Probability Axioms......Page 267
5
Conditionalization......Page 273
6
Bayesian and Classical Statistics Compared......Page 277
7
Appendix: Savage's Representation Theorem......Page 280
Bibliography......Page 281
1
Introduction......Page 284
2
Foundations......Page 286
3 The Bayesian Paradigm......Page 293
4
Reference Analysis......Page 304
5
Inference Summaries......Page 315
6
Discussion......Page 323
Bibliography......Page 326
2
Evidential Probability......Page 328
3
Second-Order Evidential Probability......Page 336
4
Objective Bayesian Epistemology......Page 343
5
EP-Calibrated Objective Bayesianism......Page 348
6
Conclusion......Page 350
Bibliography......Page 351
Introduction......Page 354
1
The Probabilistic Logic of Confirmation Functions......Page 357
2
Two Conceptions of Confirmational Probability......Page 364
3
The Logical Structure of Evidential Support and the Role of Bayes' Theorem in that Logic......Page 366
4
What is Confirmational Probability Anyway?......Page 384
5
The Likelihood Ratio Convergence Theorem......Page 392
6
When the Likelihoods are Vague AND/OR Diverse......Page 405
Bibliography......Page 408
1
Introduction......Page 412
2
Competing Accounts of the Nature of Inductive Inference......Page 415
3
Challenges to Framework Assumptions......Page 422
4
Additivity......Page 427
5
Bayesian Dynamics......Page 436
6
Further Challenges......Page 453
Acknowledgements......Page 456
Bibliography......Page 457
1
Introduction......Page 462
2
Bolzano and Partial Entailment......Page 465
3
Symmetry and its Problems......Page 468
4
Carnap's Logical Probability......Page 474
5
From Logical Probability to Probabilistic Logic......Page 480
6
Conditional Probability......Page 485
7
Countable Versus Finite Additivity......Page 487
Bibliography......Page 490
1
Bayesian Inductive Logic and Bayesian Statistics......Page 494
2
Theory of Inductive Probabilities, Confirmation, and Statistics......Page 497
3
Verisimilitude and Statistics......Page 505
Bibliography......Page 508
Likelihood Paradigm......Page 512
1
Introduction......Page 514
2
The Likelihood Paradigm......Page 517
3
Reexamination of Accumulating Data ('multiple Looks')......Page 523
4 Measuring Evidence about Several Endpoints Simultaneously ('multiple
Comparisons')......Page 525
Bibliography......Page 531
1
Introduction......Page 534
2 Quantifying Evidence, Likelihood Ratios and Evidence
Functions......Page 538
4 Global & Local
Reliability......Page 541
5 Local Reliability and the Evidential
Paradigm......Page 542
6 Evidence and Composite
Hypotheses......Page 544
8 Evidence and the Challenges of
Multiplicities......Page 545
9
Discussion......Page 548
Bibliography......Page 550
Akaikean Paradigm......Page 554
1
Introduction......Page 556
2 Estimates as
Evidence......Page 558
3 Differences in AIC
Scores......Page 559
4
Conclusion......Page 562
Appendix......Page 563
Bibliography......Page 569
Part IV:
The Likelihood Principle......Page 572
1
Introduction......Page 574
2 Calculation, Justification and
Classification......Page 578
3
Terminology and Caveats......Page 579
4 The Likelihood Principle: Precise
Statement......Page 582
5 A Proof of the Likelihood
Principle......Page 584
6 A Proof of the Likelihood Principle: Continuation from the WSP and the
WCP......Page 588
7 Other Versions of the Likelihood
Principle......Page 592
8 Is the Likelihood Function Well
Defined?......Page 597
Bibliography......Page 599
Part V: Recent Advances in Model
Selection......Page 602
Overview......Page 604
1
Examples......Page 607
2 The Akaike Information Criterion
(AIC)......Page 610
3 Bayes Factor and
BIC......Page 616
4 Comparison of AIC and BIC Through an
Example......Page 620
5 Recent Advances in Model
Selection......Page 623
Bibliography......Page 624
1 Marginal
Likelihood......Page 628
2
Asymptotics......Page 629
3 Improper
Priors......Page 632
4 Subjective Model
Weights......Page 635
5 'Fully Objective' Formal
Priors......Page 636
6 Problems with the Jeffreys
Measure......Page 640
7
Examples......Page 644
Bibliography......Page 648
A Appendix: Proof of Theorem
1......Page 649
Part VI: Attempts to Understand
Different Aspects of "Randomness”......Page 652
Defining
Randomness......Page 654
Randomness Finally
Defined......Page 655
Inadequacies and
Deficiencies......Page 657
Commonalities......Page 658
Bibliography......Page 659
1 Introduction......Page 662
2 Strings, Sequences, Cantor Space, and Lebesgue
Measure......Page 670
3 Classical Stochastic Randomness in Infinite
Sequences......Page 674
4 Algorithms and Post
Machines......Page 683
5 Von Mises' Definition of Random
Sequence......Page 689
6 Martin-Löf and Solovay
Randomness......Page 694
7 Randomness of Finite Strings: Kolmogorov
Complexity......Page 696
8 The Prefix-Free Complexity
K......Page 704
9 Kolmogorov-Chaitin Randomness and Schnorr's
Theorem......Page 709
10 Relative and Stronger Randomness.
Hierarchies......Page 713
11 Randomness via Martingales.
Other Frequentist Definitions......Page 718
12 Conclusion. The Martin-Löf-Chaitin
Thesis......Page 726
Bibliography......Page 729
Part VII: Probabilistic and Statistical
Paradoxes......Page 732
1
Introduction......Page 734
2 Puzzles and
Paradoxes......Page 739
3 Simple Puzzles of
Probability......Page 741
4 More Complex Problems of
Probability......Page 745
5 Paradoxes of Probability and
Decision......Page 749
Acknowledgements......Page 755
Bibliography......Page 756
1
Introduction......Page 758
2 Frequentist vs.
Bayesian......Page 759
3 Lindley's
Paradox......Page 762
4 Fieller-Creasy
Problem......Page 766
5 Concluding
Remark......Page 767
Bibliography......Page 768
Part VIII: Statistics and Inductive
Inference......Page 770
1 Statistical Procedures as Inductive
Logics......Page 772
2 Observational
Data......Page 773
3 Inductive
Inference......Page 775
4 Neyman-Pearson
Testing......Page 777
5 Fisher's Parameter
Estimation......Page 780
6 Carnapian
Logics......Page 782
7 Bayesian
Statistics......Page 785
8 Bayesian Inductive
Logic......Page 788
9 Neyman-Pearson Test as an
Inference......Page 790
Acknowledgements......Page 793
Bibliography......Page 794
Part IX: Various Issues about Causal
Inference......Page 796
1
Introduction......Page 798
2 Structural Equation
Models......Page 801
3 Conditioning Versus
Manipulating......Page 804
4 Estimating Manipulated Means......Page 810
5 Open
Problems......Page 827
6 Appendix......Page 829
Bibliography......Page 830
Do we need Philosophy
of Causation for a Statistical Theory of Causal Inference?......Page 834
Potential Outcomes and Structural Equations......Page 835
Causal Systems and Causal Diagrams......Page 843
Discussion......Page 849
Bibliography......Page 850
Part X: Some Philosophical Issues Concerning Statistical Learning
Theory......Page 852
Statistical Learning Theory as a Framework for the Philosophy of
Induction......Page 854
Pattern
Recognition......Page 855
Bayes Error Rate R*......Page 856
Using Data to Learn the Statistical Probability
Distribution?......Page 857
Empirical Risk Minimization......Page 858
Data Coverage Balanced Against Something
Else......Page 861
Philosophical
Implications......Page 863
Bibliography......Page 867
Testability and Statistical Learning
Theory......Page 870
VC Dimension......Page 871
Bibliography......Page 881
Part XI: Different Approaches to Simplicity Related to Inference and
Truth......Page 884
1 Introduction......Page 886
2 Encoding the Hypothesis: A Matter of Luckiness and Regret......Page 890
3 Encoding the Data......Page 895
4 The Purpose of MDL......Page 905
5 MDL in Perspective......Page 912
Bibliography......Page 919
1 Introduction......Page 922
2 Information Theory — And Varieties Thereof......Page 924
3 Probabilistic Inference, Log-Loss Scoring and Kullback-Leibler Distance — And Uniqueness......Page 934
4 Ockham's Razor (And Misunderstandings) And MML......Page 949
5 Desiderata: Statistical Invariance, Statistical Consistency, Efficiency, Small-Sample Performance, Etc.......Page 954
6 Minimum Message Length (MML) and Strict MML......Page 957
7 MML and Some Applications in Philosophy and Elsewhere......Page 973
8 Acknowledgements......Page 990
Bibliography......Page 991
1 Introduction......Page 1004
2 The Argument from Bayes Factors......Page 1008
3 The Argument from Over-Fitting......Page 1012
4 Ockham's Causal Razor......Page 1018
5 Efficient Pursuit of the Truth......Page 1021
6 Empirical Simplicity Defined......Page 1023
7 Inquiry and Ockham's Razor......Page 1025
8 A Basic Ockham Efficiency Theorem......Page 1026
9 Stability, Errors and Retraction Times......Page 1029
10 Extension to Branching Simplicity......Page 1031
11 When Defeat does not Imply Refutation......Page 1033
12 Extension to Randomized Scientific Strategies......Page 1034
13 Disjunctive Beliefs, Retraction Degrees, and a Gettier Example......Page 1035
14 Extension to Degrees of Belief......Page 1037
15 Conclusion......Page 1042
Bibliography......Page 1043
Part XII: Special Problems in Statistics/Computer
Science......Page 1046
1 Introduction......Page 1048
2 The Central Limit Theorem......Page 1049
3 The Delta Method and Slutsky's Theorem......Page 1053
4 Discrete Distributions......Page 1055
5 Randomization Inference......Page 1060
6 Likelihood-Based Inference......Page 1067
7 Bayesian Posterior Distributions......Page 1071
8 Expansions......Page 1073
A Selected Mathematical and Statistical Concepts......Page 1079
B Heuristic Proofs of Selected Theorems......Page 1082
C Regularity Conditions......Page 1088
Bibliography......Page 1089
1 Introduction......Page 1094
2 Stein's Insight......Page 1095
3 A Data Analysis Example......Page 1098
4 A Geometric Heuristic......Page 1102
5 An Empirical Bayes Perspective......Page 1107
6 A Regression Perspective......Page 1109
7 Related Methodological Developments......Page 1111
8 Stein's Phenomenon and Modern Science......Page 1116
Bibliography......Page 1117
1 Oceans of Data......Page 1120
2 Knowledge Discovery from Data......Page 1121
3 Monkeys and Typewriters; Bangladeshi Butter and the S&P 500......Page 1122
4 Underfitting and Overfitting......Page 1123
5 Testing and Evaluation......Page 1127
6 Assumptions and Violations......Page 1130
7 The Case of Associations in Association Rules......Page 1133
8 Replacing Statisticians With Computers......Page 1135
Bibliography......Page 1137
Part XIII: An Application of Statistics to Climate
Change......Page 1140
1 Introduction......Page 1142
2 Previous Research......Page 1143
3
Data......Page 1147
4 Trend Detection Methods......Page 1149
5 Additive Mixed Models......Page 1153
6
Results......Page 1157
7 Conclusion......Page 1159
Bibliography......Page 1166
Part XIV: Historical Approaches to
Probability/Statistics......Page 1168
The Subjective and the Objective......Page 1170
1 The Era of Good Feelings......Page 1171
2 Liberté. Egalité,
Probabilité......Page 1172
3 Antoine Augustin Cournot......Page 1175
4 The Influence of Kant......Page 1178
5 1842 and 1843: The Anni Mirabili......Page 1180
6 De Morgan vs. Venn: "And there has not yet been
Philosophy Enough to Expel it ..."......Page 1181
7 Bertrand: Objectivity as Consensus......Page 1184
8 Poincaré:
The Method of Arbitrary Functions......Page 1189
9 Conclusion......Page 1190
Bibliography......Page 1192
Introduction: Mathematical Pre-Requisites......Page 1196
The Notion of a Fair Game and the Frequentist Interpretation of Probability......Page 1202
Subjective Probabilities and the Underlying Logic of Sentences......Page 1211
Index......Page 1218