Optimal Bayesian classification

Contents......Page 7
Preface......Page 12
Acknowledgments......Page 16
 1.1 Classifiers......Page 17
 1.2 Constrained Classifiers......Page 21
 1.3 Error Estimation......Page 27
 1.4 Random Versus Separate Sampling......Page 32
 1.5 Epistemology and Validity......Page 34
  1.5.1 RMS bounds......Page 36
  1.5.2 Error RMS in the Gaussian model......Page 38
 2.1 The Bayesian MMSE Error Estimator......Page 41
 2.2 Evaluation of the Bayesian MMSE Error Estimator......Page 48
 2.3 Performance Evaluation at a Fixed Point......Page 49
 2.4 Discrete Model......Page 52
  2.4.1 Representation of the Bayesian MMSE error estimator......Page 53
  2.4.2 Performance and robustness in the discrete model......Page 54
 2.5 Gaussian Model......Page 62
  2.5.1 Independent covariance model......Page 63
  2.5.2 Homoscedastic covariance model......Page 66
  2.5.3 Effective class-conditional densities......Page 69
  2.5.4 Bayesian MMSE error estimator for linear classification......Page 77
  2.6.1 Fixed circular Gaussian distributions......Page 81
  2.6.2 Robustness to falsely assuming identity covariances......Page 84
  2.6.3 Robustness to falsely assuming Gaussianity......Page 85
  2.6.4 Average performance under proper priors......Page 91
 2.7 Consistency of Bayesian Error Estimation......Page 92
  2.7.1 Convergence of posteriors......Page 93
  2.7.2 Sufficient conditions for consistency......Page 95
  2.7.3 Discrete and Gaussian models......Page 98
  2.8.1 MMSE calibration function......Page 101
  2.8.2 Performance with LDA......Page 104
 2.9 Optimal Bayesian ROC-based Analysis......Page 107
  2.9.1 Bayesian MMSE FPR and TPR estimation......Page 108
  2.9.2 Bayesian MMSE ROC and AUC estimation......Page 109
  2.9.3 Performance study......Page 111
 3.1 Conditional MSE of Error Estimators......Page 117
 3.2 Evaluation of the Conditional MSE......Page 120
 3.3 Discrete Model......Page 122
  3.4.1 Effective joint class-conditional densities......Page 126
  3.4.2 Sample-conditioned MSE for linear classification......Page 133
  3.4.3 Closed-form expressions for functions I and R......Page 145
 3.5 Average Performance in the Gaussian Model......Page 151
 3.6 Convergence of the Sample-Conditioned MSE......Page 152
 3.7 A Performance Bound for the Discrete Model......Page 153
 3.8 Censored Sampling......Page 156
  3.8.1 Gaussian model......Page 157
 3.9 Asymptotic Approximation of the RMS......Page 159
  3.9.1 Bayesian–Kolmogorov asymptotic conditions......Page 161
  3.9.2 Conditional expectation......Page 163
  3.9.3 Unconditional expectation......Page 169
  3.9.4 Conditional second moments......Page 171
  3.9.5 Unconditional second moments......Page 177
  3.9.6 Unconditional MSE......Page 180
 4.1 Optimal Operator Design Under Uncertainty......Page 184
 4.2 Optimal Bayesian Classifier......Page 188
 4.3 Discrete Model......Page 189
  4.4.1 Both covariances known......Page 191
  4.4.2 Both covariances diagonal......Page 193
  4.4.3 Both covariances scaled identity or general......Page 194
  4.4.4 Mixed covariance models......Page 203
  4.4.5 Average performance in the Gaussian model......Page 205
 4.5 Transformations of the Feature Space......Page 210
 4.6 Convergence of the Optimal Bayesian Classifier......Page 211
  4.7.1 Falsely assuming homoscedastic covariances......Page 218
  4.7.2 Falsely assuming the variance of the features......Page 220
  4.7.3 Falsely assuming the mean of a class......Page 221
  4.7.4 Falsely assuming Gaussianity under Johnson distributions......Page 223
 4.8 Intrinsically Bayesian Robust Classifiers......Page 225
 4.9 Missing Values......Page 227
 4.10 Optimal Sampling......Page 234
  4.10.1 MOCU-based optimal experimental design......Page 235
  4.10.2 MOCU-based optimal sampling......Page 237
  4.11.1 Prior and posterior distributions for VAR processes......Page 240
  4.11.2 OBC for VAR processes......Page 243
5 Optimal Bayesian Risk-based Multi-class Classification......Page 251
 5.1 Bayes Decision Theory......Page 252
 5.2 Bayesian Risk Estimation......Page 253
 5.3 Optimal Bayesian Risk Classification......Page 254
 5.4 Sample-Conditioned MSE of Risk Estimation......Page 259
 5.5 Efficient Computation......Page 260
 5.6 Evaluation of Posterior Mixed Moments: Discrete Model......Page 262
  5.7.1 Known covariance......Page 264
  5.7.2 Homoscedastic general covariance......Page 265
  5.7.3 Independent general covariance......Page 269
 5.8 Simulations......Page 270
 6.1 Joint Prior Distribution......Page 275
 6.2 Posterior Distribution in the Target Domain......Page 279
 6.3 Optimal Bayesian Transfer Learning Classifier......Page 287
  6.3.1 OBC in the target domain......Page 290
 6.4 OBTLC with Negative Binomial Distribution......Page 295
7 Construction of Prior Distributions......Page 301
 7.1 Prior Construction Using Data from Discarded Features......Page 302
 7.2 Prior Knowledge from Stochastic Differential Equations......Page 309
  7.2.1 Binary classification of Gaussian processes......Page 310
  7.2.2 SDE prior knowledge in the BCGP model......Page 313
 7.3 Maximal Knowledge-Driven Information Prior......Page 317
  7.3.1 Conditional probabilistic constraints......Page 321
  7.3.2 Dirichlet prior distribution......Page 323
  7.4.1 Pathway knowledge......Page 326
  7.4.2 REMLP optimization......Page 328
  7.4.3 Application of a normal-Wishart prior......Page 329
  7.4.4 Incorporating regulation types......Page 332
  7.4.5 A synthetic example......Page 334
References......Page 339
Index......Page 355