Optimization

Preface vii......Page f007.djvu
Contents xi......Page f011.djvu
1.2 Univariate Optimization 1......Page p001.djvu
1.3 Multivariate Optimization 7......Page p007.djvu
1.4 Constrained Optimization 10......Page p010.djvu
1.5 Problems 15......Page p015.djvu
2.2 Vector and Matrix Norms 19......Page p019.djvu
2.3 Convergence and Completeness 22......Page p022.djvu
2.4 The Topology of R^n 26......Page p026.djvu
2.5 Continuous Functions 30......Page p030.djvu
2.6 Connectedness 35......Page p035.djvu
2.7 Uniform Convergence 37......Page p037.djvu
2.8 Problems 38......Page p038.djvu
3.2 Univariate Derivatives 43......Page p043.djvu
3.3 The Gauge Integral 46......Page p046.djvu
3.4 Higher-Order Derivatives 47......Page p047.djvu
3.5 Partial Derivatives 50......Page p050.djvu
3.6 Differentials 52......Page p052.djvu
3.7 Multivariate Mean Value Theorem 56......Page p056.djvu
3.8 Inverse and Implicit Function Theorems 58......Page p058.djvu
3.9 Differentials of Matrix-Valued Functions 61......Page p061.djvu
3.10 Problems 65......Page p065.djvu
4.1 Introduction 69......Page p069.djvu
4.2 The Multiplier Rule 70......Page p070.djvu
4.3 Constraint Qualification 75......Page p075.djvu
4.4 Taylor-Made Second Differentials 78......Page p078.djvu
4.5 A Sufficient Condition for a Minimum 85......Page p085.djvu
4.6 Problems 88......Page p088.djvu
5.1 Introduction 93......Page p093.djvu
5.2 Convex Sets 94......Page p094.djvu
5.3 Convex Functions 95......Page p095.djvu
5.4 Minimization of Convex Functions 104......Page p104.djvu
5.5 Moment Inequalities 109......Page p109.djvu
5.6 Problems 112......Page p112.djvu
6.1 Introduction 119......Page p119.djvu
6.2 Philosophy of the MM Algorithm 120......Page p120.djvu
6.3 Majorization and Minorization 121......Page p121.djvu
6.4 Allele Frequency Estimation 123......Page p123.djvu
6.5 Linear Regression 125......Page p125.djvu
6.7 Linear Logistic Regression 127......Page p127.djvu
6.8 Poisson Processes 128......Page p128.djvu
6.9 Transmission Tomography 129......Page p129.djvu
6.10 Problems 133......Page p133.djvu
7.1 Introduction 137......Page p137.djvu
7.2 Definition of the EM Algorithm 138......Page p138.djvu
7.3 Allele Frequency Estimation 140......Page p140.djvu
7.4 Transmission Tomography 141......Page p141.djvu
7.5 Factor Analysis 143......Page p143.djvu
7.6 Hidden Markov Chains 147......Page p147.djvu
7.7 Problems 149......Page p149.djvu
8.1 Introduction 155......Page p155.djvu
8.2 Newton\'s Method and Root Finding 156......Page p156.djvu
8.3 Newton\'s Method and Optimization 158......Page p158.djvu
8.4 MM Gradient Algorithm 160......Page p160.djvu
8.5 Ad Hoc Approximations of d^2f(theta) 161......Page p161.djvu
8.6 Scoring 164......Page p164.djvu
8.7 Problems 168......Page p168.djvu
9.1 Introduction 175......Page p175.djvu
9.2 Centers of Spheres and Centers of Ellipsoids 176......Page p176.djvu
9.3 The Conjugate Gradient Algorithm 177......Page p177.djvu
9.4 Line Search Methods 180......Page p180.djvu
9.5 Quasi-Newton Methods 182......Page p182.djvu
9.6 Trust Regions 186......Page p186.djvu
9.7 Problems 187......Page p187.djvu
10.1 Introduction 191......Page p191.djvu
10.2 Local Convergence 192......Page p192.djvu
10.3 Global Convergence of the MM Algorithm 196......Page p196.djvu
10.4 Global Convergence of Gradient Algorithms 200......Page p200.djvu
10.5 Problems 202......Page p202.djvu
11.1 Introduction 207......Page p207.djvu
11.2 Adaptive Barrier Methods 208......Page p208.djvu
11.3 Dykstra\'s Algorithm 216......Page p216.djvu
11.4 Dual Programs 219......Page p219.djvu
11.5 Linear Classification 223......Page p223.djvu
11.6 Support Vector Machines 226......Page p226.djvu
11.7 Problems 228......Page p228.djvu
A.1 Univariate Normal Random Variables 233......Page p233.djvu
A.2 Multivariate Normal Random Vectors 235......Page p235.djvu
References 237......Page p237.djvu
Index 247......Page p247.djvu