Optimization by Vector Space Methods

PREFACE......Page 9
CONTENTS......Page 11
NOTATION......Page 17
1.1 Motivation......Page 21
1.2 Applications......Page 22
1.3 The Main PrInciples......Page 28
1.4 Organization of the Book......Page 30
2.2 Definition and Examples......Page 31
2.3 Subspaces, Linear Combinations, and Linear Varieties......Page 34
2.4 Convexity and Cones......Page 37
2.5 Linear Independence and Dimension......Page 39
2.6 Definition and Examples......Page 42
2.7 Open and Closed Sets......Page 44
2.8 Convergence......Page 46
2.9 Transformations and Continuity......Page 47
*2.10 The lp and Lp Spaces......Page 49
2.11 Banach Spaces......Page 53
2.12 Complete Subsets......Page 58
*2.13 Extreme Values of Functionals and Compactness......Page 59
*2.14 Quotient SpacE~s......Page 61
*2.15 Denseness and Separability......Page 62
2.16 Problems......Page 63
REFERENCES......Page 65
3.2 Inner Products......Page 66
3.3 The Projection Theorem......Page 69
3.4 Orthogonal Complements......Page 72
3.5 The Gram-Schmidt Procedure......Page 73
3.6 The Normal lr.quations and Gram Matrices......Page 75
3.7 Fourier Series......Page 78
*3.8 Complete Orthonormal Sequences......Page 80
3.9 Approximation and Fourier Series......Page 82
3.10 The Dual Approximation Problem......Page 84
*3.11 A Control Problem......Page 88
3.12 Minimum Distance to a Convex Set......Page 89
3.13 Problems......Page 92
REFERENCES......Page 97
4.1 Introduction......Page 98
4.2 Hilbert Space of R.andom Variables......Page 99
4.3 The Least.Squares Estimate......Page 102
4.4 Minimum-Variance Unbiased Estimate (Gauss-Markov Estimate)......Page 104
4.5 Minimum-Varianc:e Estimate......Page 107
4.6 Additional Properties of Minimum-Variance Estimates......Page 110
4.7 Recursive Estimation......Page 113
4.8 Problems......Page 117
REFERENCES......Page 122
5.1 Introduction......Page 123
5.2 Basic Concepts......Page 124
5.3 Duals of Some Common Banach Spaces......Page 126
5.4 Extension of Linear Functionals......Page 130
5.5 The Dual of C[a, b]......Page 133
5.6 The Second Dual Space......Page 135
5.7 Alignment and Orthogonal Complements......Page 136
5.8 Minimum Norm Problems......Page 138
5.9 Applications......Page 142
*5.10 Weak Convergence......Page 146
S.11 Hyperplanes and Linear Functionals......Page 149
5.12 Hyperplanes and Convex Sets......Page 151
*5.13 Duality in Minimum Norm Problems......Page 154
5.14 Problems......Page 157
REFERENCES......Page 162
6.2 Fundamentals......Page 163
6.3 Linearity of Invers,!s......Page 167
6.4 The Banach Inverse Theorem......Page 168
6.5 Definition and Examples......Page 170
6.6 Relations Between Range and Nullspace......Page 175
6.7 Duality Relations for Convex Cones......Page 177
*6.8 Geometric Inlterpretation of Adjoints......Page 179
6.9 The Normal Equations......Page 180
6.10 The Dual Problem......Page 181
6.11 Pseudoinverse Operators......Page 183
6.12 Problems......Page 185
REFERENCES......Page 188
7.1 Introduction......Page 189
7.2 Gateaux and Fll\'echet Differentials......Page 191
7.3 Frechet Derivatives......Page 195
7.4 Extrema......Page 197
*7.5 Euler-Lagrange Equations......Page 199
*7.6 Problems with Variable End Points......Page 203
7.7 Problems with Constraints......Page 205
7.8 Convex and Concave Functionals......Page 210
*7.9 Properties of the Set [f, C]......Page 212
7.10 Conjugate Convex Functionals......Page 215
7.11 Conjugate Concave Functionals......Page 219
7.12 Dual Optimization J\'lroblems......Page 220
*7.13 Min-Max Theorem of Game Theory......Page 226
7.14 Problems......Page 229
REFERENCES......Page 232
8.1 Introduction......Page 233
8.2 Positive Cones and Convex Mappings......Page 234
8.3 Lagrange Multipliers......Page 236
8.4 Sufficiency......Page 239
8.5 Sensitivity......Page 241
8.6 Duality......Page 243
8.7 Applications......Page 246
8.8 Problems......Page 254
REFERENCES......Page 256
9.1 Introduction......Page 257
9.2 Inverse hnction Theorem......Page 258
9.3 Equality Constraints......Page 260
9.4 Inequality Constraints (Kuhn-Tucker Theorem)......Page 265
9.5 Basic Necessary Conditions......Page 272
*9.6 The Pontryagin Ma);imnm Principle......Page 279
9.7 Problems......Page 284
REFERENCES......Page 287
10.1 Introduction......Page 289
10.2 Successive Approximation......Page 290
10.3 Newton\'s Method......Page 295
10.4 General Philosophy......Page 301
10.5 Steepest Descent......Page 303
10.6 Fourier Series......Page 308
*10.7 Ortbogonalization of Moments......Page 311
10.8 The Conjugate Gradient Method......Page 312
10.9 Projection Methods......Page 315
10.10 The Primal-Dual Method......Page 317
10.11 Penalty Functions......Page 320
10.12 Problems......Page 326
REFERENCES......Page 329
BIBLIOGRAPBY......Page 330
SYMBOL INDEX......Page 337
SUBJECT I]~DEX......Page 339