Optimization Theory with Applications

Front cover......Page 1
Title page......Page 3
Date-line......Page 4
Preface to the Dover edition......Page 5
Preface to the first edition......Page 11
CONTENTS......Page 15
1-1 Optimization in Perspective......Page 23
1-2 The Concepts of System and State......Page 25
1-3 Performance Measures......Page 27
1-4 Constraints......Page 29
1-5 Optimization Problems......Page 30
1-6 Conditions for Optimality......Page 40
1-7 Approaches to Solution......Page 41
1-8 Forms of Solutions......Page 44
1-9 Sensitivity and Identification......Page 46
1-10 Discussion......Page 48
2-1 Introduction......Page 51
2-2 Basic Concepts and Notation......Page 52
2-3 Functions of One Variable......Page 53
2-4 Functions of Several Variables......Page 57
2-5 Equality Constraints and a Lagrange Multiplier......Page 58
2-6 General Case of Equality Constraints......Page 64
2-7 Inequality Constraints......Page 65
2-8 Extremization of Integrals......Page 67
2-9 Sensitivity Analysis......Page 69
2-10 Conclusion......Page 73
3-1 Introduction......Page 84
a. Continuity, Extrema, and Variations......Page 86
b. Classes of Problems and Equivalence Relations......Page 88
a. Problem Statement and the First Variation......Page 90
b. Fundamental Lemma......Page 92
c. First Necessary Condition and First-Variational Curves......Page 93
d. A Corner Condition......Page 94
e. The Euler-Lagrange Equation......Page 95
3-4 Isoperimetric Constraints......Page 97
3-5 Variable End-Point Conditions......Page 104
3-6 Corner Conditions......Page 109
3-7 The Problem of Lagrange: State-Vector Case......Page 112
a. Isoperimetric Constraints......Page 113
b. Constraints of the Form $g_t(\\vec{x},\\dot{\\vec{x}},t) = 0$......Page 114
c. Constraints of the Form $z_t(\\vec{x},t) = 0$......Page 116
d. Inequality Constraints......Page 117
3-9 A General Control Problem......Page 122
a. Discussion......Page 126
b. The Second Variation and the Legendre Condition......Page 127
c. Fields of Solutions......Page 129
d. The Jacobi Condition......Page 131
e. Sufficient Condition for Weak Extrema......Page 134
f. Green\'s Theorem......Page 136
g. The Weierstrass Condition and Strong Extrema......Page 137
a. Discussion......Page 140
b. Series Approximations......Page 141
c. Finite Differences......Page 143
3-12 Sensitivity Considerations......Page 145
3-13 Conclusion......Page 146
4-1 Introduction......Page 157
a. Description of the System......Page 158
b. Integral-Square-Error Problems......Page 159
c. Mean-Square-Error Problems......Page 162
a. Description of the System......Page 163
b. Maximum Peak Output......Page 165
b. Initial Steps to Solution......Page 166
c. Notation for Spectrum Factorization......Page 169
d. Solution Using Wiener-Hopf Spectrum Factorization......Page 171
4-5 Solutions to Filter, Control, and Predictor Problems......Page 172
4-6 Solutions to Optimal Pulse-Shape Problems......Page 184
b. Pulse Shape for Maximum Output Energy......Page 192
4-8 Sensitivity Considerations in Design......Page 195
4-9 Conclusion......Page 204
5-1 Introduction......Page 215
5-2 The General Problem and Its Standard Form......Page 216
5-3 Conversion to the Standard Form......Page 217
a. Prelude......Page 222
b. Convexity......Page 223
c. Extreme Point and Verticy Properties......Page 225
5-5 Simplex Algorithm Theory......Page 226
5-6 Simplex Algorithm Mechanics: The Simplex Tableau......Page 231
a. Avoiding Initial Degeneracy......Page 237
b. Generating an Initial Basic Feasible Solution......Page 239
c. Scaling......Page 240
5-8 Upper-Bounding Algorithm......Page 246
b. Symmetric Duals......Page 249
c. Other Duals......Page 251
5-10 Sensitivity Analysis......Page 254
a. Analogies......Page 256
b. Linear Programming on the General-Purpose Analog Computer......Page 257
a. Problems of Economics......Page 261
b. Control Problems......Page 262
c. Communications Problems......Page 266
d. Circuit Design Applications......Page 270
e. Field Problems......Page 273
f. Other Applications......Page 275
5-13 Conclusion......Page 276
6-1 Introduction......Page 286
a. Local Properties......Page 288
b. Regional Properties......Page 289
c. Scaling and Change of Variables......Page 292
d. Noise Considerations......Page 293
a. Newton-Raphson Search......Page 294
b. Cubic-Convergent Search without Second Derivatives......Page 296
c. Quadratic-Convergent Search without Derivatives......Page 299
d. Fibonacci Search......Page 302
e. Search by Golden Section......Page 306
f. One-Dimensional Search in $n$-Dimensional Space......Page 308
6-4 Nonsequential Methods......Page 309
b. Nonsequential Factorial Search......Page 310
b. Southwell\'s Relaxation Search......Page 314
c. Southwell-Synge Search......Page 315
a. Common Features......Page 318
b. Continuous Steepest Ascent (Descent)......Page 319
c. Discrete Steepest Ascent (Descent)......Page 321
d. Newton Search......Page 329
a. Two-Dimensional Case......Page 331
b. $n$-Dimensional Case: PARTAN......Page 334
a. Conjugate Directions......Page 336
b. Method of Fletcher and Reeves......Page 341
c. Davidoris Method via Fletcher and Powell (The DFP Method)......Page 342
b. Pattern Search......Page 344
c. Search by Directed Array......Page 346
e. Centroid Methods......Page 351
a. Equation Solution by Search......Page 353
b. Reduction of Dimensionality......Page 354
a. The Nonlinear Programming Problem......Page 355
b. Outside Penalty Functions for Inequality Constraints......Page 356
d. Minimization of the Penalized Performance Measure......Page 357
e. Inside Penalty Functions......Page 361
f. Equality Constraints and Classical Lagrange Multipliers......Page 363
g. General Constraints and Lagrange Multipliers......Page 364
6-12 Comparison of Techniques......Page 367
6-13 Conclusion......Page 372
7-1 Introduction......Page 389
a. Problem Statement and Applications......Page 391
b. Dynamic Programming Approach to Solution......Page 393
7-4 Redundancy to Improve Reliability......Page 400
a. Chain Networks......Page 402
b. Forward Solution I......Page 403
d. Backward Solution II......Page 405
e. Comparison of Forward and Backward Solutions......Page 407
a. Statement of the Problem......Page 408
b. Backward Solutions......Page 409
c. Forward Solutions......Page 412
7-7 Numerical Considerations......Page 416
7-8 A Principle of Optimality......Page 424
7-9 Placement of Transmission-Line Towers......Page 426
a. Problems and Difficulties......Page 430
b. Series Approximations......Page 433
c. Lagrange Multipliers......Page 434
d. Region-Limiting Strategies and Iterated Dynamic Programming......Page 436
a. A Control Problem......Page 440
b. An Approximation in Function Space......Page 441
c. An Approximation in Policy Space......Page 442
d. Nonoriented Minimal Chain Problems......Page 444
a. A General Control Problem......Page 445
b. Recurrence Relations with Prespecified Time Increments......Page 446
c. A Continuous Recurrence Relation......Page 448
d. Recurrence Relations with Controlled Time Increments......Page 450
a. The Problem and Its Forward Recurrence Relation......Page 453
b. Hamilton-Jacobi Equations......Page 455
c. Costate Equations......Page 456
e. Necessary Conditions: A Maximum Principle......Page 458
f. Necessary Conditions: Classical Calculus of Variations......Page 461
a. A General Case......Page 462
b. Steady-State Riccati Equations......Page 469
7-15 A Stochastic Control Problem......Page 472
a. Modal Trajectory Estimation......Page 474
b. Discrete Kalman-Bucy Filter......Page 479
7-17 Conclusion......Page 484
8-1 Introduction......Page 500
8-2 Preliminary Concepts......Page 501
8-3 A Canonical Problem Form and Equivalent Problems......Page 503
8-4 A Maximum Principle......Page 507
8-5 The Constancy of $\\mathcal{H}^\\ast$......Page 508
8-6 The General Transversality Condition......Page 515
a. Comments......Page 522
b. A Second-Order System......Page 524
c. Optimal Switch-Time Evaluation......Page 532
a. Comments......Page 535
b. Utilization of $\\mathcal{H}$ in a Search Solution......Page 537
c. A Newton-Raphson Algorithm for Linearization of Differential Equations and Solution of Two-Point Boundary-Value Problems......Page 539
d. Iterative Solutions with Stabilization via Riccati Equations......Page 541
e. A Riccati Transformation......Page 543
8-9 Non-Normal Solutions......Page 545
8-10 Singular Solutions......Page 547
a. An Equivalent Minimum Principle......Page 558
b. Necessary Conditions for End-Point Functionals......Page 559
8-12 Conclusion......Page 561
A. MATRIX IDENTITIES AND OPERATIONS......Page 577
B. TWO-SIDED LAPLACE TRANSFORM THEORY......Page 588
C. CORRELATION FUNCTIONS AND POWER-DENSITY SPECTRA......Page 594
D. INEQUALITIES AND ABSTRACT SPACES......Page 602
AUTHOR INDEX......Page 615
SUBJECT INDEX......Page 621
Back cover......Page 635