Multidimensional Particle Swarm Optimization for Machine Learning and Pattern Recognition

Preface......Page 5
Abstract......Page 6
Contents......Page 8
Acronyms......Page 12
Tables......Page 15
Figures......Page 20
1 Introduction......Page 26
1.1…Optimization Era......Page 27
1.2…Key Issues......Page 29
1.3…Synopsis of the Book......Page 32
References......Page 35
2.1…History of Optimization......Page 37
2.2.1 Gradient Descent Method......Page 53
2.2.2 Newton--Raphson Method......Page 54
2.2.3 Nelder--Mead Search Method......Page 56
2.3.1 Simulated Annealing......Page 57
2.3.2 Stochastic Approximation......Page 59
2.4.1 Genetic Algorithms......Page 61
2.4.2 Differential Evolution......Page 65
References......Page 67
3.1…Introduction......Page 69
3.2…Basic PSO Algorithm......Page 70
3.3…Some PSO Variants......Page 73
3.3.1 Tribes......Page 75
3.3.2 Multiswarms......Page 77
3.4.1 Nonlinear Function Minimization......Page 79
3.4.2 Data Clustering......Page 81
3.4.3.1 An Overview......Page 85
3.4.3.2 BP versus PSO: Comparative Performance Evaluation Over Medical Datasets......Page 89
3.5…Programming Remarks and Software Packages......Page 98
References......Page 104
4.1…The Need for Multi-dimensionality......Page 107
4.2…The Basic Idea......Page 109
4.3…The MD PSO Algorithm......Page 111
4.4.1 MD PSO Operation in PSO_MDlib Application......Page 116
4.4.2 MD PSO Operation in PSOTestApp Application......Page 118
References......Page 123
5 Improving Global Convergence......Page 124
5.1.2 PSO with FGBF......Page 125
5.1.4 Nonlinear Function Minimization......Page 127
5.2.1 Dynamic Environments: The Test Bed......Page 139
5.2.2 Multiswarm PSO......Page 140
5.2.3 FGBF for the Moving Peak Benchmark for MPB......Page 141
5.2.4 Optimization over Multidimensional MPB......Page 142
5.2.5 Performance Evaluation on Conventional MPB......Page 143
5.2.6 Performance Evaluation on Multidimensional MPB......Page 147
5.3…Who Will Guide the Guide?......Page 151
5.3.1 SPSA Overview......Page 153
5.3.2.1 First SA-Driven PSO Approach: gbest Update by SPSA......Page 154
5.3.2.2 Second SA-driven PSO Approach: aGB Formation by SPSA......Page 156
5.3.3 Applications to Non-linear Function Minimization......Page 157
5.4…Summary and Conclusions......Page 164
5.5…Programming Remarks and Software Packages......Page 165
5.5.1 FGBF Operation in PSO_MDlib Application......Page 166
5.5.2 MD PSO with FGBF Application Over MPB......Page 167
References......Page 170
6 Dynamic Data Clustering......Page 173
6.1.1 The Theory......Page 174
6.1.2 Results on 2D Synthetic Datasets......Page 177
6.2.1 Motivation......Page 182
6.2.2 Fuzzy Model over HSV-HSL Color Domains......Page 185
6.2.3 DC Extraction Results......Page 186
6.2.3.1 Comparative Evaluations Against MPEG-7 DCD......Page 187
6.2.3.2 Robustness and Parameter Insensitivity......Page 189
6.2.3.3 Computational Complexity Analysis......Page 190
6.2.4 Summary and Conclusions......Page 192
6.3.1 SA-Driven MD PSO-Based Dynamic Clustering in 2D Datasets......Page 193
6.3.2 Summary and Conclusions......Page 196
6.4.1 FGBF Operation in 2D Clustering......Page 198
6.4.2 DC Extraction in PSOTestApp Application......Page 201
6.4.3 SA-DRIVEN Operation in PSOTestApp Application......Page 205
References......Page 207
7 Evolutionary Artificial Neural Networks......Page 209
7.1…Search for the Optimal Artificial Neural Networks: An Overview......Page 210
7.2.1 PSO for Artificial Neural Networks: The Early Attempts......Page 212
7.2.2 MD PSO-Based Evolutionary Neural Networks......Page 213
7.2.3 Classification Results on Synthetic Problems......Page 215
7.2.4 Classification Results on Medical Diagnosis Problems......Page 222
7.2.5 Parameter Sensitivity and Computational Complexity Analysis......Page 225
7.3…Evolutionary RBF Classifiers for Polarimetric SAR Images......Page 227
7.3.1 Polarimetric SAR Data Processing......Page 229
7.3.2 SAR Classification Framework......Page 231
7.3.3 Polarimetric SAR Classification Results......Page 233
7.4…Summary and Conclusions......Page 239
7.5…Programming Remarks and Software Packages......Page 240
References......Page 249
8 Personalized ECG Classification......Page 253
8.1.1 Introduction and Motivation......Page 255
8.1.2.2 Feature Extraction Methodology......Page 257
8.1.2.3 Preprocessing by Principal Component Analysis......Page 260
8.1.3.1 MD PSO Optimality Evaluation......Page 261
8.1.3.2 Classification Performance......Page 263
8.1.3.3 Robustness......Page 265
8.2…Classification of Holter Registers......Page 266
8.2.1 The Related Work......Page 267
8.2.2 Personalized Long-Term ECG Classification: A Systematic Approach......Page 268
8.2.3 Experimental Results......Page 272
8.3…Summary and Conclusions......Page 275
8.4…Programming Remarks and Software Packages......Page 277
References......Page 279
9 Image Classification and Retrieval by Collective Network of Binary Classifiers......Page 281
9.1…The Era of CBIR......Page 282
9.2…Content-Based Image Classification and Retrieval Framework......Page 284
9.2.1 Overview of the Framework......Page 285
9.2.2 Evolutionary Update in the Architecture Space......Page 286
9.2.3.1 The Topology......Page 287
9.2.3.2 Evolution of the CNBC......Page 289
9.2.3.3 Incremental Evolution of the CNBC......Page 291
9.3…Results and Discussions......Page 292
9.3.1 Database Creation and Feature Extraction......Page 293
9.3.2.1 Feature Scalability and Comparative Evaluations......Page 294
9.3.2.2 Incremental CNBC Evolutions......Page 296
9.3.2.3 CNBC Class Scalability......Page 298
9.3.3 CBIR Results......Page 299
9.4…Summary and Conclusions......Page 302
9.5…Programming Remarks and Software Packages......Page 303
References......Page 315
10.1…Introduction......Page 317
10.2…Feature Synthesis and Selection: An Overview......Page 319
10.3.1 Motivation......Page 321
10.3.2.2 Encoding of the MD PSO Particles......Page 323
10.3.2.3 The Fitness Function......Page 325
10.4…Simulation Results and Discussions......Page 328
10.4.1 Performance Evaluations with Respect to Discrimination and Classification......Page 329
10.4.2 Comparative Performance Evaluations on Content-Based Image Retrieval......Page 331
10.5…Programming Remarks and Software Packages......Page 336
References......Page 343