Computational intelligence paradigms : theory & applications using MATLAB

143980902X......Page 1
COMPUTATIONAL INTELLIGENCE PARADIGMS: THEORY AND APPLICATIONS USING MATLAB®......Page 3
Contents......Page 5
About This Book......Page 17
Organization of the Book......Page 18
About the Authors......Page 21
Acknowledgment......Page 22
1.1 Introduction......Page 24
1.2.1 Optimization......Page 28
Verifier-Based Definition......Page 29
1.3 Neural Networks......Page 30
1.3.1 Feed Forward Neural Networks......Page 31
1.4.1 Fuzzy Sets......Page 32
1.4.2 Fuzzy Controllers......Page 34
1.5 Evolutionary Computing......Page 35
1.5.1 Genetic Algorithms......Page 36
1.5.2 Genetic Programming......Page 37
1.5.4 Evolutionary Strategies......Page 38
1.6 Swarm Intelligence......Page 39
1.7 Other Paradigms......Page 40
1.7.1 Granular Computing......Page 41
1.7.2 Chaos Theory......Page 43
1.7.3 Artificial Immune Systems......Page 44
1.8 Hybrid Approaches......Page 45
1.9 Relationship with Other Paradigms......Page 46
1.10 Challenges To Computational Intelligence......Page 48
Summary......Page 49
Review Questions......Page 50
2.2 A Brief History of Neural Networks......Page 52
2.3 Artificial Neural Networks......Page 55
2.3.2 Artificial Neurons......Page 56
2.3.3 Implementation of Artificial Neuron Electronically......Page 58
2.3.4 Operations of Artificial Neural Network......Page 60
2.3.5 Training an Artificial Neural Network......Page 63
Supervised Training......Page 64
Unsupervised or Adaptive Training......Page 65
2.3.6 Comparison between Neural Networks, Traditional Computing, and Expert Systems......Page 67
2.4 Neural Network Components......Page 69
Supervised Learning......Page 75
Unsupervised Learning......Page 76
2.4.2 Learning Rates......Page 77
Hebb’s Rule......Page 78
Delta Rule for Single Output Unit......Page 79
Extended Delta Rule......Page 80
Competitive Learning Law or Kohonen’s Learning Law......Page 81
Boltzmann Learning Law......Page 83
MATLAB Snippet to Implement Hebb Rule Syntax......Page 84
Review Questions......Page 90
3.1 Introduction......Page 92
3.2.1 Single-Layer Networks......Page 94
3.2.2 Multilayer Networks......Page 96
Architecture......Page 98
Algorithm......Page 99
Training Algorithm......Page 100
Perceptron Training Algorithm for Several Output Classes......Page 101
3.3.2 MATLAB Implementation of a Perceptron Network......Page 102
Architecture......Page 106
Training Algorithm......Page 108
Parameters......Page 109
Choice of Parameters......Page 111
Number of Training Pairs......Page 112
Momentum Factor......Page 113
3.3.4 Implementation of BPN Using MATLAB......Page 114
Limitations of Back Propagation Network......Page 117
3.3.5 Delta Bar Delta Network......Page 118
Learning Rule of the Delta Bar Delta Network......Page 119
3.3.6 Extended Delta Bar Delta......Page 120
3.3.7 Directed Random Search Network......Page 123
3.3.8 Functional Link Artificial Neural Network (FLANN) or Higher-Order Neural Network......Page 125
Architecture......Page 126
Review Questions......Page 129
4.2 Neural Networks Based on Classification......Page 131
4.2.1 Learning Vector Quantization......Page 132
Architecture......Page 134
Training Algorithm......Page 135
LVQ3......Page 136
4.2.2 Implementation of LVQ in MATLAB......Page 137
4.2.3 Counter-Propagation Network......Page 138
Training Phases of Full CPN......Page 141
Training Algorithm......Page 142
Initializing the Weight Vectors......Page 144
4.2.4 Probabilistic Neural Network......Page 145
Training Algorithm......Page 148
Application Algorithm......Page 149
4.2.5 Implementation of the Probabilistic Neural Net Using MATLAB......Page 150
4.3 Data Association Networks......Page 151
4.3.1 Hopfield Network......Page 152
Training Algorithm......Page 153
Application Algorithm......Page 154
4.3.2 Implementation of Hopfield Network in MATLAB......Page 155
4.3.3 Boltzmann Machine......Page 156
Architecture......Page 157
Application Algorithm......Page 158
4.3.4 Hamming Network......Page 159
Architecture......Page 161
Application Procedure......Page 162
FIGURE 4.11: Bi-Directional Associative Memory Example......Page 163
Architecture......Page 164
Activation Function......Page 165
Continuous BAM......Page 166
Application Algorithm......Page 167
Background......Page 168
ART1: The Simplified Neural Network Model......Page 169
Supplemental Units......Page 170
Training Algorithm......Page 171
4.4.2 Implementation of ART Algorithm in MATLAB......Page 173
4.4.3 Self-Organizing Map......Page 175
Architecture......Page 178
Training Algorithm......Page 179
4.5 Applications Areas of ANN......Page 180
4.5.1 Language Processing......Page 181
4.5.4 Pattern Recognition......Page 182
4.5.6 Financial......Page 183
Review Questions......Page 184
5.1 Illustration 1: Coin Detection Using Euclidean Distance (Hamming Net)......Page 186
Observations......Page 190
5.2 Illustration 2: Learning Vector Quantization - Clustering Data Drawn from Different Regions......Page 192
5.3 Illustration 3: Character Recognition Using Kohonen Som Network......Page 195
Observations......Page 202
5.4 Illustration 4: The Hopfield Network as an Associative Memory......Page 203
Observations......Page 206
5.5 Illustration 5: Generalized Delta Learning Rule and Back-Propagation of Errors for a Multilayer Network......Page 208
Observations......Page 209
5.6 Illustration 6: Classification of Heart Disease Using Learning Vector Quantization......Page 210
5.7 Illustration 7: Neural Network Using MATLAB Simulink......Page 219
Observations......Page 220
Review Questions......Page 221
6.1 Introduction......Page 223
6.3 Crisp and Fuzzy Logic......Page 225
6.4 Fuzzy Sets......Page 227
6.5 Universe......Page 229
6.6 Membership Functions......Page 230
Triangular Function......Page 232
Gaussian Function......Page 233
6.6.2 Membership Functions in the MATLAB Fuzzy Logic Toolbox......Page 234
Trapezoidal Membership Function......Page 237
Gaussian Membership Function Syntax......Page 238
Generalized Bell Shaped Membership Function......Page 239
Sigmoidal Shaped Membership Function......Page 240
Z Curve Membership Function......Page 241
S Curve Membership Function......Page 242
6.6.4 Translation of Parameters between Membership Functions Using MATLAB......Page 243
6.7 Singletons......Page 244
6.9 Operations on Fuzzy Sets......Page 245
6.9.1 Fuzzy Complements......Page 247
6.9.2 Fuzzy Intersections: t-norms......Page 250
6.9.3 Fuzzy Unions: t-conorms......Page 253
6.9.4 Combinations of Operations......Page 255
Complement......Page 256
Union......Page 257
Intersection......Page 258
6.9.6 Aggregation Operations......Page 259
6.10 Fuzzy Arithmetic......Page 262
6.10.1 Arithmetic Operations on Intervals......Page 263
6.10.2 Arithmetic Operations on Fuzzy Numbers......Page 264
Illustration......Page 266
6.11 Fuzzy Relations......Page 267
6.12 Fuzzy Composition......Page 271
6.12.1 MATLAB Code to Implement Fuzzy Composition......Page 274
Review Questions......Page 280
7.2 Fuzzy Rules......Page 281
7.2.2 Disintegration of Rules......Page 282
7.2.3 Aggregation of Rules......Page 283
7.3.1 Fuzzification......Page 284
7.3.2 Fuzzy Rule Base and Fuzzy IF-THEN Rules......Page 285
7.3.3 Fuzzy Inference Machine......Page 286
Max-Membership principle......Page 287
Center of gravity method (COG)......Page 288
Weighted Average method......Page 289
Center of sums......Page 290
Center of largest area......Page 291
First or Last of maxima......Page 292
7.3.5 Implementation of Defuzzification using MATLAB Fuzzy Logic Toolbox......Page 293
Centroid method......Page 294
Mean of Maximum methods......Page 295
7.4 Fuzzy Inference Methods......Page 296
7.4.1 Mamdani’s Fuzzy Inference Method......Page 298
7.4.2 Takagi–Sugeno Fuzzy Inference Method......Page 301
7.4.3 Tsukamoto Fuzzy Inference Method......Page 302
7.5 Fuzzy Inference Systems in MATLAB......Page 303
7.5.1 Mamdani-Type Fuzzy Inference......Page 306
Application of the Fuzzy Operator......Page 308
Application of Implication......Page 309
Defuzzify......Page 310
Advantages of the Mamdani Method......Page 311
7.5.2 Sugeno-Type Fuzzy Inference......Page 312
7.5.3 Conversion of Mamdani to Sugeno System......Page 315
7.6 Fuzzy Automata and Languages......Page 317
7.7 Fuzzy Control......Page 318
7.7.1 Fuzzy Controllers......Page 320
7.7.2 A Fuzzy Controller in MATLAB......Page 322
Review Questions......Page 325
8.1 Illustration 1: Application of Fuzzy Controller Using MATLAB — Fuzzy Washing Machine......Page 327
8.2 Illustration 2 - Fuzzy Control System for a Tanker Ship......Page 337
8.3 Illustration 3 - Approximation of Any Function Using Fuzzy Logic......Page 356
8.4 Illustration 4 - Building Fuzzy Simulink Models......Page 363
Review Questions......Page 368
9.1 Introduction......Page 370
9.3 Fused Neuro-Fuzzy Systems......Page 371
9.3.2 Adaptive Neuro-Fuzzy Inference System (ANFIS)......Page 372
9.3.3 Generalized Approximate Reasoning-Based Intelligent Control (GARIC)......Page 374
NEFCON Learning Algorithm......Page 375
Decremental Rule Learning......Page 376
Incremental Rule Learning......Page 377
9.3.5 Fuzzy Inference and Neural Network in Fuzzy Inference Software (FINEST)......Page 379
9.3.6 Fuzzy Net (FUN)......Page 381
9.3.7 Evolving Fuzzy Neural Network (EFuNN)......Page 382
9.3.9 Evolutionary Design of Neuro-Fuzzy Systems......Page 383
9.4.1 Architecture of Adaptive Neuro-Fuzzy Inference System......Page 386
Layer 2 - Product Layer......Page 387
Layer 5 - Total Output Layer......Page 388
Forward Pass: Least Squares Estimate Method......Page 389
Backward Pass: Steepest Descent Method......Page 390
9.5.1 CART — Introduction......Page 391
9.5.2 Node Splitting Criteria......Page 392
9.5.3 Classification Trees......Page 393
9.5.5 Computational Issues of CART......Page 394
9.5.6 Computational Steps......Page 395
9.5.7 Accuracy Estimation in CART......Page 398
9.5.8 Advantages of Classification and Regression Trees......Page 399
9.6.1 System Identification Using Fuzzy Clustering......Page 401
9.6.2 Hard C-Means Clustering......Page 402
9.6.3 Fuzzy C-Means (FCM) Clustering......Page 405
Algorithm......Page 406
9.6.4 Subtractive Clustering......Page 407
9.6.5 Experiments......Page 408
Review Questions......Page 412
10.1 Illustration 1 - Fuzzy Art Map......Page 414
10.2 Illustration 2: Fuzzy C-Means Clustering - Comparative Case Study......Page 421
10.3 Illustration 3 - Kmeans Clustering......Page 422
10.4 Illustration 4 - Neuro-Fuzzy System Using Simulink......Page 430
10.5 Illustration 5 - Neuro-Fuzzy System Using Takagi–Sugeno and ANFIS GUI of MATLAB......Page 433
Summary......Page 436
Review Questions......Page 437
11.1 Introduction......Page 438
11.2 Evolutionary Computation......Page 439
11.3 Brief History of Evolutionary Computation......Page 441
11.4.2 Biological Evolution Inspired by Nature......Page 442
11.4.3 Evolutionary Biology......Page 444
11.5 Flow Diagram of a Typical Evolutionary Algorithm......Page 447
11.6.1 Genetic Algorithms (GA)......Page 449
11.6.2 Genetic Programming (GP)......Page 450
11.6.3 Evolutionary Programming (EP)......Page 454
11.6.4 Evolutionary Strategies (ESs)......Page 455
Randomness......Page 457
11.7 Evolutionary Algorithms......Page 458
11.7.2 Solution Representation......Page 461
11.7.3 Fitness Function......Page 462
11.7.5 Selection Mechanisms......Page 463
Illustration......Page 464
Local Selection......Page 466
Truncation Selection......Page 468
Tournament Selection......Page 469
11.7.6 Crossover Technique......Page 470
Discrete Recombination......Page 471
Arithmetic Crossover......Page 472
Crossover Probability......Page 473
Real Valued Mutation......Page 474
Boundary......Page 475
Mutation Probability......Page 476
11.7.8 Reproduction Operator......Page 477
11.8.1 History......Page 478
11.8.2 Procedure of Evolutionary Programming......Page 479
Choosing a Solution’s Coding Scheme......Page 480
Extending the Coding Scheme......Page 484
Generating an Initial Population......Page 485
The Current Population......Page 488
Offspring Generation in Evolutionary Programming Methods......Page 489
Merging the Populations......Page 494
Choosing a Current Population......Page 496
11.9 Evolutionary Strategies......Page 497
11.9.2 Mutation......Page 499
Discrete Recombination......Page 500
11.9.4 Population Assessment......Page 501
11.9.6 Computational Considerations......Page 502
11.9.7 Algorithm Performance......Page 503
11.10 Advantages and Disadvantages of Evolutionary Computation......Page 504
Summary......Page 506
Review Questions......Page 507
12.1 Illustration 1: Differential Evolution Optimizer......Page 509
12.2 Illustration 2: Design of a Proportional-Derivative Controller Using Evolutionary Al-gorithm for Tanker Ship Heading Regulation......Page 520
12.3 Illustration 3: Maximizing the Given One-Dimensional Function with the Boundaries Using Evolutionary Algorithm......Page 541
12.4 Illustration 4: Multiobjective Optimization Using Evolutionary Algorithm (MOEA)......Page 544
12.5 Illustration 5: Evolutionary Strategy for Nonlinear Function Minimization......Page 559
Review Questions......Page 563
13.1 Introduction......Page 565
13.2 Encoding and Optimization Problems......Page 566
13.3 Historical Overview of Genetic Algorithm......Page 567
13.4 Genetic Algorithm Description......Page 568
13.5 Role of Genetic Algorithms......Page 570
13.6 Solution Representation of Genetic Algorithms......Page 571
Grefenstette Settings......Page 572
13.8 Schema Theorem and Theoretical Background......Page 573
13.8.1 Building Block Hypothesis......Page 574
13.8.2 The Dynamics of a Schema......Page 575
13.8.3 Illustrations Based on Schema Theorem......Page 577
13.9.2 2-Point Crossover......Page 581
13.9.3 Linkage and Defining Length......Page 582
13.9.4 Linkage and Inversion......Page 583
13.10 Genotype and Fitness......Page 584
13.11.2 Epistasis......Page 585
13.11.3 Deception......Page 586
13.11.7 Diploidy and Dominance......Page 587
13.12.1 Neural Nets......Page 588
13.12.4 Iterated Search......Page 589
13.13 Benefits of GA......Page 590
13.14.1 Illustration 1: Maximizing the Given One-Dimensional Function within Given Boundaries......Page 591
13.14.2 Illustration 2: Solving Economic Dispatch Problem Using Genetic Algorithm......Page 594
13.14.3 Illustration 3: Traveling Salesman Problem......Page 598
Summary......Page 605
Review Questions......Page 606
14.1 Introduction......Page 608
14.2 Growth of Genetic Programming......Page 611
14.3 The Lisp Programming Language......Page 612
14.4.1 Generation of an Individual and Population......Page 613
14.4.2 Creating a Random Population......Page 614
14.4.5 The Genetic Operations......Page 616
14.4.6 Selection Functions......Page 617
14.4.7 MATLAB Routine for Selection......Page 618
14.4.8 Crossover Operation......Page 621
14.4.9 MATLAB Routine for Crossover......Page 624
14.4.11 MATLAB Routine for Mutation......Page 625
Connectionist Model......Page 627
Inductive Model......Page 628
14.6 Elementary Steps of Genetic Programming......Page 629
14.6.3 The Fitness Function......Page 630
14.6.5 The Termination Criterion......Page 631
14.7 Flowchart of Genetic Programming......Page 632
14.9.1 Illustration 1: Static Function Identification......Page 635
14.9.2 Illustration 2: Dynamical Input-Output Model Identification......Page 646
14.9.3 Illustration 3 - Symbolic Regression Problem Using Genetic Programming Toolbox......Page 649
Summary......Page 662
Review Questions......Page 663
15.1 Introduction to Swarms......Page 665
15.2 Biological Background......Page 666
15.3 Swarm Robots......Page 668
15.4 Stability of Swarms......Page 669
15.5 Swarm Intelligence......Page 670
15.5.1 Properties of a Swarm Intelligence System......Page 671
15.6 Particle Swarm Optimization (PSO)......Page 672
15.6.1 Mathematical Model of PSO......Page 673
15.6.3 Parameters and Tuning of Parameters in PSO......Page 674
15.6.4 Neighborhood Topologies......Page 675
Von Neumann Topology......Page 676
15.7.2 PSO-II......Page 677
15.7.3 PSO-III......Page 678
15.8 Ant Colony Optimization......Page 679
15.8.2 Ant Colony Optimization Algorithm......Page 680
15.9.2 Clustering Behavior of Ants......Page 681
15.9.4 Machine Scheduling......Page 682
15.9.5 Quadratic Knapsack Problem......Page 683
15.10 MATLAB Examples of Swarm Intelligence......Page 684
15.10.1 Illustration 1: Simulation of the Movement of Swarm to Minimize the Objective Function......Page 685
15.10.2 Illustration 2: Behavior of Particle Swarm Optimization......Page 687
15.10.3 Illustration 3: Ant Colony Optimization to Determine the Shortest Path......Page 700
15.10.4 Illustration 4: Ant Algorithm for the Quadratic Assignment Problem (QAP)......Page 703
Review Questions......Page 707
A......Page 709
B......Page 711
C......Page 713
D......Page 718
E......Page 719
F......Page 723
G......Page 726
H......Page 729
I......Page 731
L......Page 733
M......Page 735
N......Page 737
O......Page 739
P......Page 741
R......Page 743
S......Page 744
T......Page 748
V......Page 750
X......Page 751
Appendix B: List of Abbreviations......Page 752
C.1 Genetic Algorithm Toolbox for MATLAB......Page 755
Crossover Operators......Page 756
C.2 Fuzzy Logic Toolbox 2.2.7......Page 757
Membership Functions......Page 758
Working in Simulink Environment......Page 759
C.3 Neural Network Toolbox 6.0......Page 760
Distance Functions......Page 761
Line Search Functions......Page 762
New Networks Functions......Page 763
Plotting Functions......Page 764
Training Functions......Page 765
Transfer Functions......Page 766
Weight and Bias Initialization Functions......Page 767
C.4 Genetic Algorithm and Direct Search Toolbox 2.2......Page 768
Simulated Annealing......Page 769
C.5 GPLAB - A Genetic Programming Toolbox for MATLAB......Page 770
Running the Algorithm and Testing Result......Page 771
Description of Parameter and State Variables......Page 772
Filtering of New Individuals......Page 773
Artificial Ant Functions......Page 774
Expected Number of Children......Page 775
Fitness......Page 776
Automatic Operator Probability Adaptation......Page 777
Utilitarian Functions......Page 778
D.2 ComputerAnts......Page 779
D.5 Evolution Machine......Page 780
D.7 GAC, GAL......Page 781
D.9 GAGS......Page 782
D.11 GALOPPS......Page 783
D.12 GAMusic......Page 784
D.15 GECO......Page 785
D.17 GENEsYs......Page 786
D.20 Genitor......Page 787
D.23 GPEIST......Page 788
D.25 JAG......Page 789
D.28 PGA......Page 790
D.29 PGAPack......Page 791
D.31 Splicer......Page 792
D.33 WOLF......Page 793
D.35 XFUZZY: A Simulation Environment for Fuzzy Logic Control Systems......Page 794
D.37 COMDALE/C, COMDALE/X, and ProcessVision......Page 795
E.1 An Evolutionary Algorithm for Global Optimization Based on Level-Set Evolution and Latin Squares......Page 797
E.3 Analog Genetic Encoding for the Evolution of Circuits and Networks......Page 798
E.5 Solving the Register Allocation Problem for Embedded Systems Using a Hybrid Evolutionary Algorithm......Page 799
E.6 Semantic Understanding of General Linguistic Items by Means of Fuzzy Set Theory......Page 800
E.8 An Ant Colony Optimization Approach to the Probabilistic Traveling Salesman Problem......Page 801
E.9 Neural Processing of Symbolic Data......Page 802
References......Page 803
References......Page 831