Practical Neural Network Recipes in C++

Contents......Page all_10073_to_00502.cpc0006.djvu
Preface......Page all_10073_to_00502.cpc0013.djvu
1. Foundations......Page all_10073_to_00502.cpc0015.djvu
Motivation......Page all_10073_to_00502.cpc0016.djvu
New Life for Old Techniques......Page all_10073_to_00502.cpc0017.djvu
Perceptrons and Linear Separability......Page all_10073_to_00502.cpc0018.djvu
Neural Network Capabilities......Page all_10073_to_00502.cpc0020.djvu
Basic Structure of a Neural Network......Page all_10073_to_00502.cpc0022.djvu
Training......Page all_10073_to_00502.cpc0023.djvu
Validation......Page all_10073_to_00502.cpc0024.djvu
Leave-k-out Method......Page all_10073_to_00502.cpc0026.djvu
2. Classification......Page all_10073_to_00502.cpc0029.djvu
Binary Decisions......Page all_10073_to_00502.cpc0030.djvu
Making the Decision......Page all_10073_to_00502.cpc0031.djvu
Reject Category......Page all_10073_to_00502.cpc0032.djvu
Other Encoding Schemes......Page all_10073_to_00502.cpc0033.djvu
Supervised versus Unsupervised Training......Page all_10073_to_00502.cpc0035.djvu
3. Autoassociation......Page all_10073_to_00502.cpc0037.djvu
Autoassociative Filtering......Page all_10073_to_00502.cpc0038.djvu
Code for Autoassociative Filtering......Page all_10073_to_00502.cpc0042.djvu
Noise Reduction......Page all_10073_to_00502.cpc0043.djvu
Learning a Prototype from Exemplars......Page all_10073_to_00502.cpc0045.djvu
Exposing Isolated Events......Page all_10073_to_00502.cpc0046.djvu
Pattern Completion......Page all_10073_to_00502.cpc0054.djvu
Error Correction......Page all_10073_to_00502.cpc0055.djvu
Encoding Words......Page all_10073_to_00502.cpc0056.djvu
Data Compression......Page all_10073_to_00502.cpc0058.djvu
4. Time-Series Prediction......Page all_10073_to_00502.cpc0060.djvu
The Basic Model......Page all_10073_to_00502.cpc0062.djvu
Input Data......Page all_10073_to_00502.cpc0063.djvu
Trend Elimination......Page all_10073_to_00502.cpc0064.djvu
Code for Detrending and Retrending......Page all_10073_to_00502.cpc0068.djvu
Seasonal Variation......Page all_10073_to_00502.cpc0071.djvu
Scaling......Page all_10073_to_00502.cpc0073.djvu
Multiple Prediction......Page all_10073_to_00502.cpc0074.djvu
Multiple Predictors......Page all_10073_to_00502.cpc0075.djvu
Measuring Prediction Error......Page all_10073_to_00502.cpc0077.djvu
5. Function Approximation......Page all_10073_to_00502.cpc0080.djvu
Univariate Function Approximation......Page all_10073_to_00502.cpc0081.djvu
Inverse Modeling......Page all_10073_to_00502.cpc0085.djvu
Multiple Regression......Page all_10073_to_00502.cpc0087.djvu
6. Multilayer Feedforward Networks......Page all_10073_to_00502.cpc0090.djvu
Basic Architecture......Page all_10073_to_00502.cpc0091.djvu
Activation Functions......Page all_10073_to_00502.cpc0093.djvu
Example Network......Page all_10073_to_00502.cpc0095.djvu
Linear Output Neurons......Page all_10073_to_00502.cpc0097.djvu
Theoretical Discussion......Page all_10073_to_00502.cpc0098.djvu
Bibliography of Feedforward Network Theory......Page all_10073_to_00502.cpc0101.djvu
Algorithms for Executing the Network......Page all_10073_to_00502.cpc0103.djvu
Training the Network......Page all_10073_to_00502.cpc0107.djvu
Training by Backpropagation of Errors......Page all_10073_to_00502.cpc0113.djvu
Training by Conjugate Gradients......Page all_10073_to_00502.cpc0118.djvu
Minimizing along a Direction......Page all_10073_to_00502.cpc0119.djvu
Choosing the Direction for Minimization......Page all_10073_to_00502.cpc0123.djvu
Eluding Local Minima in Learning......Page all_10073_to_00502.cpc0124.djvu
Local Minima Happen Easily......Page all_10073_to_00502.cpc0125.djvu
Mistaken Minima......Page all_10073_to_00502.cpc0127.djvu
Other Means of Escape......Page all_10073_to_00502.cpc0128.djvu
When to Use a Multiple-Layer Feedforward Network......Page all_10073_to_00502.cpc0129.djvu
7. Eluding Local Minima I: Simulated Annealing......Page all_10073_to_00502.cpc0130.djvu
Overview......Page all_10073_to_00502.cpc0131.djvu
Choosing the Annealing Parameters......Page all_10073_to_00502.cpc0132.djvu
Implementation in Feedforward Network Learning......Page all_10073_to_00502.cpc0134.djvu
A Sample Program......Page all_10073_to_00502.cpc0135.djvu
A Sample Function......Page all_10073_to_00502.cpc0139.djvu
Random Number Generation......Page all_10073_to_00502.cpc0141.djvu
Going on from Here......Page all_10073_to_00502.cpc0145.djvu
8. Eluding Local Minima II: Genetic Optimization......Page all_10073_to_00502.cpc0148.djvu
Overview......Page all_10073_to_00502.cpc0149.djvu
Designing the Genetic Structure......Page all_10073_to_00502.cpc0151.djvu
Evaluation......Page all_10073_to_00502.cpc0153.djvu
Parent Selection......Page all_10073_to_00502.cpc0157.djvu
Reproduction......Page all_10073_to_00502.cpc0160.djvu
Mutation......Page all_10073_to_00502.cpc0161.djvu
A Genetic Minimization Subroutine......Page all_10073_to_00502.cpc0162.djvu
Some Functions for Genetic Optimization......Page all_10073_to_00502.cpc0168.djvu
Gray Codes......Page all_10073_to_00502.cpc0170.djvu
Two-Point Crossover......Page all_10073_to_00502.cpc0172.djvu
9. Regression and Neural Networks......Page all_10073_to_00502.cpc0178.djvu
Overview......Page all_10073_to_00502.cpc0179.djvu
Singular-Value Decomposition......Page all_10073_to_00502.cpc0180.djvu
Regression in Neural Networks......Page all_10073_to_00502.cpc0182.djvu
10. Designing Feedforward Network Architectures......Page all_10073_to_00502.cpc0186.djvu
How Many Hidden Layers?......Page all_10073_to_00502.cpc0187.djvu
How Many Hidden Neurons?......Page all_10073_to_00502.cpc0189.djvu
How Long Do I Train This Thing???......Page all_10073_to_00502.cpc0193.djvu
11. Interpreting Weights: How Does This Thing Work?......Page all_10073_to_00502.cpc0199.djvu
Features Used by Networks in General......Page all_10073_to_00502.cpc0202.djvu
Examination of Weight Vectors......Page all_10073_to_00502.cpc0203.djvu
Hinton Diagrams......Page all_10073_to_00502.cpc0204.djvu
Clustering......Page all_10073_to_00502.cpc0206.djvu
Sensitivity Analysis......Page all_10073_to_00502.cpc0207.djvu
Stereotypical Inputs......Page all_10073_to_00502.cpc0209.djvu
12. Probabilistic Neural Networks......Page all_10073_to_00502.cpc0213.djvu
Overview......Page all_10073_to_00502.cpc0214.djvu
Computational Aspects......Page all_10073_to_00502.cpc0220.djvu
Optimizing Sigma......Page all_10073_to_00502.cpc0221.djvu
Related Models......Page all_10073_to_00502.cpc0222.djvu
A Sample Program......Page all_10073_to_00502.cpc0223.djvu
Optimizing Sigma......Page all_10073_to_00502.cpc0225.djvu
Other Optimization Criteria......Page all_10073_to_00502.cpc0230.djvu
Bayesian Confidence Measures......Page all_10073_to_00502.cpc0231.djvu
Autoassociative Versions......Page all_10073_to_00502.cpc0232.djvu
When to Use a Probabilistic Neural Network......Page all_10073_to_00502.cpc0233.djvu
13. Functional Link Networks......Page all_10073_to_00502.cpc0235.djvu
Application to Nonlinear Approximation......Page all_10073_to_00502.cpc0238.djvu
Mathematics of the Functional Link Network......Page all_10073_to_00502.cpc0239.djvu
When to Use a Functional Link Network......Page all_10073_to_00502.cpc0241.djvu
14. Hybrid Networks......Page all_10073_to_00502.cpc0243.djvu
Functional Link Net as a Hidden Layer......Page all_10073_to_00502.cpc0244.djvu
Fast Bayesian Confidences......Page all_10073_to_00502.cpc0247.djvu
Training......Page all_10073_to_00502.cpc0250.djvu
Attention-based Processing......Page all_10073_to_00502.cpc0251.djvu
Factorable Problems......Page all_10073_to_00502.cpc0254.djvu
Training the Data Reduction Networks......Page all_10073_to_00502.cpc0255.djvu
Splitting Is Not Always Effective......Page all_10073_to_00502.cpc0256.djvu
15. Designing the Training Set......Page all_10073_to_00502.cpc0257.djvu
Number of Samples......Page all_10073_to_00502.cpc0258.djvu
Overfitting......Page all_10073_to_00502.cpc0259.djvu
Network Size Affects Training Set Size......Page all_10073_to_00502.cpc0260.djvu
Borderline Cases......Page all_10073_to_00502.cpc0261.djvu
Hidden Bias......Page all_10073_to_00502.cpc0262.djvu
Fudging Cases......Page all_10073_to_00502.cpc0263.djvu
16. Preparing Input Data......Page all_10073_to_00502.cpc0265.djvu
General Considerations......Page all_10073_to_00502.cpc0266.djvu
Nominal Variables......Page all_10073_to_00502.cpc0267.djvu
Ordinal Variables......Page all_10073_to_00502.cpc0271.djvu
Interval Variables......Page all_10073_to_00502.cpc0274.djvu
Is Scaling Always Necessary?......Page all_10073_to_00502.cpc0278.djvu
Transformations......Page all_10073_to_00502.cpc0279.djvu
Circular Discontinuity......Page all_10073_to_00502.cpc0282.djvu
View Angles......Page all_10073_to_00502.cpc0283.djvu
Hue......Page all_10073_to_00502.cpc0284.djvu
Outliers......Page all_10073_to_00502.cpc0286.djvu
Discarding Data......Page all_10073_to_00502.cpc0287.djvu
Missing Data......Page all_10073_to_00502.cpc0288.djvu
17. Fuzzy Data and Processing......Page all_10073_to_00502.cpc0291.djvu
Treating Fuzzy Values as Nominal and Ordinal......Page all_10073_to_00502.cpc0293.djvu
Advantages of Fuzzy Set Processing......Page all_10073_to_00502.cpc0294.djvu
The Neural Network - Fuzzy Set Interface......Page all_10073_to_00502.cpc0295.djvu
Membership Functions......Page all_10073_to_00502.cpc0296.djvu
Continuous Variables......Page all_10073_to_00502.cpc0299.djvu
Multivariate Domains......Page all_10073_to_00502.cpc0300.djvu
Hedges......Page all_10073_to_00502.cpc0301.djvu
Negation, Conjunction, and Disjunction......Page all_10073_to_00502.cpc0302.djvu
Modus Ponens......Page all_10073_to_00502.cpc0304.djvu
Combining Operations......Page all_10073_to_00502.cpc0307.djvu
Defuzzification......Page all_10073_to_00502.cpc0311.djvu
Maximum Height Method......Page all_10073_to_00502.cpc0312.djvu
Centroid Method......Page all_10073_to_00502.cpc0313.djvu
Constructors......Page all_10073_to_00502.cpc0315.djvu
Negation and Scaling......Page all_10073_to_00502.cpc0319.djvu
Conjunction and Disjunction......Page all_10073_to_00502.cpc0320.djvu
Centroid......Page all_10073_to_00502.cpc0326.djvu
Simplifying Interactions......Page all_10073_to_00502.cpc0328.djvu
Fuzzy One-of-n Coding......Page all_10073_to_00502.cpc0329.djvu
Simple Membership Output......Page all_10073_to_00502.cpc0331.djvu
Postprocessing with Defuzzification......Page all_10073_to_00502.cpc0332.djvu
18. Unsupervised Training......Page all_10073_to_00502.cpc0339.djvu
Input Normalization......Page all_10073_to_00502.cpc0342.djvu
Z-Axis Normalization......Page all_10073_to_00502.cpc0343.djvu
Training the Kohonen Network......Page all_10073_to_00502.cpc0344.djvu
Updating the Weights......Page all_10073_to_00502.cpc0346.djvu
Learning Rate......Page all_10073_to_00502.cpc0348.djvu
Measuring Network Error......Page all_10073_to_00502.cpc0349.djvu
Determining Convergence......Page all_10073_to_00502.cpc0350.djvu
Neurons That Refuse to Learn......Page all_10073_to_00502.cpc0351.djvu
Self-Organization......Page all_10073_to_00502.cpc0352.djvu
19. Evaluating Performance of Neural Networks......Page all_10073_to_00502.cpc0354.djvu
Mean Square Error......Page all_10073_to_00502.cpc0355.djvu
Problems with Mean Square Error......Page all_10073_to_00502.cpc0356.djvu
Relatives of Mean Square Error......Page all_10073_to_00502.cpc0357.djvu
Cost Functions......Page all_10073_to_00502.cpc0358.djvu
Confusion Matrix......Page all_10073_to_00502.cpc0359.djvu
ROC (Receiver Operating Characteristic) Curves......Page all_10073_to_00502.cpc0362.djvu
Computing the ROC Curve Area......Page all_10073_to_00502.cpc0365.djvu
Cost Functions and ROC Curves......Page all_10073_to_00502.cpc0368.djvu
Signal-to-Noise Ratio......Page all_10073_to_00502.cpc0370.djvu
20. Confidence Measures......Page all_10073_to_00502.cpc0372.djvu
Testing Individual Hypotheses......Page all_10073_to_00502.cpc0373.djvu
Computing Confidence......Page all_10073_to_00502.cpc0378.djvu
Confidence in the Null Hypothesis......Page all_10073_to_00502.cpc0379.djvu
Multiple Classes......Page all_10073_to_00502.cpc0380.djvu
Confidence in the Confidence......Page all_10073_to_00502.cpc0381.djvu
Example Programs......Page all_10073_to_00502.cpc0382.djvu
Sorting......Page all_10073_to_00502.cpc0383.djvu
Estimating the Distribution......Page all_10073_to_00502.cpc0384.djvu
Estimating Confidences......Page all_10073_to_00502.cpc0385.djvu
Bayesian Methods......Page all_10073_to_00502.cpc0387.djvu
Example Program......Page all_10073_to_00502.cpc0392.djvu
Multiple Classes......Page all_10073_to_00502.cpc0393.djvu
Hypothesis Testing versus Bayes\' Method......Page all_10073_to_00502.cpc0395.djvu
21. Optimizing the Decision Threshold......Page all_10073_to_00502.cpc0400.djvu
22. Using the NEURAL Program......Page all_10073_to_00502.cpc0413.djvu
GENERAL Model......Page all_10073_to_00502.cpc0415.djvu
The LAYER Network Model......Page all_10073_to_00502.cpc0416.djvu
Initialization by Genetic Optimization......Page all_10073_to_00502.cpc0417.djvu
Learning......Page all_10073_to_00502.cpc0418.djvu
The KOHONEN Network Model......Page all_10073_to_00502.cpc0419.djvu
Initialization and Learning......Page all_10073_to_00502.cpc0420.djvu
Saving Weights and Execution Results......Page all_10073_to_00502.cpc0422.djvu
Alphabetical Glossary of Commands......Page all_10073_to_00502.cpc0423.djvu
Verification of Program Operation......Page all_10073_to_00502.cpc0427.djvu
Appendix......Page all_10073_to_00502.cpc0433.djvu
Bibliography......Page all_10073_to_00502.cpc0488.djvu
Index......Page all_10073_to_00502.cpc0500.djvu