Deep learning: algorithms and applications

Preface......Page 6
Contents......Page 7
 1 Introduction......Page 13
  2.1 Linear Activation......Page 16
  2.2 Binary Step Activation......Page 17
  2.3 Sigmoid and Softmax......Page 18
  2.4 Tanh (Hyperbolic Tan)......Page 20
  2.5 The ReLU Family......Page 22
  2.6 Softplus......Page 27
  2.7 Maxout......Page 28
  2.8 Swish Activation Function......Page 29
 3 Comparison of Activation Functions......Page 30
  4.1 Learning Adaptive Piecewise Linear (APL) Activation Functions......Page 32
  4.2 SLAF: Learning Non-linear Approximation of Activation Functions......Page 33
 5 Conclusion......Page 39
 References......Page 40
Adversarial Examples in Deep Neural Networks: An Overview......Page 43
 1 Introduction......Page 44
  1.1 Notation and Preliminaries......Page 45
 2 Adversarial Perturbation Design......Page 46
  2.1 White-Box Attacks......Page 49
  2.2 Black-Box Attacks and Universal Adversarial Perturbations......Page 56
 3 Theoretical Explanations of the Nature of Adversarial Examples......Page 59
  3.1 Linearity Hypothesis and Curvature of Decision Boundaries......Page 61
  3.2 Boundary Tilting and Other Explanations......Page 64
  3.3 Feature Selection and No Free Lunch Theorems  for Adversarial Robustness......Page 66
  3.4 Generalization Bounds for Adversarial Examples......Page 68
  4.1 Obfuscated Gradients and Adversarial Training......Page 70
  4.2 Robust Regularization......Page 71
 5 Future Directions......Page 73
 References......Page 74
Representation Learning in Power Time Series Forecasting......Page 78
 1 Introduction......Page 79
 2 Regression in Power Time Series Forecasting......Page 80
  2.1 Renewable Power Time Series Forecasting......Page 81
  2.2 Challenges of Power Time Series Forecasting......Page 82
 3 Foundations of Representation Learning......Page 83
  3.1 Traditional Dimensionality Reduction Techniques......Page 84
  3.2 Deep Architectures for Latent Feature Extraction......Page 87
 4 Evaluation of Representation Learning in Regression Tasks......Page 92
  4.1 Evaluation of a Regression Model......Page 93
  4.2 Evaluation of the Learned Feature Representation......Page 94
  5.1 The Power Time Series......Page 96
  5.2 Representation Learning Experiments......Page 97
  5.3 Principal Component Analysis for Feature Extraction......Page 99
  5.4 Deep Architectures for Feature Extraction......Page 103
  5.5 Fine-Tuning for Power Forecasting......Page 106
 6 Conclusion......Page 109
 References......Page 110
Deep Learning Application: Load Forecasting in€Big Data of€Smart Grids......Page 113
 1 Introduction......Page 114
 2 Background......Page 116
  2.2 Load Forecasting......Page 117
  2.3 Influential Factors of€Load Forecasting Models......Page 119
 3 Forecasting Modeling Issues in€Smart Grids......Page 120
  3.2 Traditional Load Forecasting Models......Page 121
  3.3 Traditional Load Forecasting Models......Page 127
  4.1 Guidelines and€Solutions to€Modeling Issues......Page 128
  4.2 Case Study......Page 131
 5 Conclusions and€Future Trends......Page 135
 References......Page 136
Fast and Accurate Seismic Tomography via Deep Learning......Page 139
 1 Introduction......Page 140
  2.1 Seismic Data......Page 141
  2.2 Seismic Tomography......Page 143
  3.1 Deep Neural Networks for Inverse Imaging Problems......Page 145
  3.2 Velocity Semblance as Input Feature for Deep Networks......Page 148
  4.1 Experimental Setup......Page 149
  4.3 Results and Analysis......Page 151
  5.1 Industry VMB Methods......Page 153
  5.2 Full Waveform Inversion......Page 154
  5.3 Baseline Comparison Setup......Page 156
  5.5 Results......Page 157
  5.6 Discussion......Page 158
 6 Feature Extraction-Free Results......Page 162
 References......Page 164
Traffic Light and€Vehicle Signal Recognition with€High Dynamic Range Imaging and€Deep Learning......Page 167
 1 Introduction......Page 168
  2.1 HDR Imaging......Page 170
  2.2 Dark Images for€Detecting Traffic Light Candidates......Page 171
  2.3 Saliency Map Filtering......Page 173
  2.4 Auto Exposure for€Uncontrolled Illumination......Page 174
  2.5 Region of€Interest (ROI)......Page 175
  3.1 Dual Channel Mechanism......Page 177
  3.2 Customized Convolutional Neural Network......Page 178
 4 Temporal Trajectory Analysis......Page 180
  5.1 Evaluation of€Performance......Page 182
  5.2 Comparison with€State of€the€Art Algorithms......Page 183
 6 HDR Imaging Vehicle Signal Recognition......Page 193
  6.3 Vehicle Detection......Page 195
  6.4 Brake Light Pattern and€Recognition......Page 196
  6.5 Experimental Results......Page 197
 7 Conclusion and€Future Work......Page 199
 References......Page 200
The Application of Deep Learning  in Marine Sciences......Page 203
 1 Introduction......Page 204
 2 Methodology......Page 205
 3 Seagrass Segmentation......Page 207
  3.1 Deep Learning Approach......Page 209
  3.2 Experimental Framework......Page 212
  3.3 Classification Results......Page 217
 4 Jellyfish Detection and Identification......Page 223
  4.1 Deep Learning Approach......Page 224
  4.2 Experimental Framework......Page 228
  4.3 Results......Page 233
 5 Conclusions......Page 236
 References......Page 237
Deep Learning Case Study on€Imbalanced Training Data for€Automatic Bird Identification......Page 241
 1 Introduction......Page 242
 2 The System......Page 243
 3 Related Work......Page 244
 4 Data......Page 245
  4.1 Data Augmentation......Page 247
  4.2 Grouping Data......Page 248
 5 Classification......Page 250
  5.2 Feature Extraction......Page 251
  5.4 Dealing with€Imbalanced Data......Page 254
  5.5 Hybrid of€Hierarchical and€Cascade Model......Page 256
  5.6 Top-Level Classification......Page 258
  5.7 Classification of€Waterfowl......Page 262
  5.8 Classification of€Gulls and€Terns......Page 263
 6 Results......Page 266
 7 Discussion......Page 270
 References......Page 271
Deep Learning for€Person Re-identification in€Surveillance Videos......Page 273
 1 Introduction......Page 274
 2 Preliminaries of€Deep Neural Networks......Page 275
 4 Deep Learning Architectures for€Person Re-identification......Page 290
 5 Experimental Setup......Page 299
 6 Conclusions and€Future Work......Page 302
 References......Page 303
Deep Learning in Gait Analysis  for Security and Healthcare......Page 308
 1 Introduction......Page 309
 2 Gait Analysis Review......Page 310
  2.1 Non-wearable Sensors......Page 312
  2.2 Wearable Sensors......Page 314
  2.3 A Review of Floor Sensor Systems and Datasets for Gait Analysis......Page 315
 3 Deep Learning for Gait Analysis......Page 316
  3.1 Convolutional Neural Networks......Page 318
  4.1 Aims and General Method......Page 320
  4.3 Methodology......Page 321
  4.4 Experiments for Age-Related Classification......Page 322
  4.5 Spatio-Temporal Deep Learning Model......Page 323
  4.6 Results for Age-Related Differences......Page 324
  4.8 Discussion......Page 326
  4.9 Future Directions......Page 329
  5.1 Aims and General Method......Page 330
  5.2 Footstep Data as a Biometric......Page 331
  5.3 Deep Residual Network Model......Page 332
  5.4 Spatial and Temporal Architectures......Page 334
  5.6 Results......Page 335
  5.8 Discussion......Page 336
 6 Conclusions......Page 338
 References......Page 339
Deep Learning for Building Occupancy Estimation Using Environmental Sensors......Page 344
 1 Introduction......Page 345
 2 Literature Review......Page 346
  3.1 Overview......Page 347
  3.2 Convolutional Operation......Page 349
  3.3 Deep Bi-directional LSTM......Page 350
  3.5 Training Process of the CDBLSTM......Page 353
 4 Evaluation Results......Page 354
  4.1 Data Collection......Page 355
  4.2 Evaluation Setup......Page 356
  4.3 Evaluation Results......Page 357
  4.4 HyperParameters......Page 360
  4.5 The Impact of Noise......Page 361
  4.6 Generalization Performance......Page 362
  4.7 Additional Evaluation with Data from Another Environment......Page 363
 References......Page 364
Index......Page 367