Deep Belief Nets in C++ and CUDA C 3 Convolutional Nets

Contents......Page 3
Intro......Page 6
Multiple-Layer Feedforward Networks......Page 8
Wide vs Deep Nets......Page 11
Locally Connected Layers......Page 13
Pooling Layers......Page 21
Output Layer......Page 22
Back Propagation of Errors for the Gradient......Page 25
Programming Algorithms......Page 30
Model  Declarations......Page 31
Initializations in the Model Constructor......Page 33
Finding all Activations......Page 36
Activating Fully Connected Layer......Page 37
Activating Locally Connected Layer......Page 38
Activating Convolutional Layer......Page 41
Activating Pooling Layer......Page 43
Evaluating the Criterion......Page 46
Evaluating the Gradient......Page 49
Gradient for Fully Connected Layer......Page 53
Gradient for Locally Connected Layer......Page 55
Gradient for Convolutional Layer......Page 58
Gradient for Pooled Layer (Not!)......Page 59
Backpropagating Delta from a Nonpooled Layer......Page 60
Backpropagating Delta from Pooled Layer......Page 63
Multithreading Gradient Computation......Page 65
CUDA Code......Page 73
Weight Layout in the CUDA Implementation......Page 74
Global Variables on the Device......Page 75
Initialization......Page 77
Copying Weights to the Device......Page 78
Activating the Output Layer......Page 85
Activating Locally Connected and Convolutional  Layers......Page 87
Activating a Pooled Layer......Page 107
SoftMax and Log Likelihood by Reduction......Page 111
Computing Delta for the Output Layer......Page 115
Backpropagating from a Fully Connected Layer......Page 117
Backpropagating  from Convolutional    and Local Layers......Page 119
Backpropagating from a Pooling Layer......Page 125
Gradient of a Fully Connected Layer......Page 128
Gradient of a Locally Connected or Convolutional  Layer......Page 130
Fetching  the Gradient......Page 141
Putting It All Together......Page 147
Menu Options......Page 152
Read Control File......Page 155
Display Options......Page 165
CONVNET.LOG file......Page 171
CUDA.LOG File......Page 177
Index......Page 178