Introduction to Deep Learning: With Complexe Python and TensorFlow Examples

"How are they called? Neutrons?"......Page 6
Convolutional Neural Networks drive the boom......Page 7
Neuroscience as a treasure for machine learning......Page 15
About this book......Page 18
Exponential growth of interest......Page 21
Acquisition of DL startups......Page 24
Hardware for DL......Page 27
Software for DL......Page 31
Your brain - A fascinating computing device......Page 33
Structure of a neuron......Page 36
Signal processing by action potentials......Page 38
Synapses......Page 40
Neuronal plasticity......Page 41
Spike-Timing Dependent Plasticity (STDP)......Page 43
What is the function of a biological neuron?......Page 47
Neurons as spatial feature or evidence detectors......Page 48
Neurons as temporal coincidence detectors......Page 52
Perceptron neuron model......Page 53
Neurons as filters......Page 56
Other neuron models......Page 61
Neural Coding......Page 62
The Perceptron neuro-computer......Page 65
Perceptron learning......Page 67
Perceptron in Python......Page 69
Limitations of the Perceptron......Page 77
The SOM neural network model......Page 83
A SOM in Python......Page 89
SOM and the Cortex......Page 101
The goal......Page 108
Basic idea is gradient descent......Page 109
Splitting the weight change formula into three parts......Page 111
Computing the first part......Page 112
Computing the third part......Page 113
Backpropagation pseudo code......Page 117
MLP in Python......Page 119
Visualization of decision boundaries......Page 134
The need for non-linear transfer functions......Page 138
Introduction......Page 141
Training a linear model with TensorFlow......Page 150
A MLP with TensorFlow......Page 152
Introduction......Page 160
Some history about the CNN model......Page 164
Convolutional and pooling layers in TensorFlow......Page 167
Parameters to be defined for a convolution layer......Page 173
How to compute the dimension of an output tensor......Page 178
A CNN in TensorFlow......Page 179
Fighting against vanishing gradients......Page 195
Momentum optimization......Page 197
Nesterov Momentum Optimization......Page 200
AdaGrad......Page 201
RMSProp......Page 202
Adam......Page 203
Comparison of optimizers......Page 204
Batch normalization......Page 207
Principle of attention......Page 210
Principle of lifelong learning......Page 211
Principle of embodiment......Page 212
Principle of prediction......Page 213
Cognitive architectures......Page 214
Ex. 1 - Preparing to work with Python......Page 217
Ex. 2 - Python syntax......Page 221
Ex. 3 - Understanding convolutions......Page 224
Ex. 4 - NumPy......Page 227
Ex. 5 - Perceptron......Page 232
Ex. 6 - Speech Recognition with a SOM......Page 234
Ex. 7 - MLP with feedfoward step......Page 235
Ex. 8 - Backpropagation......Page 236
Ex. 9 - A MLP with TensorFlow......Page 237
Ex. 10 - CNN Experiments......Page 238
Ex. 11 - CNN for word recognition using Keras......Page 239
Ex. 12 - Vanishing gradients problem......Page 240
Ex. 13 - Batch normalization in TensorFlow......Page 241