Building machine learning projects with TensorFlow: engaging projects that will teach you how complex data can be exploited to gain the most insight

Building Machine Learning Projects with TensorFlow......Page 13
Credits......Page 14
About the Author......Page 15
About the Reviewer......Page 16
www.PacktPub.com......Page 17
Why subscribe?......Page 18
Preface......Page 19
What this book covers......Page 20
What you need for this book......Page 21
Who this book is for......Page 22
Conventions......Page 23
Reader feedback......Page 24
Customer support......Page 25
Downloading the example code......Page 26
Errata......Page 27
Piracy......Page 28
Questions......Page 29
1. Exploring and Transforming Data......Page 30
TensorFlow's main data structure - tensors......Page 31
Tensor rank......Page 32
Tensor data types......Page 33
Useful method......Page 35
Getting things done - interacting with TensorFlow......Page 36
Handling the computing workflow - TensorFlow's data flow graph......Page 37
Useful operation object methods......Page 38
Feeding......Page 39
Variable initialization......Page 40
Example graph building......Page 41
Running our programs - Sessions......Page 43
Basic tensor methods......Page 44
Reduction......Page 45
Tensor segmentation......Page 46
Sequences......Page 48
Tensor shape transformations......Page 49
Tensor slicing and joining......Page 50
Dataflow structure and results visualization - TensorBoard......Page 52
Command line use......Page 53
Adding Summary nodes......Page 54
Special Summary functions......Page 55
Interacting with TensorBoard's GUI......Page 57
The Iris dataset......Page 58
Reading image data......Page 59
Loading and processing the images......Page 60
Reading from the standard TensorFlow format......Page 61
Summary......Page 62
2. Clustering......Page 63
Learning from data - unsupervised learning......Page 64
Clustering......Page 65
k-means......Page 66
Algorithm iteration criterion......Page 67
k-means algorithm breakdown......Page 68
Pros and cons of k-means......Page 70
k-nearest neighbors......Page 72
Mechanics of k-nearest neighbors......Page 73
Pros and cons of k-nn......Page 74
Sample synthetic data plotting......Page 75
scikit-learn dataset module......Page 76
Synthetic dataset types......Page 77
Moon dataset......Page 78
Project 1 - k-means clustering on synthetic datasets......Page 79
Generating the dataset......Page 80
Model architecture......Page 81
Loss function description and optimizer loop......Page 83
Stop condition......Page 84
Results description......Page 85
Full source code......Page 86
k-means on circle synthetic data......Page 88
Project 2 - nearest neighbor on synthetic datasets......Page 90
Dataset generation......Page 91
Model architecture......Page 92
Loss function description......Page 93
Stop condition......Page 94
Results description......Page 95
Full source code......Page 96
Summary......Page 97
3. Linear Regression......Page 98
Univariate linear modelling function......Page 99
Sample data generation......Page 100
Determination of the cost function......Page 101
Least squares......Page 102
Minimizing the cost function......Page 103
General minima for least squares......Page 104
Iterative methods - gradient descent......Page 105
Example section......Page 106
The tf.train.Optimizer class......Page 107
Other Optimizer instance types......Page 108
Example 1 - univariate linear regression......Page 109
Dataset description......Page 110
Model architecture......Page 111
Cost function description and Optimizer loop......Page 113
Stop condition......Page 115
Results description......Page 116
Reviewing results with TensorBoard......Page 117
Full source code......Page 118
Example - multivariate linear regression......Page 120
Pandas library......Page 121
Dataset description......Page 122
Model architecture......Page 124
Loss function description and Optimizer loop......Page 126
Stop condition......Page 127
Results description......Page 128
Full source code......Page 129
Summary......Page 131
4. Logistic Regression......Page 132
Problem description......Page 133
Logistic function predecessor - the logit functions......Page 135
Bernoulli distribution......Page 136
Link function......Page 137
Logit function......Page 138
The importance of the logit inverse......Page 139
The logistic function......Page 140
Final estimated regression equation......Page 141
Loss function......Page 143
Cost function......Page 144
Data normalization for iterative methods......Page 145
One hot representation of outputs......Page 146
Example 1 - univariate logistic regression......Page 148
TensorFlow's softmax implementation......Page 149
Dataset loading and preprocessing implementation......Page 150
Model architecture......Page 152
Loss function description and optimizer loop......Page 153
Stop condition......Page 154
Fitting function representations across epochs......Page 155
Full source code......Page 156
Graphical representation......Page 157
Example 2 - Univariate logistic regression with skflow......Page 159
Useful libraries and methods......Page 160
Dataset description......Page 161
Model architecture......Page 162
Results description......Page 164
Full source code......Page 165
Summary......Page 166
5. Simple FeedForward Neural Networks......Page 167
Preliminary concepts......Page 168
Artificial neurons......Page 169
Perceptron algorithm......Page 170
Neural network layers......Page 171
Neural Network activation functions......Page 172
Gradients and the back propagation algorithm......Page 173
Neural networks problem choice - Classification vs Regression......Page 174
Sklearn preprocessing utilities......Page 175
First project - Non linear synthetic function regression......Page 177
Dataset description and loading......Page 178
Dataset preprocessing......Page 179
Modeling architecture - Loss Function description......Page 180
Loss function optimizer......Page 181
Example code......Page 182
Results description......Page 184
Second project - Modeling cars fuel efficiency with non linear regression......Page 185
Dataset description and loading......Page 186
Dataset preprocessing......Page 187
Modeling architecture......Page 188
Convergency test......Page 189
Results description......Page 190
Third project - Learning to classify wines: Multiclass classification......Page 192
Dataset description and loading......Page 193
Dataset preprocessing......Page 194
Modeling architecture......Page 195
Loss function description......Page 196
Loss function optimizer......Page 197
Convergence test......Page 198
Results description......Page 199
Full source code......Page 200
Summary......Page 202
6. Convolutional Neural Networks......Page 203
Origin of convolutional neural networks......Page 204
Continuous convolution......Page 205
Discrete convolution......Page 206
Kernels and convolutions......Page 207
Interpretation of the convolution operations......Page 208
Sample code - applying convolution to a grayscale image......Page 209
Sample kernels results......Page 210
Subsampling layers implementation performance.......Page 212
Sample code......Page 213
Applying the dropout operation in TensorFlow......Page 215
Sample code......Page 216
Subsampling layer......Page 217
Example 1 - MNIST digit classification......Page 218
Dataset description and loading......Page 219
Dataset preprocessing......Page 222
Modelling architecture......Page 223
Loss function description......Page 224
Loss function optimizer......Page 225
Accuracy test......Page 226
Result description......Page 227
Full source code......Page 228
Example 2 - image classification with the CIFAR10 dataset......Page 231
Dataset description and loading......Page 232
Dataset preprocessing......Page 233
Modelling architecture......Page 234
Training and accuracy tests......Page 235
Full source code......Page 236
Summary......Page 238
7. Recurrent Neural Networks and LSTM......Page 239
Recurrent neural networks......Page 240
Exploding and vanishing gradients......Page 242
LSTM neural networks......Page 243
The gate operation - a fundamental component......Page 244
Part 2 - set values to keep, change state......Page 246
Part 3 - output filtered cell state......Page 247
Other RNN architectures......Page 248
class MultiRNNCell(RNNCell)......Page 249
learn.ops.split_squeeze(dim, num_split, tensor_in)......Page 250
Example 1 - univariate time series prediction with energy consumption data......Page 251
Dataset description and loading......Page 252
Dataset preprocessing......Page 254
Modelling architecture......Page 255
Loss function description......Page 258
Convergency test......Page 259
Results description......Page 260
Full source code......Page 261
Example 2 - writing music "a la" Bach......Page 264
Character sequences and probability representation......Page 265
ABC format data organization......Page 266
Saving and restoring variables and models......Page 267
Variable restoring......Page 268
Dataset description and loading......Page 269
Network Training......Page 270
Modelling architecture......Page 271
Loss function description......Page 273
Stop condition......Page 274
Results description......Page 275
Full source code......Page 276
Summary......Page 281
8. Deep Neural Networks......Page 282
Deep neural network definition......Page 283
Deep network architectures through time......Page 284
LeNet 5......Page 285
Alexnet......Page 286
Main features......Page 287
The original inception model......Page 289
GoogLenet (Inception V1)......Page 290
Batch normalized inception (V2)......Page 292
Inception v3......Page 293
Residual Networks (ResNet)......Page 296
Other deep neural network architectures......Page 297
Example - painting with style - VGG style transfer......Page 298
Useful libraries and methods......Page 299
Dataset description and loading......Page 300
Dataset preprocessing......Page 301
Modeling architecture......Page 302
Content loss function......Page 303
Loss optimization loop......Page 304
Convergency test......Page 305
Program execution......Page 306
Full source code......Page 307
Summary......Page 314
9. Running Models at Scale – GPU and Serving......Page 315
GPU support on TensorFlow......Page 316
Querying the computing capabilities......Page 317
Device naming......Page 318
Example 1 - assigning an operation to the GPU......Page 319
Example 2 - calculating Pi number in parallel......Page 320
Solution implementation......Page 322
Source code......Page 323
Distributed TensorFlow......Page 324
Combined overview......Page 325
ClusterSpec definition format......Page 326
Creating tf.Train.Server......Page 327
Sample distributed code structure......Page 328
Example 3 - distributed Pi calculation......Page 330
Server script......Page 331
Client script......Page 332
Full source code......Page 333
Example 4 - running a distributed model in a cluster......Page 334
Sample code......Page 335
Summary......Page 338
10. Library Installation and Additional Tips......Page 339
Linux installation......Page 340
Initial requirements......Page 341
Ubuntu preparation tasks (need to apply before any method)......Page 342
Testing your installation......Page 343
GPU support......Page 344
Environment test......Page 345
Allowing Docker to run with a normal user......Page 347
Run the TensorFlow container......Page 348
Adding the Bazel distribution URI as a package source......Page 350
Installing CUDA system packages......Page 351
Installing cuDNN......Page 352
Configuring TensorFlow build......Page 353
Testing the installation......Page 354
Windows installation......Page 356
Downloading the Docker toolbox installer......Page 357
Creating the Docker machine......Page 359
MacOS X installation......Page 362
Install pip......Page 363
Summary......Page 367