Java for Data Science

Java for Data Science......Page 11
Credits......Page 12
About the Authors......Page 13
About the Reviewers......Page 14
www.PacktPub.com......Page 15
eBooks, discount offers, and more......Page 16
Why subscribe?......Page 17
Customer Feedback......Page 18
Preface......Page 19
What this book covers......Page 20
What you need for this book......Page 22
Who this book is for......Page 23
Conventions......Page 24
Reader feedback......Page 25
Customer support......Page 26
Downloading the example code......Page 27
Errata......Page 28
Piracy......Page 29
Questions......Page 30
1. Getting Started with Data Science......Page 31
Problems solved using data science......Page 33
Understanding the data science problem -  solving approach......Page 34
Using Java to support data science......Page 35
Acquiring data for an application......Page 36
The importance and process of cleaning data......Page 37
Visualizing data to enhance understanding......Page 40
The use of statistical methods in data science......Page 42
Machine learning applied to data science......Page 45
Using neural networks in data science......Page 47
Deep learning approaches......Page 49
Performing text analysis......Page 51
Visual and audio analysis......Page 53
Improving application performance using parallel techniques......Page 55
Assembling the pieces......Page 57
Summary......Page 58
2. Data Acquisition......Page 59
Understanding the data formats used in data science applications......Page 60
Overview of CSV data......Page 61
Overview of spreadsheets......Page 62
Overview of databases......Page 63
Overview of PDF files......Page 65
Overview of JSON......Page 66
Overview of XML......Page 67
Overview of streaming data......Page 69
Overview of audio/video/images in Java......Page 70
Data acquisition techniques......Page 71
Using the HttpUrlConnection class......Page 72
Creating your own web crawler......Page 74
Using the crawler4j web crawler......Page 77
Web scraping in Java......Page 81
Handing Twitter......Page 84
Handling Wikipedia......Page 86
Handling Flickr......Page 88
Searching by keyword......Page 91
Summary......Page 95
3. Data Cleaning......Page 96
Handling data formats......Page 98
Handling CSV data......Page 99
Handling Excel spreadsheets......Page 101
Handling PDF files......Page 103
Using JSON streaming API......Page 104
Using the JSON tree API......Page 108
The nitty gritty of cleaning text......Page 110
Third-party tokenizers and libraries......Page 113
Simple text cleaning......Page 115
Removing stop words......Page 116
Finding words in text......Page 118
Finding and replacing text......Page 119
Data imputation......Page 121
Subsetting data......Page 123
Sorting text......Page 125
Validating data types......Page 129
Validating dates......Page 130
Validating e-mail addresses......Page 131
Validating ZIP codes......Page 132
Validating names......Page 133
Cleaning images......Page 135
Changing the contrast of an image......Page 136
Smoothing an image......Page 137
Brightening an image......Page 140
Resizing an image......Page 141
Converting images to different formats......Page 142
Summary......Page 143
4. Data Visualization......Page 144
Understanding plots and graphs......Page 145
Visual analysis goals......Page 151
Creating index charts......Page 152
Creating bar charts......Page 155
Using country as the category......Page 157
Using decade as the category......Page 159
Creating stacked graphs......Page 161
Creating pie charts......Page 163
Creating scatter charts......Page 165
Creating histograms......Page 167
Creating donut charts......Page 170
Creating bubble charts......Page 172
Summary......Page 174
5. Statistical Data Analysis Techniques......Page 175
Working with mean, mode, and median......Page 176
Using Java 8 techniques to find mean......Page 177
Using Apache Commons to find mean......Page 178
Using simple Java techniques to find median......Page 180
Using Apache Commons to find the median......Page 181
Calculating the mode......Page 182
Using ArrayLists to find multiple modes......Page 183
Using a Apache Commons to find multiple modes......Page 184
Standard deviation......Page 186
Sample size determination......Page 189
Hypothesis testing......Page 190
Regression analysis......Page 191
Using simple linear regression......Page 193
Using multiple regression......Page 197
Summary......Page 202
6. Machine Learning......Page 203
Supervised learning techniques......Page 205
Decision trees......Page 206
Decision tree libraries......Page 207
Using a decision tree with a book dataset......Page 208
Testing the book decision tree......Page 210
Support vector machines......Page 213
Using an SVM for camping data......Page 214
Testing individual instances......Page 217
Bayesian networks......Page 218
Using a Bayesian network......Page 219
Unsupervised machine learning......Page 222
Association rule learning......Page 223
Using association rule learning to find buying relationships......Page 224
Reinforcement learning......Page 226
Summary......Page 228
7. Neural Networks......Page 229
Training a neural network......Page 232
Getting started with neural network architectures......Page 233
Understanding static neural networks......Page 235
A basic Java example......Page 236
Understanding dynamic neural networks......Page 243
Building the model......Page 244
Evaluating the model......Page 245
Predicting other values......Page 246
Saving and retrieving the model......Page 247
Learning vector quantization......Page 248
Using a SOM......Page 249
Displaying the SOM results......Page 250
Additional network architectures and algorithms......Page 254
The k-Nearest Neighbors algorithm......Page 255
Instantaneously trained networks......Page 256
Spiking neural networks......Page 257
Cascading neural networks......Page 258
Holographic associative memory......Page 259
Backpropagation and neural networks......Page 260
Summary......Page 261
8. Deep Learning......Page 262
Deeplearning4j architecture......Page 264
Reading in a CSV file......Page 265
Using hyperparameters in ND4J......Page 267
Instantiating the network model......Page 269
Training a model......Page 270
Testing a model......Page 271
Deep learning and regression analysis......Page 272
Preparing the data......Page 273
Setting up the class......Page 274
Reading and preparing the data......Page 275
Building the model......Page 276
Evaluating the model......Page 278
Restricted Boltzmann Machines......Page 280
Reconstruction in an RBM......Page 283
Configuring an RBM......Page 285
Deep autoencoders......Page 286
Configuring the network......Page 288
Saving and retrieving a network......Page 289
Specialized autoencoders......Page 290
Convolutional networks......Page 291
Building the model......Page 292
Evaluating the model......Page 295
Recurrent Neural Networks......Page 296
Summary......Page 297
9. Text Analysis......Page 298
Implementing named entity recognition......Page 300
Using OpenNLP to perform NER......Page 301
Identifying location entities......Page 303
Classifying text......Page 305
Word2Vec and Doc2Vec......Page 306
Classifying text by labels......Page 307
Classifying text by similarity......Page 310
Understanding tagging and POS......Page 313
Using OpenNLP to identify POS......Page 314
Understanding POS tags......Page 316
Extracting relationships from sentences......Page 318
Using OpenNLP to extract relationships......Page 319
Sentiment analysis......Page 321
Downloading and extracting the Word2Vec model......Page 322
Building our model and classifying text......Page 325
Summary......Page 328
10. Visual and Audio Analysis......Page 329
Text-to-speech......Page 330
Using FreeTTS......Page 332
Getting information about voices......Page 333
Gathering voice information......Page 335
Understanding speech recognition......Page 336
Using CMUPhinx to convert speech to text......Page 337
Obtaining more detail about the words......Page 339
Extracting text from an image......Page 341
Using Tess4j to extract text......Page 342
Identifying faces......Page 344
Using OpenCV to detect faces......Page 345
Classifying visual data......Page 347
Creating a Neuroph Studio project for classifying visual images......Page 348
Training the model......Page 355
Summary......Page 360
11. Mathematical and Parallel Techniques for Data Analysis......Page 361
Implementing basic matrix operations......Page 363
Using GPUs with DeepLearning4j......Page 365
Using map-reduce......Page 367
Using Apache's Hadoop to perform map-reduce......Page 368
Writing the map method......Page 369
Writing the reduce method......Page 370
Creating and executing a new Hadoop job......Page 371
Various mathematical libraries......Page 373
Using the jblas API......Page 374
Using the Apache Commons math API......Page 375
Using the ND4J API......Page 376
Using OpenCL......Page 378
Using Aparapi......Page 379
Creating an Aparapi application......Page 380
Using Aparapi for matrix multiplication......Page 383
Using Java 8 streams......Page 385
Understanding Java 8 lambda expressions and streams......Page 386
Using Java 8 to perform matrix multiplication......Page 387
Using Java 8 to perform map-reduce......Page 389
Summary......Page 391
12. Bringing It All Together......Page 392
Defining the purpose and scope of our application......Page 393
Understanding the application's architecture......Page 394
Data acquisition using Twitter......Page 398
Understanding the TweetHandler class......Page 400
Extracting data for a sentiment analysis model......Page 402
Building the sentiment model......Page 404
Processing the JSON input......Page 405
Cleaning data to improve our results......Page 407
Removing stop words......Page 408
Performing sentiment analysis......Page 409
Analysing the results......Page 410
Other optional enhancements......Page 411
Summary......Page 412