Data Science with Python

Cover......Page 1
Meet Your Course Guide......Page 3
Table of Contents......Page 10
Introduction and First Steps – Take a Deep Breath......Page 32
A proper introduction......Page 33
Enter the Python......Page 35
Developer productivity......Page 36
Satisfaction and enjoyment......Page 37
Setting up the environment......Page 38
Python 2 versus Python 3 – the great debate......Page 39
What you need for this course......Page 40
Installing IPython......Page 41
Installing additional packages......Page 43
How you can run a Python program......Page 44
Running the Python interactive shell......Page 45
Running Python as a GUI application......Page 47
How is Python code organized......Page 48
How do we use modules and packages......Page 49
Names and namespaces......Page 52
Scopes......Page 54
Guidelines on how to write good code......Page 57
The Python culture......Page 58
A note on the IDEs......Page 60
Introducing object-oriented......Page 62
Objects and classes......Page 64
Specifying attributes and behaviors......Page 66
Data describes objects......Page 67
Behaviors are actions......Page 68
Hiding details and creating the 
public interface......Page 70
Composition......Page 72
Inheritance......Page 75
Inheritance provides abstraction......Page 77
Multiple inheritance......Page 78
Case study......Page 79
Creating Python classes......Page 90
Adding attributes......Page 92
Talking to yourself......Page 93
More arguments......Page 94
Initializing the object......Page 96
Explaining yourself......Page 98
Modules and packages......Page 100
Absolute imports......Page 103
Relative imports......Page 104
Organizing module contents......Page 106
Who can access my data?......Page 109
Third-party libraries......Page 111
Case study......Page 112
Basic inheritance......Page 124
Extending built-ins......Page 127
Overriding and super......Page 128
Multiple inheritance......Page 130
The diamond problem......Page 132
Different sets of arguments......Page 137
Polymorphism......Page 140
Using an abstract base class......Page 143
Creating an abstract base class......Page 144
Demystifying the magic......Page 146
Case study......Page 147
Expecting the Unexpected......Page 164
Raising exceptions......Page 165
Raising an exception......Page 166
The effects of an exception......Page 168
Handling exceptions......Page 169
The exception hierarchy......Page 175
Defining our own exceptions......Page 176
Case study......Page 181
Treat objects as objects......Page 194
Adding behavior to class data with properties......Page 198
Properties in detail......Page 201
Decorators – another way to create properties......Page 203
Deciding when to use properties......Page 205
Manager objects......Page 207
Removing duplicate code......Page 209
In practice......Page 211
Case study......Page 214
Empty objects......Page 226
Tuples and named tuples......Page 228
Named tuples......Page 230
Dictionaries......Page 231
Using defaultdict......Page 235
Counter......Page 237
Lists......Page 238
Sorting lists......Page 240
Sets......Page 244
Extending built-ins......Page 248
Queues......Page 253
FIFO queues......Page 254
LIFO queues......Page 256
Priority queues......Page 257
Case study......Page 259
Python built-in functions......Page 268
Reversed......Page 269
Enumerate......Page 271
File I/O......Page 272
Placing it in context......Page 274
An alternative to method overloading......Page 276
Default arguments......Page 277
Variable argument lists......Page 279
Unpacking arguments......Page 283
Functions are objects too......Page 284
Using functions as attributes......Page 288
Callable objects......Page 289
Case study......Page 290
Strings......Page 300
String manipulation......Page 301
String formatting......Page 303
Escaping braces......Page 304
Keyword arguments......Page 305
Container lookups......Page 306
Object lookups......Page 307
Making it look right......Page 308
Strings are Unicode......Page 310
Converting bytes to text......Page 311
Converting text to bytes......Page 312
Mutable byte strings......Page 314
Regular expressions......Page 315
Matching patterns......Page 316
Matching a selection of characters......Page 317
Escaping characters......Page 318
Matching multiple characters......Page 319
Grouping patterns together......Page 320
Getting information from regular expressions......Page 321
Serializing objects......Page 323
Customizing pickles......Page 325
Serializing web objects......Page 328
Case study......Page 331
Design patterns in brief......Page 340
Iterators......Page 341
The iterator protocol......Page 342
List comprehensions......Page 344
Set and dictionary comprehensions......Page 347
Generator expressions......Page 348
Generators......Page 350
Yield items from another iterable......Page 352
Coroutines......Page 355
Back to log parsing......Page 358
Closing coroutines and throwing exceptions......Page 360
The relationship between coroutines, generators, and functions......Page 361
Case study......Page 362
The decorator pattern......Page 372
A decorator example......Page 373
Decorators in Python......Page 376
The observer pattern......Page 378
An observer example......Page 379
The strategy pattern......Page 381
A strategy example......Page 382
The state pattern......Page 384
A state example......Page 385
The singleton pattern......Page 391
Singleton implementation......Page 392
A template example......Page 396
The adapter pattern......Page 402
The facade pattern......Page 406
The flyweight pattern......Page 408
The command pattern......Page 412
The abstract factory pattern......Page 417
The composite pattern......Page 422
Why test?......Page 430
Test-driven development......Page 432
Unit testing......Page 433
Assertion methods......Page 435
Reducing boilerplate and cleaning up......Page 437
Organizing and running tests......Page 438
Ignoring broken tests......Page 439
Testing with py.test......Page 441
One way to do setup and cleanup......Page 443
A completely different way to set up variables......Page 446
Skipping tests with py.test......Page 450
Imitating expensive objects......Page 451
How much testing is enough?......Page 455
Case study......Page 458
Implementing it......Page 459
Concurrency......Page 468
Threads......Page 469
Shared memory......Page 472
The global interpreter lock......Page 473
Multiprocessing......Page 474
Multiprocessing pools......Page 476
Queues......Page 479
Futures......Page 481
AsyncIO......Page 484
AsyncIO in action......Page 485
Reading an AsyncIO future......Page 486
AsyncIO for networking......Page 487
Using executors to wrap blocking code......Page 490
Executors......Page 492
Case study......Page 493
Introducing Data Analysis and Libraries......Page 506
Data analysis and processing......Page 507
An overview of the libraries in data analysis......Page 510
NumPy......Page 513
PyMongo......Page 514
The scikit-learn library......Page 515
NumPy Arrays and Vectorized Computation......Page 518
Data types......Page 519
Array creation......Page 521
Indexing and slicing......Page 523
Fancy indexing......Page 524
Numerical operations on arrays......Page 525
Array functions......Page 526
Data processing using arrays......Page 528
Saving an array......Page 530
Linear algebra with NumPy......Page 531
NumPy random numbers......Page 533
An overview of the pandas package......Page 538
Series......Page 539
The DataFrame......Page 541
Reindexing and altering labels......Page 545
Head and tail......Page 546
Binary operations......Page 547
Functional statistics......Page 548
Function application......Page 550
Sorting......Page 551
Indexing and selecting data......Page 553
Computational tools......Page 555
Working with missing data......Page 556
Hierarchical indexing......Page 559
The Panel data......Page 562
Data Visualization......Page 566
The matplotlib API primer......Page 567
Line properties......Page 570
Figures and subplots......Page 572
Scatter plots......Page 575
Bar plots......Page 576
Contour plots......Page 577
Histogram plots......Page 578
Legends and annotations......Page 579
Plotting functions with pandas......Page 583
Bokeh......Page 586
MayaVi......Page 587
Time series primer......Page 590
Working with date and time objects......Page 591
Resampling time series......Page 599
Downsampling time series data......Page 600
Upsampling time series data......Page 602
Timedeltas......Page 606
Time series plotting......Page 607
Reading data from text format......Page 614
Writing data to text format......Page 619
Interacting with data in binary format......Page 620
HDF5......Page 621
Interacting with data in MongoDB......Page 622
The simple value......Page 627
List......Page 628
Set......Page 629
Ordered set......Page 630
Data Analysis Application Examples......Page 634
Data munging......Page 635
Cleaning data......Page 637
Filtering......Page 640
Merging data......Page 643
Reshaping data......Page 647
Data aggregation......Page 648
Grouping data......Page 651
Getting Started with 
Data Mining......Page 660
Introducing data mining......Page 661
What is affinity analysis?......Page 662
Loading the dataset with NumPy......Page 663
Implementing a simple ranking of rules......Page 665
Ranking to find the best rules......Page 668
What is classification?......Page 671
Loading and preparing the dataset......Page 672
Implementing the OneR algorithm......Page 673
Testing the algorithm......Page 676
scikit-learn estimators......Page 680
Nearest neighbors......Page 681
Distance metrics......Page 682
Loading the dataset......Page 684
Moving towards a standard workflow......Page 686
Running the algorithm......Page 687
Setting parameters......Page 688
Preprocessing using pipelines......Page 691
Standard preprocessing......Page 692
Pipelines......Page 693
Loading the dataset......Page 698
Collecting the data......Page 699
Using pandas to load the dataset......Page 700
Cleaning up the dataset......Page 701
Extracting new features......Page 702
Decision trees......Page 704
Parameters in decision trees......Page 705
Using decision trees......Page 706
Putting it all together......Page 707
Random forests......Page 711
How do ensembles work?......Page 712
Applying Random forests......Page 713
Engineering new features......Page 715
Affinity analysis......Page 718
Algorithms for affinity analysis......Page 719
Choosing parameters......Page 720
Loading with pandas......Page 721
Sparse data formats......Page 722
The Apriori implementation......Page 723
The Apriori algorithm......Page 725
Implementation......Page 726
Extracting association rules......Page 729
Evaluation......Page 733
Extracting Features with Transformers......Page 738
Representing reality in models......Page 739
Common feature patterns......Page 741
Creating good features......Page 744
Feature selection......Page 745
Selecting the best individual features......Page 747
Feature creation......Page 751
Creating your own transformer......Page 753
The transformer API......Page 754
Implementation details......Page 755
Unit testing......Page 756
Putting it all together......Page 758
Social Media Insight 
Using Naive Bayes......Page 760
Disambiguation......Page 761
Downloading data from a social network......Page 762
Loading and classifying the dataset......Page 764
Creating a replicable dataset from Twitter......Page 769
Text transformers......Page 773
Bag-of-words......Page 774
N-grams......Page 775
Naive Bayes......Page 776
Bayes\' theorem......Page 777
Naive Bayes algorithm......Page 778
How it works......Page 779
Application......Page 780
Extracting word counts......Page 781
Training the Naive Bayes classifier......Page 782
Putting it all together......Page 783
Evaluation using the F1-score......Page 784
Getting useful features from models......Page 785
Discovering Accounts to Follow Using Graph Mining......Page 790
Loading the dataset......Page 791
Classifying with an existing model......Page 792
Getting follower information from Twitter......Page 794
Building the network......Page 797
Creating a graph......Page 800
Creating a similarity graph......Page 802
Finding subgraphs......Page 806
Connected components......Page 807
Optimizing criteria......Page 810
Beating CAPTCHAs with Neural Networks......Page 818
Artificial neural networks......Page 819
An introduction to neural networks......Page 820
Creating the dataset......Page 822
Drawing basic CAPTCHAs......Page 823
Splitting the image into individual letters......Page 824
Creating a training dataset......Page 826
Adjusting our training dataset to our methodology......Page 828
Training and classifying......Page 829
Back propagation......Page 831
Predicting words......Page 832
Ranking mechanisms for words......Page 837
Putting it all together......Page 838
Authorship Attribution......Page 842
Applications and use cases......Page 843
Attributing authorship......Page 844
Getting the data......Page 846
Function words......Page 849
Counting function words......Page 850
Classifying with function words......Page 851
Support vector machines......Page 852
Kernels......Page 854
Character n-grams......Page 855
Extracting character n-grams......Page 856
Using the Enron dataset......Page 857
Creating a dataset loader......Page 858
Putting it all together......Page 863
Evaluation......Page 864
Obtaining news articles......Page 868
Using a Web API to get data......Page 869
Reddit as a data source......Page 872
Getting the data......Page 873
Extracting text from arbitrary websites......Page 875
Finding the stories in arbitrary websites......Page 876
Putting it all together......Page 877
Grouping news articles......Page 879
The k-means algorithm......Page 880
Evaluating the results......Page 883
Extracting topic information from clusters......Page 885
Using clustering algorithms as transformers......Page 886
Evidence accumulation......Page 887
How it works......Page 891
Implementation......Page 892
An introduction to online learning......Page 893
Implementation......Page 894
Classifying Objects in Images Using Deep Learning......Page 900
Application scenario and goals......Page 901
Use cases......Page 904
Deep neural networks......Page 905
Implementation......Page 906
An introduction to Theano......Page 907
An introduction to Lasagne......Page 908
Implementing neural networks with nolearn......Page 913
GPU optimization......Page 917
When to use GPUs for computation......Page 918
Running our code on a GPU......Page 919
Setting up the environment......Page 920
Getting the data......Page 923
Creating the neural network......Page 924
Putting it all together......Page 926
Working with Big Data......Page 930
Big data......Page 931
Application scenario and goals......Page 932
Intuition......Page 934
A word count example......Page 936
Hadoop MapReduce......Page 937
Application......Page 938
Getting the data......Page 939
Extracting the blog posts......Page 941
Training Naive Bayes......Page 944
Putting it all together......Page 947
Training on Amazon\'s EMR infrastructure......Page 952
Scikit-learn tutorials......Page 958
Comparing classifiers......Page 959
The Eclat algorithm......Page 960
More complex algorithms......Page 961
Local n-grams......Page 962
Mahotas......Page 963
More resources......Page 964
Giving Computers the Ability to Learn from Data......Page 970
The three different types of 
machine learning......Page 971
Classification for predicting class labels......Page 972
Regression for predicting continuous outcomes......Page 973
Discovering hidden structures with unsupervised learning......Page 975
Dimensionality reduction for data compression......Page 976
An introduction to the basic terminology and notations......Page 978
A roadmap for building machine learning systems......Page 980
Training and selecting a predictive model......Page 981
Using Python for machine learning......Page 982
Training Machine Learning Algorithms for Classification......Page 986
Artificial neurons – a brief glimpse into the early history of machine learning......Page 987
Implementing a perceptron learning algorithm in Python......Page 993
Training a perceptron model on the Iris dataset......Page 996
Adaptive linear neurons and the convergence of learning......Page 1002
Minimizing cost functions with gradient descent......Page 1003
Implementing an Adaptive Linear Neuron in Python......Page 1005
Large scale machine learning and stochastic gradient descent......Page 1011
Choosing a classification algorithm......Page 1018
Training a perceptron via scikit-learn......Page 1019
Logistic regression intuition and conditional probabilities......Page 1025
Learning the weights of the logistic cost function......Page 1029
Training a logistic regression model with scikit-learn......Page 1031
Tackling overfitting via regularization......Page 1034
Maximum margin classification with support vector machines......Page 1038
Maximum margin intuition......Page 1039
Dealing with the nonlinearly separable case using slack variables......Page 1040
Alternative implementations in scikit-learn......Page 1043
Solving nonlinear problems using a kernel SVM......Page 1044
Using the kernel trick to find separating hyperplanes in higher dimensional space......Page 1046
Decision tree learning......Page 1049
Maximizing information gain – getting the most bang for the buck......Page 1051
Building a decision tree......Page 1057
Combining weak to strong learners via random forests......Page 1059
K-nearest neighbors – a lazy learning algorithm......Page 1061
Dealing with missing data......Page 1068
Eliminating samples or features with missing values......Page 1070
Understanding the scikit-learn estimator API......Page 1071
Handling categorical data......Page 1073
Encoding class labels......Page 1074
Performing one-hot encoding on nominal features......Page 1076
Partitioning a dataset in training and test sets......Page 1077
Bringing features onto the same scale......Page 1079
Selecting meaningful features......Page 1081
Sparse solutions with L1 regularization......Page 1082
Sequential feature selection algorithms......Page 1088
Assessing feature importance with random forests......Page 1094
Compressing Data via Dimensionality Reduction......Page 1098
Unsupervised dimensionality reduction via principal component analysis......Page 1099
Total and explained variance......Page 1100
Feature transformation......Page 1104
Principal component analysis in scikit-learn......Page 1106
Supervised data compression via linear discriminant analysis......Page 1109
Computing the scatter matrices......Page 1111
Selecting linear discriminants for the new feature subspace......Page 1114
Projecting samples onto the new feature space......Page 1116
LDA via scikit-learn......Page 1117
Kernel functions and the kernel trick......Page 1119
Implementing a kernel principal component analysis in Python......Page 1125
Example 1 – separating half-moon shapes......Page 1126
Example 2 – separating concentric circles......Page 1130
Projecting new data points......Page 1133
Kernel principal component analysis in 
scikit-learn......Page 1137
Streamlining workflows with pipelines......Page 1140
Loading the Breast Cancer Wisconsin dataset......Page 1141
Combining transformers and estimators in a pipeline......Page 1142
The holdout method......Page 1144
K-fold cross-validation......Page 1146
Debugging algorithms with learning and validation curves......Page 1150
Diagnosing bias and variance problems with learning curves......Page 1151
Addressing overfitting and underfitting with validation curves......Page 1154
Fine-tuning machine learning models via grid search......Page 1156
Tuning hyperparameters via grid search......Page 1157
Algorithm selection with nested 
cross-validation......Page 1158
Looking at different performance evaluation metrics......Page 1160
Reading a confusion matrix......Page 1161
Optimizing the precision and recall of a classification model......Page 1162
Plotting a receiver operating characteristic......Page 1164
The scoring metrics for multiclass classification......Page 1168
Learning with ensembles......Page 1172
Implementing a simple majority vote classifier......Page 1176
Combining different algorithms for classification with majority vote......Page 1183
Evaluating and tuning the ensemble classifier......Page 1186
Bagging – building an ensemble of classifiers from bootstrap samples......Page 1192
Leveraging weak learners via adaptive boosting......Page 1197
Predicting Continuous 
Target Variables with Regression Analysis......Page 1208
Introducing a simple linear regression model......Page 1209
Exploring the Housing Dataset......Page 1210
Visualizing the important characteristics of a dataset......Page 1211
Solving regression for regression parameters with gradient descent......Page 1216
Estimating the coefficient of a regression model via scikit-learn......Page 1220
Fitting a robust regression model using RANSAC......Page 1222
Evaluating the performance of linear regression models......Page 1225
Using regularized methods for regression......Page 1228
Turning a linear regression model into a curve – polynomial regression......Page 1230
Modeling nonlinear relationships in the Housing Dataset......Page 1232
Dealing with nonlinear relationships using random forests......Page 1235
Decision tree regression......Page 1236
Random forest regression......Page 1238
Chapter 2: Object-oriented Design......Page 1244
Chapter 6: Interacting with Databases......Page 1245
Chapter 3: Predicting Sports Winners with Decision Trees......Page 1246
Chapter 8: Beating CAPTCHAs with Neural Networks......Page 1247
Chapter 1: Giving Computers the Ability to Learn from Data......Page 1248
Chapter 6: Learning Best Practices for Model Evaluation and Hyperparameter Tuning......Page 1249
Chapter 8: Predicting Continuous Target Variables with Regression Analysis......Page 1250