Machine Learning: The Art and Science of Algorithms that Make Sense of Data

Cover
......Page 1
MACHINE LEARNING: The Art and Science of Algorithms that Make Sense of Data......Page 3
Title......Page 5
Copyright......Page 6
Dedication......Page 7
Brief Contents......Page 9
Contents......Page 11
Preface......Page 17
Acknowledgements......Page 18
Prologue: A machine learning sampler......Page 21
CHAPTER 1 The ingredients of machine learning......Page 33
1.1 Tasks: the problems that can be solved with machine learning......Page 34
Looking for structure......Page 36
Evaluating performance on a task......Page 38
1.2 Models: the output of machine learning......Page 40
Geometric models......Page 41
Probabilistic models......Page 45
Logical models......Page 52
Grouping and grading......Page 56
1.3 Features: the workhorses of machine learning......Page 58
Two uses of features......Page 60
Feature construction and transformation......Page 61
Interaction between features......Page 64
1.4 Summary and outlook......Page 66
What you’ll find in the rest of the book......Page 68
CHAPTER 2 Binary classification and related tasks......Page 69
2.1 Classification......Page 72
Assessing classification performance......Page 73
Visualising classification performance......Page 78
2.2 Scoring and ranking......Page 81
Assessing and visualising ranking performance......Page 83
Turning rankers into classifiers......Page 89
2.3 Class probability estimation......Page 92
Assessing class probability estimates......Page 93
Turning rankers into class probability estimators......Page 96
2.4 Binary classification and related tasks: Summary and further reading......Page 99
3.1 Handling more than two classes......Page 101
Multi-class classification......Page 102
Multi-class scores and probabilities......Page 106
3.2 Regression......Page 111
3.3 Unsupervised and descriptive learning......Page 115
Predictive and descriptive clustering......Page 116
Other descriptive models......Page 120
3.4 Beyond binary classification: Summary and further reading......Page 122
CHAPTER 4 Concept learning......Page 124
4.1 The hypothesis space......Page 126
Least general generalisation......Page 128
Internal disjunction......Page 130
4.2 Paths through the hypothesis space......Page 132
Closed concepts......Page 136
4.3 Beyond conjunctive concepts......Page 139
Using first-order logic......Page 142
4.4 Learnability......Page 144
4.5 Concept learning: Summary and further reading......Page 147
CHAPTER 5 Tree models......Page 149
5.1 Decision trees......Page 153
5.2 Ranking and probability estimation trees......Page 158
Sensitivity to skewed class distributions......Page 163
Regression trees......Page 168
Clustering trees......Page 172
5.4 Tree models: Summary and further reading......Page 175
CHAPTER 6 Rule models......Page 177
6.1 Learning ordered rule lists......Page 178
Rule lists for ranking and probability estimation......Page 184
6.2 Learning unordered rule sets......Page 187
Rule sets for ranking and probability estimation......Page 193
A closer look at rule overlap......Page 194
6.3 Descriptive rule learning......Page 196
Rule learning for subgroup discovery......Page 198
Association rule mining......Page 202
6.4 First-order rule learning......Page 209
6.5 Rule models: Summary and further reading......Page 212
CHAPTER 7 Linear models......Page 214
7.1 The least-squares method......Page 216
Multivariate linear regression......Page 221
Regularised regression......Page 224
Using least-squares regression for classification......Page 225
7.2 The perceptron......Page 227
7.3 Support vector machines......Page 231
Soft margin SVM......Page 236
7.4 Obtaining probabilities from linear classifiers......Page 239
7.5 Going beyond linearity with kernel methods......Page 244
7.6 Linear models: Summary and further reading......Page 248
8.1 So many roads. . .......Page 251
8.2 Neighbours and exemplars......Page 257
8.3 Nearest-neighbour classification......Page 262
8.4 Distance-based clustering......Page 265
K-means algorithm......Page 267
Clustering around medoids......Page 270
Silhouettes......Page 272
8.5 Hierarchical clustering......Page 273
8.6 From kernels to distances......Page 278
8.7 Distance-based models: Summary and further reading......Page 280
CHAPTER 9 Probabilistic models......Page 282
9.1 The normal distribution and its geometric interpretations......Page 286
9.2 Probabilistic models for categorical data......Page 293
Using a naive Bayes model for classification......Page 295
Training a naive Bayes model......Page 299
9.3 Discriminative learning by optimising conditional likelihood......Page 302
9.4 Probabilistic models with hidden variables......Page 306
Expectation-Maximisation......Page 308
Gaussian mixture models......Page 309
9.5 Compression-based models......Page 312
9.6 Probabilistic models: Summary and further reading......Page 315
CHAPTER 10 Features......Page 318
Calculations on features......Page 319
Categorical, ordinal and quantitative features......Page 324
Structured features......Page 325
10.2 Feature transformations......Page 327
Thresholding and discretisation......Page 328
Normalisation and calibration......Page 334
Incomplete features......Page 341
10.3 Feature construction and selection......Page 342
Matrix transformations and decompositions......Page 344
10.4 Features: Summary and further reading......Page 347
CHAPTER 11 Model ensembles......Page 350
11.1 Bagging and random forests......Page 351
11.2 Boosting......Page 354
Boosted rule learning......Page 357
Bias, variance and margins......Page 358
Other ensemble methods......Page 359
Meta-learning......Page 360
11.4 Model ensembles: Summary and further reading......Page 361
CHAPTER 12 Machine learning experiments......Page 363
12.1 What to measure......Page 364
12.2 How to measure it......Page 368
12.3 How to interpret it......Page 371
Interpretation of results over multiple data sets......Page 374
12.4 Machine learning experiments: Summary and further reading......Page 377
Epilogue: Where to go from here......Page 380
Important points to remember......Page 383
References......Page 387
Index......Page 403