Data Preparation for Data Mining Using SAS (The Morgan Kaufmann Series in Data Management Systems)

0123735777......Page 1
Copyright Page......Page 5
Contents......Page 6
List of Figures......Page 16
List of Tables......Page 18
Preface......Page 22
1.2 Methodologies of Data Mining......Page 26
1.3 The Mining View......Page 28
1.5 Notes on Data Mining Software......Page 29
2.1 Data Mining Tasks......Page 32
2.2 Data Mining Competencies......Page 34
2.3 The Data Flow......Page 35
2.4 Types of Variables......Page 36
2.5 The Mining View and the Scoring View......Page 37
2.6 Steps of Data Preparation......Page 38
3.2 Regression Models......Page 40
3.3 Decision Trees......Page 46
3.4 Neural Networks......Page 47
3.5 Cluster Analysis......Page 50
3.8 Support Vector Machines......Page 51
4.1 Introduction:Why Macros?......Page 54
4.2 The Basics: The Macro and Its Variables......Page 55
4.3 Doing Calculations......Page 57
4.4 Programming Logic......Page 58
4.5 Working with Strings......Page 60
4.6 Macros That Call Other Macros......Page 61
4.7 Common Macro Patterns and Caveats......Page 62
4.8 Where to Go From Here......Page 66
5.2 Sources of Data......Page 68
5.3 Variable Types......Page 70
5.4 Data Rollup......Page 72
5.5 Rollup with Sums, Averages, and Counts......Page 79
5.6 Calculation of the Mode......Page 80
5.7 Data Integration......Page 81
6.1 Introduction......Page 88
6.3 Dataset Schema Checks......Page 91
6.4 Nominal Variables......Page 95
6.5 Continuous Variables......Page 101
7.2 Common EDA Procedures......Page 108
7.3 Univariate Statistics......Page 109
7.5 Detection of Outliers......Page 111
7.6 Testing Normality......Page 121
7.8 Investigating Data Structures......Page 122
8.1 Introduction......Page 124
8.2 Contents of Samples......Page 125
8.3 Random Sampling......Page 126
8.4 Balanced Sampling......Page 129
8.5 Minimum Sample Size......Page 135
8.6 Checking Validity of Sample......Page 138
9.1 Raw and Analytical Variables......Page 140
9.2 Scope of Data Transformations......Page 141
9.3 Creation of New Variables......Page 144
9.4 Mapping of Nominal Variables......Page 151
9.5 Normalization of Continuous Variables......Page 155
9.6 Changing the Variable Distribution......Page 156
10.1 Introduction......Page 166
10.2 Cardinality Reduction......Page 167
10.3 Binning of Continuous Variables......Page 182
11.1 Introduction......Page 196
11.2 Simple Replacement......Page 199
11.3 Imputing Missing Values......Page 204
11.4 Imputation Methods and Strategy......Page 206
11.5 SAS Macros for Multiple Imputation......Page 210
11.6 Predicting Missing Values......Page 229
12.1 Introduction......Page 232
12.2 Metrics of Predictive Power......Page 233
12.3 Methods of Variable Reduction......Page 234
12.4 Variable Reduction: Before or During Modeling......Page 235
13.2 Contingency Tables......Page 236
13.3 Notation and Definitions......Page 237
13.4 Contingency Tables for Binary Variables......Page 239
13.5 Contingency Tables for Multicategory Variables......Page 250
13.6 Analysis of Ordinal Variables......Page 252
13.7 Implementation Scenarios......Page 256
14.2 When Is Binning Necessary?......Page 258
14.3 Measures of Association......Page 259
14.4 Correlation Coefficients......Page 264
15.1 Introduction......Page 272
15.2 Mathematical Formulations......Page 273
15.3 Implementing and Using PCA......Page 274
15.4 Comments on Using PCA......Page 279
16.1 Introduction......Page 282
16.3 Implementation of Factor Analysis......Page 288
17.2 Data with Binary Dependent Variables......Page 292
17.4 Variable Reduction Strategies......Page 300
18.2 The Process of Data Preparation......Page 304
18.3 Case Study: The Bookstore......Page 306
A.1 Copyright and Software License......Page 322
A.2 Dependencies between Macros......Page 323
A.3 Data Acquisition and Integration......Page 324
A.4 Integrity Checks......Page 329
A.5 Exploratory Data Analysis......Page 335
A.6 Sampling and Partitioning......Page 338
A.7 Data Transformations......Page 343
A.8 Binning and Reduction of Cardinality......Page 350
A.9 Treatment of Missing Values......Page 366
A.10 Analysis of Nominal and Ordinal Variables......Page 377
A.11 Analysis of Continuous Variables......Page 383
A.12 Principal Component Analysis......Page 385
A.13 Factor Analysis......Page 387
A.14 Predictive Power and Variable Reduction II......Page 388
A.15 Other Macros......Page 397
Bibliography......Page 398
Index......Page 400
About the Author......Page 418