Introduction to Data Mining

Front Cover
Title Page
Copyright Page
Dedication
Preface to the Second Edition
Contents
1 Introduction
	1.1 What Is Data Mining?
	1.2 Motivating Challenges
	1.3 the Origins of Data Mining
	1.4 Data Mining Tasks
	1.5 Scope and Organization of the Book
	1.6 Bibliographic Notes
	1.7 Exercises
2 Data
	2.1 Types of Data
		2.1.1 Attributes and Measurement
		2.1.2 Types of Data Sets
	2.2 Data Quality
		2.2.1 Measurement and Data Collection Issues
		2.2.2 Issues Related to Applications
	2.3 Data Preprocessing
		2.3.1 Aggregation
		2.3.2 Sampling
		2.3.3 Dimensionality Reduction
		2.3.4 Feature Subset Selection
		2.3.5 Feature Creation
		2.3.6 Discretization and Binarization
		2.3.7 Variable Transformation
	2.4 Measures of Similarity and Dissimilarity
		2.4.1 Basics
		2.4.2 Similarity and Dissimilarity Between Simple Attributes
		2.4.3 Dissimilarities Between Data Objects
		2.4.4 Similarities Between Data Objects
		2.4.5 Examples of Proximity Measures
		2.4.6 Mutual Information
		2.4.7 Kernel Functions*
		2.4.8 Bregman Divergence*
		2.4.9 Issues in Proximity Calculation
		2.4.10 Selecting the Right Proximity Measure
	2.5 Bibliographic Notes
	2.6 Exercises
3 Classification: Basic Concepts and Techniques
	3.1 Basic Concepts
	3.2 General Framework for Classification
	3.3 Decision Tree Classifier
		3.3.1 A Basic Algorithm to Build a Decision Tree
		3.3.2 Methods for Expressing Attribute Test Conditions
		3.3.3 Measures for Selecting an Attribute Test Condition
		3.3.4 Algorithm for Decision Tree Induction
		3.3.5 Example Application: Web Robot Detection
		3.3.6 Characteristics of Decision Tree Classifiers
	3.4 Model Overfitting
		3.4.1 Reasons for Model Overfitting
	3.5 Model Selection
		3.5.1 Using a Validation Set
		3.5.2 Incorporating Model Complexity
		3.5.3 Estimating Statistical Bounds
		3.5.4 Model Selection for Decision Trees
	3.6 Model Evaluation
		3.6.1 Holdout Method
		3.6.2 Cross-validation
	3.7 Presence of Hyper-parameters
		3.7.1 Hyper-parameter Selection
		3.7.2 Nested Cross-validation
	3.8 Pitfalls of Model Selection and Evaluation
		3.8.1 Overlap Between Training and Test Sets
		3.8.2 Use of Validation Error as Generalization Error
	3.9 Model Comparison*
		3.9.1 Estimating the Confidence Interval for Accuracy
		3.9.2 Comparing the Performance of Two Models
	3.10 Bibliographic Notes
	3.11 Exercises
4 Association Analysis: Basic Concepts and Algorithms
	4.1 Preliminaries
	4.2 Frequent Itemset Generation
		4.2.1 The Apriori Principle
		4.2.2 Frequent Itemset Generation in the Algorithm
		4.2.3 Candidate Generation and Pruning
		4.2.4 Support Counting
		4.2.5 Computational Complexity
	4.3 Rule Generation
		4.3.1 Confidence-based Pruning
		4.3.2 Rule Generation in Algorithm
		4.3.3 an Example: Congressional Voting Records
	4.4 Compact Representation of Frequent Itemsets
		4.4.1 Maximal Frequent Itemsets
		4.4.2 Closed Itemsets
	4.5 Alternative Methods for Generating Frequent Itemsets*
	4.6 FP-Growth Algorithm*
		4.6.1 FP-Tree Representation
		4.6.2 Frequent Itemset Generation in FP-Growth Algorithm
	4.7 Evaluation of Association Patterns
		4.7.1 Objective Measures of Interestingness
		4.7.2 Measures Beyond Pairs of Binary Variables
		4.7.3 Simpson’s Paradox
	4.8 Effect of Skewed Support Distribution
	4.9 Bibliographic Notes
	4.10 Exercises
5 Cluster Analysis: Basic Concepts and Algorithms
	5.1 Overview
		5.1.1 What Is Cluster Analysis?
		5.1.2 Different Types of Clusterings
		5.1.3 Different Types of Clusters
		Road Map
	5.2 K-means
		5.2.1 The Basic K-means Algorithm
		5.2.2 K-means: Additional Issues
		5.2.3 Bisecting K-means
		5.2.4 K-means and Different Types of Clusters
		5.2.5 Strengths and Weaknesses
		5.2.6 K-means as an Optimization Problem
	5.3 Agglomerative Hierarchical Clustering
		5.3.1 Basic Agglomerative Hierarchical Clustering Algorithm
		5.3.2 Specific Techniques
		5.3.3 The Lance-williams Formula for Cluster Proximity
		5.3.4 Key Issues in Hierarchical Clustering
		5.3.5 Outliers
		5.3.6 Strengths and Weaknesses
	5.4 DBSCAN
		5.4.1 Traditional Density: Center-based Approach
		5.4.2 The Dbscan Algorithm
		5.4.3 Strengths and Weaknesses
	5.5 Cluster Evaluation
		5.5.1 Overview
		5.5.2 Unsupervised Cluster Evaluation Using Cohesion and Separation
		5.5.3 Unsupervised Cluster Evaluation Using the Proximity Matrix
		5.5.4 Unsupervised Evaluation of Hierarchical Clustering
		5.5.5 Determining the Correct Number of Clusters
		5.5.6 Clustering Tendency
		5.5.7 Supervised Measures of Cluster Validity
		5.5.8 Assessing the Significance of Cluster Validity Measures
		5.5.9 Choosing a Cluster Validity Measure
	5.6 Bibliographic Notes
	5.7 Exercises
6 Classification: Alternative Techniques
	6.1 Types of Classifiers
	6.2 Rule-Based Classifier
		6.2.1 How a Rule-Based Classifier Works
		6.2.2 Properties of a Rule Set
		6.2.3 Direct Methods for Rule Extraction
		6.2.4 Indirect Methods for Rule Extraction
		6.2.5 Characteristics of Rule-Based Classifiers
	6.3 Nearest Neighbor Classifiers
		6.3.1 Algorithm
		6.3.2 Characteristics of Nearest Neighbor Classifiers
	6.4 Na¨ive Bayes Classifier
		6.4.1 Basics of Probability Theory
		6.4.2 Na¨ive Bayes Assumption
	6.5 Bayesian Networks
		6.5.1 Graphical Representation
		6.5.2 Inference and Learning
		6.5.3 Characteristics of Bayesian Networks
	6.6 Logistic Regression
		6.6.1 Logistic Regression as a Generalized Linear Model
		6.6.2 Learning Model Parameters
		6.6.3 Characteristics of Logistic Regression
	6.7 Artificial Neural Network (ann)
		6.7.1 Perceptron
		6.7.2 Multi-layer Neural Network
		6.7.3 Characteristics of Ann
	6.8 Deep Learning
		6.8.1 Using Synergistic Loss Functions
		6.8.2 Using Responsive Activation Functions
		6.8.3 Regularization
		6.8.4 Initialization of Model Parameters
		6.8.5 Characteristics of Deep Learning
	6.9 Support Vector Machine (svm)
		6.9.1 Margin of a Separating Hyperplane
		6.9.2 Linear SVM
		6.9.3 Soft-margin SVM
		6.9.4 Nonlinear SVM
		6.9.5 Characteristics of SVM
	6.10 Ensemble Methods
		6.10.1 Rationale for Ensemble Method
		6.10.2 Methods for Constructing an Ensemble Classifier
		6.10.3 Bias-Variance Decomposition
		6.10.4 Bagging
		6.10.5 Boosting
		6.10.6 Random Forests
		6.10.7 Empirical Comparison Among Ensemble Methods
	6.11 Class Imbalance Problem
		6.11.1 Building Classifiers with Class Imbalance
		6.11.2 Evaluating Performance with Class Imbalance
		6.11.3 Finding an Optimal Score Threshold
		6.11.4 Aggregate Evaluation of Performance
	6.12 Multiclass Problem
	6.13 Bibliographic Notes
	6.14 Exercises
7 Association Analysis: Advanced Concepts
	7.1 Handling Categorical Attributes
	7.2 Handling Continuous Attributes
		7.2.1 Discretization-Based Methods
		7.2.2 Statistics-Based Methods
		7.2.3 Non-Discretization Methods
	7.3 Handling a Concept Hierarchy
	7.4 Sequential Patterns
		7.4.1 Preliminaries
		7.4.2 Sequential Pattern Discovery
		7.4.3 Timing Constraints*
		7.4.4 Alternative Counting Schemes*
	7.5 Subgraph Patterns
		7.5.1 Preliminaries
		7.5.2 Frequent Subgraph Mining
		7.5.3 Candidate Generation
		7.5.4 Candidate Pruning
		7.5.5 Support Counting
	7.6 Infrequent Patterns*
		7.6.1 Negative Patterns
		7.6.2 Negatively Correlated Patterns
		7.6.3 Comparisons Among Infrequent Patterns, Negative Patterns, and Negatively Correlated Patterns
		7.6.4 Techniques for Mining Interesting Infrequent Patterns
		7.6.5 Techniques Based on Mining Negative Patterns
		7.6.6 Techniques Based on Support Expectation
	7.7 Bibliographic Notes
	7.8 Exercises
8 Cluster Analysis: Additional Issues and Algorithms
	8.1 Characteristics of Data, Clusters, and Clustering Algorithms
		8.1.1 Example: Comparing K-means and Dbscan
		8.1.2 Data Characteristics
		8.1.3 Cluster Characteristics
		8.1.4 General Characteristics of Clustering Algorithms
		Road Map
	8.2 Prototype-based Clustering
		8.2.1 Fuzzy Clustering
		8.2.2 Clustering Using Mixture Models
		8.2.3 Self-organizing Maps (SOM)
	8.3 Density-Based Clustering
		8.3.1 Grid-Based Clustering
		8.3.2 Subspace Clustering
		8.3.3 Denclue: A Kernel-Based Scheme for Density-based Clustering
	8.4 Graph-Based Clustering
		8.4.1 Sparsification
		8.4.2 Minimum Spanning Tree (MST) Clustering
		8.4.3 Opossum: Optimal Partitioning of Sparse Similarities Using Metis
		8.4.4 Chameleon: Hierarchical Clustering with Dynamic Modeling
		8.4.5 Spectral Clustering
		8.4.6 Shared Nearest Neighbor Similarity
		8.4.7 the Jarvis-patrick Clustering Algorithm
		8.4.8 SNN Density
		8.4.9 SNN Density-Based Clustering
	8.5 Scalable Clustering Algorithms
		8.5.1 Scalability: General Issues and Approaches
		8.5.2 Birch
		8.5.3 Cure
	8.6 Which Clustering Algorithm?
	8.7 Bibliographic Notes
	8.8 Exercises
9 Anomaly Detection
	9.1 Characteristics of Anomaly Detection Problems
		9.1.1 A Definition of an Anomaly
		9.1.2 Nature of Data
		9.1.3 How Anomaly Detection is Used
	9.2 Characteristics of Anomaly Detection Methods
	9.3 Statistical Approaches
		9.3.1 Using Parametric Models
		9.3.2 Using Non-Parametric Models
		9.3.3 Modeling Normal and Anomalous Classes
		9.3.4 Assessing Statistical Significance
		9.3.5 Strengths and Weaknesses
	9.4 Proximity-Based Approaches
		9.4.1 Distance-Based Anomaly Score
		9.4.2 Density-Based Anomaly Score
		9.4.3 Relative Density-Based Anomaly Score
		9.4.4 Strengths and Weaknesses
	9.5 Clustering-Based Approaches
		9.5.1 Finding Anomalous Clusters
		9.5.2 Finding Anomalous Instances
		9.5.3 Strengths and Weaknesses
	9.6 Reconstruction-Based Approaches
		9.6.1 Strengths and Weaknesses
	9.7 One-Class Classification
		9.7.1 Use of Kernels
		9.7.2 The Origin Trick
		9.7.3 Strengths and Weaknesses
	9.8 Information Theoretic Approaches
		9.8.1 Strengths and Weaknesses
	9.9 Evaluation of Anomaly Detection
	9.10 Bibliographic Notes
	9.11 Exercises
10 Avoiding False Discoveries
	10.1 Preliminaries: Statistical Testing
		10.1.1 Significance Testing
		10.1.2 Hypothesis Testing
		10.1.3 Multiple Hypothesis Testing
		10.1.4 Pitfalls in Statistical Testing
	10.2 Modeling Null and Alternative Distributions
		10.2.1 Generating Synthetic Data Sets
		10.2.2 Randomizing Class Labels
		10.2.3 Resampling Instances
		10.2.4 Modeling the Distribution of the Test Statistic
	10.3 Statistical Testing for Classification
		10.3.1 Evaluating Classification Performance
		10.3.2 Binary Classification as Multiple Hypothesis Testing
		10.3.3 Multiple Hypothesis Testing in Model Selection
	10.4 Statistical Testing for Association Analysis
		10.4.1 Using Statistical Models
		10.4.2 Using Randomization Methods
	10.5 Statistical Testing for Cluster Analysis
		10.5.1 Generating a Null Distribution for Internal Indices
		10.5.2 Generating a Null Distribution for External Indices
		10.5.3 Enrichment
	10.6 Statistical Testing for Anomaly Detection
	10.7 Bibliographic Notes
	10.8 Exercises
Author Index
Subject Index
Copyright Permissions
Back Cover