Text Data Mining

Foreword
Preface
Acknowledgments
Contents
About the Authors
Acronyms
1 Introduction
	1.1 The Basic Concepts
	1.2 Main Tasks of Text Data Mining
	1.3 Existing Challenges in Text Data Mining
	1.4 Overview and Organization of This Book
	1.5 Further Reading
	Exercises
2 Data Annotation and Preprocessing
	2.1 Data Acquisition
	2.2 Data Preprocessing
	2.3 Data Annotation
	2.4 Basic Tools of NLP
		2.4.1 Tokenization and POS Tagging
		2.4.2 Syntactic Parser
		2.4.3 N-gram Language Model
	2.5 Further Reading
	Exercises
3 Text Representation
	3.1 Vector Space Model
		3.1.1 Basic Concepts
		3.1.2 Vector Space Construction
		3.1.3 Text Length Normalization
		3.1.4 Feature Engineering
		3.1.5 Other Text Representation Methods
	3.2 Distributed Representation of Words
		3.2.1 Neural Network Language Model
		3.2.2 C&W Model
		3.2.3 CBOW and Skip-Gram Model
		3.2.4 Noise Contrastive Estimation and Negative Sampling
		3.2.5 Distributed Representation Based on the Hybrid Character-Word Method
	3.3 Distributed Representation of Phrases
		3.3.1 Distributed Representation Based on the Bag-of-Words Model
		3.3.2 Distributed Representation Based on Autoencoder
	3.4 Distributed Representation of Sentences
		3.4.1 General Sentence Representation
		3.4.2 Task-Oriented Sentence Representation
	3.5 Distributed Representation of Documents
		3.5.1 General Distributed Representation of Documents
		3.5.2 Task-Oriented Distributed Representation of Documents
	3.6 Further Reading
	Exercises
4 Text Representation with Pretraining and Fine-Tuning
	4.1 ELMo: Embeddings from Language Models
		4.1.1 Pretraining Bidirectional LSTM Language Models
		4.1.2 Contextualized ELMo Embeddings for Downstream Tasks
	4.2 GPT: Generative Pretraining
		4.2.1 Transformer
		4.2.2 Pretraining the Transformer Decoder
		4.2.3 Fine-Tuning the Transformer Decoder
	4.3 BERT: Bidirectional Encoder Representations from Transformer
		4.3.1 BERT: Pretraining
		4.3.2 BERT: Fine-Tuning
		4.3.3 XLNet: Generalized Autoregressive Pretraining
		4.3.4 UniLM
	4.4 Further Reading
	Exercises
5 Text Classification
	5.1 The Traditional Framework of Text Classification
	5.2 Feature Selection
		5.2.1 Mutual Information
		5.2.2 Information Gain
		5.2.3 The Chi-Squared Test Method
		5.2.4 Other Methods
	5.3 Traditional Machine Learning Algorithms for Text Classification
		5.3.1 Naïve Bayes
		5.3.2 Logistic/Softmax and Maximum Entropy
		5.3.3 Support Vector Machine
		5.3.4 Ensemble Methods
	5.4 Deep Learning Methods
		5.4.1 Multilayer Feed-Forward Neural Network
		5.4.2 Convolutional Neural Network
		5.4.3 Recurrent Neural Network
	5.5 Evaluation of Text Classification
	5.6 Further Reading
	Exercises
6 Text Clustering
	6.1 Text Similarity Measures
		6.1.1 The Similarity Between Documents
		6.1.2 The Similarity Between Clusters
	6.2 Text Clustering Algorithms
		6.2.1 K-Means Clustering
		6.2.2 Single-Pass Clustering
		6.2.3 Hierarchical Clustering
		6.2.4 Density-Based Clustering
	6.3 Evaluation of Clustering
		6.3.1 External Criteria
		6.3.2 Internal Criteria
	6.4 Further Reading
	Exercises
7 Topic Model
	7.1 The History of Topic Modeling
	7.2 Latent Semantic Analysis
		7.2.1 Singular Value Decomposition of the Term-by-Document Matrix
		7.2.2 Conceptual Representation and Similarity Computation
	7.3 Probabilistic Latent Semantic Analysis
		7.3.1 Model Hypothesis
		7.3.2 Parameter Learning
	7.4 Latent Dirichlet Allocation
		7.4.1 Model Hypothesis
		7.4.2 Joint Probability
		7.4.3 Inference in LDA
		7.4.4 Inference for New Documents
	7.5 Further Reading
	Exercises
8 Sentiment Analysis and Opinion Mining
	8.1 History of Sentiment Analysis and Opinion Mining
	8.2 Categorization of Sentiment Analysis Tasks
		8.2.1 Categorization According to Task Output
		8.2.2 According to Analysis Granularity
	8.3 Methods for Document/Sentence-Level Sentiment Analysis
		8.3.1 Lexicon- and Rule-Based Methods
		8.3.2 Traditional Machine Learning Methods
		8.3.3 Deep Learning Methods
	8.4 Word-Level Sentiment Analysis and Sentiment Lexicon Construction
		8.4.1 Knowledgebase-Based Methods
		8.4.2 Corpus-Based Methods
		8.4.3 Evaluation of Sentiment Lexicons
	8.5 Aspect-Level Sentiment Analysis
		8.5.1 Aspect Term Extraction
		8.5.2 Aspect-Level Sentiment Classification
		8.5.3 Generative Modeling of Topics and Sentiments
	8.6 Special Issues in Sentiment Analysis
		8.6.1 Sentiment Polarity Shift
		8.6.2 Domain Adaptation
	8.7 Further Reading
	Exercises
9 Topic Detection and Tracking
	9.1 History of Topic Detection and Tracking
	9.2 Terminology and Task Definition
		9.2.1 Terminology
		9.2.2 Task
	9.3 Story/Topic Representation and Similarity Computation
	9.4 Topic Detection
		9.4.1 Online Topic Detection
		9.4.2 Retrospective Topic Detection
	9.5 Topic Tracking
	9.6 Evaluation
	9.7 Social Media Topic Detection and Tracking
		9.7.1 Social Media Topic Detection
		9.7.2 Social Media Topic Tracking
	9.8 Bursty Topic Detection
		9.8.1 Burst State Detection
		9.8.2 Document-Pivot Methods
		9.8.3 Feature-Pivot Methods
	9.9 Further Reading
	Exercises
10 Information Extraction
	10.1 Concepts and History
	10.2 Named Entity Recognition
		10.2.1 Rule-based Named Entity Recognition
		10.2.2 Supervised Named Entity Recognition Method
		10.2.3 Semisupervised Named Entity Recognition Method
		10.2.4 Evaluation of Named Entity Recognition Methods
	10.3 Entity Disambiguation
		10.3.1 Clustering-Based Entity Disambiguation Method
		10.3.2 Linking-Based Entity Disambiguation
		10.3.3 Evaluation of Entity Disambiguation
	10.4 Relation Extraction
		10.4.1 Relation Classification Using Discrete Features
		10.4.2 Relation Classification Using Distributed Features
		10.4.3 Relation Classification Based on Distant Supervision
		10.4.4 Evaluation of Relation Classification
	10.5 Event Extraction
		10.5.1 Event Description Template
		10.5.2 Event Extraction Method
		10.5.3 Evaluation of Event Extraction
	10.6 Further Reading
	Exercises
11 Automatic Text Summarization
	11.1 Main Tasks in Text Summarization
	11.2 Extraction-Based Summarization
		11.2.1 Sentence Importance Estimation
		11.2.2 Constraint-Based Summarization Algorithms
	11.3 Compression-Based Automatic Summarization
		11.3.1 Sentence Compression Method
		11.3.2 Automatic Summarization Based on Sentence Compression
	11.4 Abstractive Automatic Summarization
		11.4.1 Abstractive Summarization Based on Information Fusion
		11.4.2 Abstractive Summarization Based on the Encoder-Decoder Framework
	11.5 Query-Based Automatic Summarization
		11.5.1 Relevance Calculation Based on the Language Model
		11.5.2 Relevance Calculation Based on Keyword Co-occurrence
		11.5.3 Graph-Based Relevance Calculation Method
	11.6 Crosslingual and Multilingual Automatic Summarization
		11.6.1 Crosslingual Automatic Summarization
		11.6.2 Multilingual Automatic Summarization
	11.7 Summary Quality Evaluation and Evaluation Workshops
		11.7.1 Summary Quality Evaluation Methods
		11.7.2 Evaluation Workshops
	11.8 Further Reading
	Exercises
References