Applied Text Mining

Foreword
Preface
	Organization of the Book
	To the Instructor
	To the Student
	To the Professional
Contents
About the Authors
Part I: Text Mining Basics
	1: Introduction to Text Mining
		1.1	 Textual Data and Its Components
			1.1.1	 Components of Textual Data
			1.1.2	 Formats of Textual Data
		1.2	 Sources of Textual Data
		1.3	 Text Mining
		1.4	 Core Text Mining Operations
			1.4.1	 Distribution
			1.4.2	 Frequent Concept Sets
			1.4.3	 Associations
		1.5	 Challenges of Text Mining
		1.6	 Text Indexing Process
			1.6.1	 Tokenization
			1.6.2	 Stemming
			1.6.3	 Stop-Word Removal
			1.6.4	 Term Weighting
		1.7	 Text Information System and Its Functions
			1.7.1	 Information Access
			1.7.2	 Knowledge Acquisition
			1.7.3	 Text Organization
		1.8	 Conceptual Framework for Text Information Systems
		1.9	 Text Patterns
		1.10	 Documents and Corpus
			1.10.1	 Processing Documents
			1.10.2	 Corpus as Baseline
		1.11	 Regular Expression
		1.12	 Summary
		1.13	 Exercises
	2: Text Processing
		2.1	 Natural Language
			2.1.1	 What Is Natural Language
			2.1.2	 The Philosophy of Language
			2.1.3	 Language Acquisition and Usage
		2.2	 Linguistics
			2.2.1	 Language Syntax and Structure
			2.2.2	 Words
			2.2.3	 Phrases
			2.2.4	 Clauses
			2.2.5	 Grammar
			2.2.6	 Word-Order Typology
		2.3	 Language Semantics
			2.3.1	 Lexical Semantic Relations
			2.3.2	 Semantic Networks and Models
			2.3.3	 Representation of Semantics
		2.4	 Text Corpora
			2.4.1	 Corpora Annotation and Utilities
			2.4.2	 Popular Corpora
			2.4.3	 Accessing Text Corpora
		2.5	 Text Preprocessing
			2.5.1	 Sentence Segmentation
			2.5.2	 Word Tokenization
			2.5.3	 POS Tagging
			2.5.4	 Named Entity Recognition
		2.6	 Sentence Structure
		2.7	 Information Extraction from Text
		2.8	 Architecture of Information Extraction System
			2.8.1	 Tokenization
			2.8.2	 Morphological and Lexical Analysis
			2.8.3	 Syntactic Analysis
			2.8.4	 Domain Analysis
		2.9	 Summary
		2.10	 Exercises
	3: Text Mining Applications
		3.1	 Sentiment Analysis
			3.1.1	 Sentiment Analysis Applications
			3.1.2	 Sentiment Analysis Problems
			3.1.3	 Opinion Summarization
			3.1.4	 Opinion Types
		3.2	 Sentiment Classification
			3.2.1	 Supervised Sentiment Classification
			3.2.2	 Unsupervised Sentiment Classification
		3.3	 Aspect-Based Sentiment Analysis
			3.3.1	 Aspect-Based Sentiment Classification
			3.3.2	 Aspect Extraction
			3.3.3	 Aspect Categories
			3.3.4	 Word Sense Disambiguation
		3.4	 Opinion Summarization
			3.4.1	 Aspect-Based Opinion Summarization
			3.4.2	 Contrastive View Summarization
			3.4.3	 Traditional Summarization
		3.5	 Analysis of Comparative Opinions
		3.6	 Opinion Search and Retrieval
		3.7	 Opinion Spam Detection
			3.7.1	 Types of Spam
			3.7.2	 Supervised Spam Detection
			3.7.3	 Unsupervised Spam Detection
		3.8	 Summary
		3.9	 Exercises
Part II: Text Analytics
	4: Feature Engineering for Text Representations
		4.1	 Introduction to Features
		4.2	 Feature Engineering
		4.3	 Traditional Feature Engineering Models
			4.3.1	 Bag-of-Words Model
			4.3.2	 Bag-of-N-Grams Model
			4.3.3	 TF-IDF Model
			4.3.4	 Extracting Features for New Documents
			4.3.5	 Document Similarity
			4.3.6	 Topic Models
		4.4	 Advanced Feature Engineering Models
			4.4.1	 Loading the Bible Corpus
			4.4.2	 Word2Vec Model
			4.4.3	 Robust Word2Vec Models with Gensim
			4.4.4	 Applying Word2Vec Features for Machine Learning Tasks
			4.4.5	 The GloVe Model
			4.4.6	 Applying GloVe Features for Machine Learning Tasks
			4.4.7	 The FastText Model
			4.4.8	 Applying FastText Features to Machine Learning Tasks
		4.5	 Summary
		4.6	 Exercises
	5: Text Classification
		5.1	 What Is Text Classification?
		5.2	 Automated Text Classification
		5.3	 Text Classification Blueprint
		5.4	 Data Retrieval
		5.5	 Data Preprocessing and Normalization
		5.6	 Training and Test Datasets
		5.7	 Feature Engineering Techniques
			5.7.1	 Traditional Feature Engineering Models
			5.7.2	 Advanced Feature Engineering Models
		5.8	 Classification Models
			5.8.1	 Multinomial Naive Bayes
			5.8.2	 Logistic Regression
			5.8.3	 Support Vector Machines
			5.8.4	 Ensemble Models
			5.8.5	 Random Forest
			5.8.6	 Gradient Boosting Machines
		5.9	 Evaluating Classification Models
		5.10	 Building and Evaluating Text Classifier
		5.11	 Applications
		5.12	 Summary
		5.13	 Exercises
	6: Text Clustering
		6.1	 Introduction to Text Clustering
			6.1.1	 K-Means Clustering
			6.1.2	 Hierarchical Clustering
			6.1.3	 Density-Based Spatial Clustering of Applications with Noise (DBSCAN)
			6.1.4	 Latent Dirichlet Allocation (LDA)
		6.2	 Clustering Types
			6.2.1	 Static Clustering and Dynamic Clustering
			6.2.2	 Crisp Clustering and Fuzzy Clustering
			6.2.3	 Flat Clustering and Hierarchical Clustering
			6.2.4	 Single-Viewed Clustering and Multiple-Viewed Clustering
		6.3	 Derived Tasks from Text Clustering
		6.4	 Text Clustering Algorithms
			6.4.1	 Simple Clustering Algorithms
			6.4.2	 K-Means Algorithm
			6.4.3	 Competitive Learning
		6.5	 Implementation of Text Clustering
		6.6	 Clustering Evaluation
			6.6.1	 Clustering Evaluation
			6.6.2	 Cluster Validation
			6.6.3	 Clustering Indexes
			6.6.4	 Parameter Tuning
		6.7	 Summary
		6.8	 Exercises
	7: Text Summarization and Topic Modeling
		7.1	 Introduction to Text Summarization
		7.2	 Summarization Types
			7.2.1	 Manual Versus Automatic Text Summarization
			7.2.2	 Single Versus Multiple Text Summarization
			7.2.3	 Flat Versus Hierarchical Text Summarization
			7.2.4	 Abstraction-Versus Query-Based Summarization
		7.3	 Approaches to Text Summarization
			7.3.1	 Heuristics-Based Approaches
			7.3.2	 Mapping Summarization as a Classification Task
			7.3.3	 Sampling Schemes
		7.4	 Important Concepts
			7.4.1	 Documents
			7.4.2	 Text Normalization
			7.4.3	 Feature Extraction
			7.4.4	 Feature Matrix
			7.4.5	 Singular Value Decomposition
			7.4.6	 Text Normalization
			7.4.7	 Feature Extraction
		7.5	 Keyphrase Extraction
			7.5.1	 Collocations
			7.5.2	 Weighted Tag-Based Phrase Extraction
		7.6	 Topic Modeling and Its Objectives
			7.6.1	 Latent Semantic Indexing
			7.6.2	 Latent Dirichlet Allocation
			7.6.3	 Non-negative Matrix Factorization
		7.7	 Modeling Case Study
			7.7.1	 Topic Modeling Using Gensim
			7.7.2	 Topic Modeling Using Scikit-Learn
		7.8	 Automated Document Summarization
			7.8.1	 Text Wrangling
			7.8.2	 Text Representation with Feature Engineering
			7.8.3	 Latent Semantic Analysis
		7.9	 Challenges of Text Summarization
		7.10	 Summary
		7.11	 Exercises
	8: Taxonomy Generation and Dynamic Document Organization
		8.1	 Introduction to Taxonomy Generation
		8.2	 Taxonomy Generation Tasks
			8.2.1	 Keyword Extraction
			8.2.2	 Word Categorization
			8.2.3	 Word Clustering
			8.2.4	 Topic Routing
		8.3	 Taxonomy Generation Schemes
			8.3.1	 Index-Based Scheme
			8.3.2	 Clustering-Based Scheme
			8.3.3	 Association-Based Scheme
			8.3.4	 Link Analysis-Based Scheme
		8.4	 Taxonomy Governance
			8.4.1	 Taxonomy Maintenance
			8.4.2	 Taxonomy Growth
			8.4.3	 Taxonomy Integration
			8.4.4	 Ontology
		8.5	 Dynamic Document Organization
		8.6	 Online Clustering
		8.7	 Online Clustering Algorithms
			8.7.1	 Online Clustering in Conceptual and Functional View
			8.7.2	 Online K-Means Algorithms
			8.7.3	 Online Unsupervised K-Nearest Neighbors Algorithms
			8.7.4	 Fuzzy Clustering
		8.8	 Dynamic Organization
			8.8.1	 Execution Process
			8.8.2	 Maintenance Mode
			8.8.3	 Creation Mode
			8.8.4	 Additional Tasks
		8.9	 Challenges of Dynamic Document Organization
			8.9.1	 Text Representation
			8.9.2	 Binary Decomposition
			8.9.3	 DDO System Variants
		8.10	 Summary
		8.11	 Exercises
	9: Visualization Approaches
		9.1	 Introduction and Importance of Text Visualization
		9.2	 Visualization Layer in the Text Mining System
		9.3	 Concept Graphs
			9.3.1	 Simple Concept Graphs
			9.3.2	 Simple Concept Set Graphs
			9.3.3	 Simple Concept Association Graphs
		9.4	 Histograms
		9.5	 Line Graphs
		9.6	 Circle Graphs
		9.7	 Category Connecting Maps
		9.8	 Self-Organizing Maps (SOMs)
		9.9	 Hyperbolic Trees
		9.10	 Summary
		9.11	 Exercises
Part III: Deep Learning in Text Mining
	10: Text Mining Through Deep Learning
		10.1	 Role of Deep Learning in Text Mining
		10.2	 Deep Learning Models for Processing Text
			10.2.1	 Feed-Forward Neural Networks
			10.2.2	 Convolutional Neural Networks
			10.2.3	 Multi-layer Perceptron (MLP)
				10.2.3.1	 Regression MLPs
				10.2.3.2	 Classification MLPs
				10.2.3.3	 Setting Up MLPs with Keras
			10.2.4	 Recurrent Neural Networks
				10.2.4.1	 Memory Cells
				10.2.4.2	 Input-Output Sequences
				10.2.4.3	 RNNs Training
				10.2.4.4	 RNN Implementation
			10.2.5	 Long Short-Term Memory
				10.2.5.1	 LSTM Architecture
				10.2.5.2	 LSTM in Text Mining
				10.2.5.3	 Key Applications of LSTM in Text Mining
				10.2.5.4	 Python Implementation
			10.2.6	 Transformers
				10.2.6.1	 Transformers in Text Mining
				10.2.6.2	 Basic Architecture
				10.2.6.3	 Self-Attention and Multi-head Attention
				10.2.6.4	BERT (Bidirectional Encoder Representations from Transformers)
				10.2.6.5	 Handling Long Sequences with Transformers
				10.2.6.6	 Applications of Transformers in Text Mining
		10.3	 Deep Learning in Sentiment Analysis
			10.3.1	 Neural Networks
			10.3.2	 Word Embedding
			10.3.3	 Sentiment Analysis Tasks
			10.3.4	 Neural Network Architectures for Sentiment Analysis
		10.4	 ChatGPT
			10.4.1	 Foundation of ChatGPT
			10.4.2	 Ethical and Societal Considerations
		10.5	 Summary
		10.6	 Exercises
	11: Lexical Analysis and Parsing Using Deep Learning
		11.1	 Introduction to Lexical Analysis and Parsing Using Deep Learning
			11.1.1	 Word Segmentation
			11.1.2	 Syntactic Parsing
			11.1.3	 Structured Prediction
			11.1.4	 Advantages and Disadvantages of Conventional Lexical Analysis Techniques
		11.2	 Conventional Lexical Analysis Case Study
		11.3	 Structured Prediction Methods
			11.3.1	 Graph-Based Methods
			11.3.2	 Transition-Based Methods
		11.4	 Neural Graph-Based Methods
			11.4.1	 Neural Conditional Random Fields
			11.4.2	 Neural Graph-Based Dependency Parsing
		11.5	 Neural Transition-Based Methods
			11.5.1	 Neural Greedy Shift-Reduce Dependency Parsing
			11.5.2	 Neural Greedy Sequence Labeling
			11.5.3	 Globally Optimized Models
		11.6	 Deep Learning-Based Lexical Analysis Case Study
		11.7	 Advantages and Disadvantages of Deep Learning-Based Lexical Analysis Techniques
		11.8	 Summary
		11.9	 Exercises
	12: Machine Translation Using Deep Learning
		12.1	 Machine Translation
		12.2	 Ambiguity
			12.2.1	 Word Translation Problems
			12.2.2	 Phrase Translation Problems
			12.2.3	 Syntactic Translation Problems
			12.2.4	 Semantic Translation Problems
		12.3	 Practical Issues
			12.3.1	 Data Availability
			12.3.2	 Evaluation Campaign
		12.4	 Applications of Machine Translation
			12.4.1	 Information Access
			12.4.2	 Aiding Human Translators
			12.4.3	 Communication
			12.4.4	 Natural Language Processing Pipelines
		12.5	 Machine Translation Approaches
		12.6	 Introduction to Deep Learning Techniques for MT
		12.7	 Component-Wise Deep Learning for Machine Translation
			12.7.1	 Translation Models
			12.7.2	 Reordering Models
			12.7.3	 Language Models
		12.8	 End-to-End Deep Learning Models for Machine Translation
			12.8.1	 Sequence-to-Sequence Neural Network
			12.8.2	 Encoder Network
			12.8.3	 Decoder Network
		12.9	 Translation of Highly Repetitive Content
		12.10	 Translation of User-Generated Content
		12.11	 Online Customer Service
		12.12	 Summary
		12.13	 Exercise