Automatic Language Identification in Texts

Foreword
Acknowledgments
	References
Contents
About the Authors
1 Introduction to Language Identification
	1.1 A Brief History of Language Identification (LI)
	1.2 What is LI Used For?
	1.3 What are the Main Challenges that Make LI Difficult?
2 Features and Methods
	2.1 On Notation
	2.2 What Textual Features Are Used for LI and How Are They Collected and Calculated?
		2.2.1 Feature Smoothing
	2.3 What Classification Methods Are Used for LI and How Do They Work?
		2.3.1 Decision Rules, Trees and Random Forests
		2.3.2 Simple Scoring
		2.3.3 Sum or Average of Values
		2.3.4 Product of Values
		2.3.5 Similarity Measures
		2.3.6 Logistic Regression
		2.3.7 Support Vector Machines
		2.3.8 Neural Networks
		2.3.9 Ensemble Methods
	2.4 Machine Learning Toolkits and Libraries
3 Evaluation and Measurement
	3.1 How is LI Performance Evaluated? What Are the Measures …
	3.2 What Material Can Be Used in Training and Evaluating Language Identifiers?
	3.3 LI Shared Tasks
4 Specific Challenges of Variation and Text Types
	4.1 Language Similarity
		4.1.1 LI for Similar Languages, Varieties, and Dialects
	4.2 Low-Resource Languages
	4.3 Orthography and Its Variations
	4.4 Short Texts
5 Large Scale, Multi-domain Language Identification
	5.1 Number of Languages
	5.2 Unseen Languages
	5.3 Multilingual Texts
	5.4 Domain Compatibility
6 Applications and Related Tasks
	[DELETE]
	6.1 Applications
		6.1.1 Monolingual NLP Components
		6.1.2 Machine Translation
		6.1.3 Multilingual Document Storage and Retrieval
	6.2 Related Tasks
		6.2.1 Native Language Identification
		6.2.2 Author Profiling and Identification
7 Conclusion and Future Directions