Language Corpora Annotation and Processing

Preface
Acknowledgments
Introduction
	Why This Book
	Text Annotation Versus Text Processing
	Summary of the Chapters
	Value of this Book
	References
Contents
About the Author
Abbreviations
1 Corpus Text Annotation
	1.1 Introduction: What Is Text Annotation?
	1.2 Characteristics of Annotation
	1.3 Kinds of Text Annotation
	1.4 Criteria of Text Annotation
	1.5 Maxims of Text Annotation
	1.6 Justification of Text Annotation
	1.7 Annotation Schemas and Models
	1.8 Types of Text Annotation
		1.8.1 Intralinguistic Annotation
		1.8.2 Extralinguistic Annotation
	1.9 Present State of Text Annotation
	1.10 Utilization of Annotated Texts
	References
2 Principles and Rules of Part-of-Speech Annotation
	2.1 Introduction
	2.2 Principles of POS Annotation
	2.3 Rules for POS Annotation
	2.4 Conclusion
	References
3 Part-of-Speech Annotation
	3.1 Introduction
	3.2 Concept of POS Annotation
	3.3 Morphological Analysis Versus POS Annotation
	3.4 Levels of POS Annotation
	3.5 Stages of POS Annotation
		3.5.1 Pre-editing Stage
		3.5.2 POS Assignment Stage
		3.5.3 Postediting Stage
	3.6 Earlier Methods POS Annotation
	3.7 POS Annotation: Indian Scenario
	3.8 The BIS POS Tagset
	3.9 The BIS Tagset and Bengali
		3.9.1 Utility of a POS Annotated Text
	References
4 Extratextual Annotation
	4.1 Introduction
	4.2 Definition of Extratextual Annotation
	4.3 Intratextual and Extratextual Annotation
	4.4 Relevance of Extratextual Annotation
	4.5 Extratextual Annotation: Some Early Works
	4.6 Extratextual Annotation Types
		4.6.1 File Name: A Gateway
		4.6.2 Annotation of Text Categories
		4.6.3 Subject Category Annotation
		4.6.4 Annotating Title of a Text
		4.6.5 Header Part: Metadata Depository
	4.7 Conclusion
	References
5 Etymological Annotation
	5.1 Introduction
	5.2 Lexical Borrowing: Some Scenarios
	5.3 Vocabulary Classification
	5.4 Defining Etymological Annotation Tagset
	5.5 Defining Annotation Strategy
		5.5.1 Annotating Borrowed Words
		5.5.2 Annotating Portmanteau Words
		5.5.3 Annotating Affixed Words
		5.5.4 Annotating Inflected Words
	5.6 Process of Etymological Annotation
	5.7 Findings of an Etymologically Annotated Text
	5.8 Frequently Used English Words in Bengali
	5.9 System Adoption at Lexical Level
	5.10 Conclusion
	References
6 More Types of Corpus Annotation
	6.1 Introduction
	6.2 Orthographic Annotation
	6.3 Prosodic Annotation
	6.4 Semantic Annotation
	6.5 Discourse Annotation
	6.6 Rhetoric Annotation
	6.7 Conclusion
	References
7 Morphological Processing of Words
	7.1 Introduction
	7.2 Models and Approaches
		7.2.1 Two-Level Morphology-Based Approach
		7.2.2 Paradigm-Based Approach
		7.2.3 Stemmer-Based Approach
		7.2.4 Acyclic Graph-Based Approach
		7.2.5 Morph-Based Approach
		7.2.6 Corpus-Based Approach
		7.2.7 Suffix Stripping-Based Approach
	7.3 Issues in Morphological Processing
	7.4 Method of Information Storage
	7.5 Method of Morphological Processing
	7.6 Processing Detached Words
	7.7 Results of Morphological Processing
		7.7.1 Rightly Processed Words
		7.7.2 Wrongly Processed Words
		7.7.3 Double Processed Words
		7.7.4 Non-Processed Words
	7.8 Ambiguity in Morphological Processing
	7.9 Conclusion
	References
8 Lemmatization of Inflected Nouns
	8.1 Introduction
	8.2 Lemma(-tization)
	8.3 Lemmatization and Stemming
		8.3.1 Lemmatization is Similar to Stemming
		8.3.2 Lemmatization is Different from Stemming
	8.4 Lemmatization in English and Other Languages
	8.5 Surface Structure of Bengali Nouns
	8.6 Stages for Noun Lemmatization
		8.6.1 Stage 1: POS Annotation
		8.6.2 Stage 2: Noun Identification and Isolation
		8.6.3 Stage 3: Alphabetical Sorting of Nouns
		8.6.4 Stage 4: Noun Classification
		8.6.5 Stage 5: Tokenization
	8.7 Operation of Lemmatization Process
	8.8 Conclusion
	Appendix
	References
9 Decomposition of Inflected Verbs
	9.1 Introduction
	9.2 Lexical Decomposition
	9.3 Some Early Works
	9.4 Morpheme Structure of Bengali Verbs
		9.4.1 Root Part (dhātu)
		9.4.2 Suffix Part (Bibhakti)
	9.5 Conjugation of Bengali Verbs
	9.6 Categorization of Bengali Verb Roots
		9.6.1 Verb Root Categorization
		9.6.2 Verb Suffix Categorization
	9.7 Issues in Lexical Decomposition
	9.8 Method of Information Storage
	9.9 Decomposing Non-conjugated Verbs
	9.10 Decomposing Conjugated Verbs
	9.11 Data Creation and Storage
		9.11.1 Suffix and Root Detection
		9.11.2 Suffix––Root Mapping
	9.12 Some Special Cases
	9.13 The Resultant Output
	9.14 Performance of the System
	9.15 Conclusion
	References
10 Syntactic Annotation
	10.1 Introduction
	10.2 Ambiguity of the Term
	10.3 Transition of the Concept of Syntactic Annotation
	10.4 Goals of Syntactic Annotation
	10.5 Challenges Involved in Syntactic Annotation
	10.6 What is a Syntactic Annotator (Parser)?
		10.6.1 Detection of End of a Sentence
		10.6.2 Tokenization of Words in Sentences
		10.6.3 Grammatical Annotation
		10.6.4 Chunking Larger Lexical Blocks
		10.6.5 Finding Matrix Verb in Sentence
		10.6.6 Identifying Non-Matrix Clauses
		10.6.7 Identification of Phrases
	10.7 Types of Syntactic Annotation
	10.8 Treebank
	10.9 Utility of Syntactic Annotated Texts
	10.10 Conclusion
	References
Author Index
Subject Index