A Course in Natural Language Processing

Preface
	From ELIZA to ChatGPT
	Pedagogical Objectives
		For Whom Is This Book Written, and How To Read It
		Acknowledgments
Contents
Chapter 1 Introduction
	1.1 What Is Language in the First Place?
	1.2 Principles of Linguistics and of Language
		1.2.1 Signifier and Signified
		1.2.2 Opposition, Etics and Emics
		1.2.3 Paradigmatic Axis and Syntagmatic Axis
		1.2.4 Compositionality
		1.2.5 Modalities of Language
		1.2.6 Functions of Language
		1.2.7 Sapir-Whorf and the Eskimo Vocabulary Hoax
	1.3 A Terminological Issue: Data–Information–Knowledge
	1.4 Notations
	1.5 Exercises and Hints
	1.6 Resources and Errata
	References
Part I Linguistics
	Chapter 2 Phonetics/Phonology
		2.1 Articulatory Phonetics
			2.1.1 (Pulmonic) Consonants
			2.1.2 Vowels
		2.2 Acoustic Phonetics
		2.3 From Phonetics to Phonemics
			2.3.1 Features
			2.3.2 Phonemes
		2.4 Phonological Rules
			2.4.1 Underlying Representation
		2.5 Suprasegmental Aspects
			2.5.1 Syllables
			2.5.2 Stress and Foot
			2.5.3 Mora
			2.5.4 Tone
			2.5.5 Prosody
		2.6 iPA Phonetics, an App for Learning Phonetics
		2.7 Psycholinguistic Aspects, Perceptual Phonetics
		2.8 Further Reading
			2.8.1 Literature
			2.8.2 LATEX
			2.8.3 Science Fiction
		2.9 Exercises
			Exercise 1-1: English Accents
			Exercise 1-2: Phonotactics of English
			Exercise 1-3: Tonotactics of Vietnamese
			Exercise 1-4: Classification of Voice Files
		References
	Chapter 3 Graphetics/Graphemics
		3.1 Graphetics
			3.1.1 Descriptive Graphetics
				3.1.1.1 Cheirographetics, or the Study of Handwriting
				3.1.1.2 Typographetics
				3.1.1.3 Descriptive Levels
				3.1.1.4 Kerning and Ligatures
				3.1.1.5 Typographetic Functions and Connotations
		3.2 Graphemics
			3.2.1 Writing Systems and Scripts
			3.2.2 Pictography, Emoji
			3.2.3 Orthography
			3.2.4 Hyphenation and Non-breakability
			3.2.5 Graphemic Gender-neutral Methods
			3.2.6 Sinographemics
		3.3 Psycholinguistic Aspects of Reading
		3.4 Further Reading
			3.4.1 Literature
			3.4.2 LATEX
			3.4.3 Science Fiction
		3.5 Exercises
			Exercise 2-1: Evaluating ALA-LC Transcriptions of Arabic and Greek
			Exercise 2-2: Graphotactics of English
			Exercise 2-3: Greek Car License Plate and Signs
			Exercise 2-4: Predictability of New Sinograms
			Exercise 2-5: Exotype Classification
		References
	Chapter 4 Morphemes, Words, Terms
		4.1 Words
		4.2 Lexemes
		4.3 Parts of Speech
		4.4 Morphemes
		4.5 Inflection
		4.6 Derivation
		4.7 Compounding
		4.8 Astonishing Morphologies: Semitic Languages and Lojban
			4.8.1 Semitic Languages
			4.8.2 Lojban
		4.9 Terms and Collocations
		4.10 Psycholinguistic Aspects
			Finding Words
			Building Words
			Phonological Encoding
			Keylogs
		4.11 Further Reading
			4.11.1 Literature
			4.11.2 Science Fiction
				Orwell’s Newspeak
				Time Travel and Verb Morphology
				The Golem
		4.12 Exercises
			Exercise 3-1: English and French Verb Conjugation Compared
			Exercise 3-2: Jules Verne and French Verbs
			Exercise 3-3: The Combinatorics of Neoclassical Morphemes
			Exercise 3-4: The Morphology of Lojban
			Exercise 3-5: Long and Round Ess in German
		References
	Chapter 5 Syntax
		5.1 Constituents and Clauses
			5.1.1 Constituency Tests
			5.1.2 Agreement
			5.1.3 Clauses
			5.1.4 Topology
			5.1.5 Ambiguity
		5.2 Syntax Theories
		5.3 Chomsky’s Context-Free Phrase Structure Grammar
			5.3.1 Parsing Context-Free Phrase Structure Grammar in Python
		5.4 Chomsky’s Transformational Grammar
		5.5 Binding Theory
			5.5.1 Domination
			5.5.2 Precedence
			5.5.3 C-command
			5.5.4 Referring Expressions, Anaphors, Binding
		5.6 X Theory
			5.6.1 Tense Phrases
		5.7 Head-Driven Phrase Structure Grammars
		5.8 Combinatory Categorial Grammars
			5.8.1 From Phrase-Structure Grammars to Categorial Grammars
			5.8.2 Conjunction
			5.8.3 Composition, Bluebird
			5.8.4 Type Raising, Thrush
			5.8.5 A Python Parser for CCGs
		5.9 Dependency Syntax
			5.9.1 Some History
			5.9.2 Strings and Catenae
			5.9.3 Types of Dependency Relations
			5.9.4 From Constituents to Dependencies
			5.9.5 Parsing Dependency Grammars in Python
			5.9.6 Surface-Syntactic Universal Dependencies
		5.10 Psycholinguistic Aspects
		5.11 Further Reading
			5.11.1 Literature
			5.11.2 LATEX
			5.11.3 Science Fiction
		5.12 Exercises
			Exercise 4-1: Constituency parser comparison
			Exercise 4-2: How well do stanza and spacy parse Yoda?
			Exercise 4-3: The Syntax of Lojban
			Exercise 4-4: Find Perfectly Ambiguous Sentences in English
			Exercise 4-5: Find emoji that behave like noun phrases
		References
	Chapter 6 Semantics (and Pragmatics)
		6.1 Sense Relations
		6.2 Structuralist Approaches to Semantics
			6.2.1 Lexical Field Theory
			6.2.2 Componential Analysis, Formal Concept Analysis
			6.2.3 Relational Semantics
			6.2.4 WordNet
		6.3 Neostructuralist Approaches to Semantics
			6.3.1 Wierzbicka’s Natural Semantic Metalanguage
			6.3.2 Conceptual Semantics
			6.3.3 Generative Lexicon
		6.4 Cognitive Semantics
			6.4.1 Prototype Theory
			6.4.2 Fillmore’s Frames
			6.4.3 FrameNet
			6.4.4 Minsky’s Frames
			6.4.5 Frames and Humor
			6.4.6 Idealized Cognitive Models and Conceptual Theory of Metaphor
			6.4.7 MetaNet
		6.5 Formal Semantics
			6.5.1 Frege, Sense, Denotation, and Truth
			6.5.2 Montague Formal Semantics
				6.5.2.1 Python Implementation of Formal Semantics
				6.5.2.2 Formal Semantics through CCGs
		6.6 Discourse Semantics
			6.6.1 Rhetorical Structure Theory
			6.6.2 Discourse Representation Theory
		6.7 Implicatures and Conversation Maxims
		6.8 Psycholinguistic Aspects
			6.8.1 Independence of Syntactic and Semantic Processing
			6.8.2 Architecture of the Language Processing System
		6.9 Further Reading
			6.9.1 Literature
			6.9.2 Science Fiction
		6.10 Exercises
			Exercise 5-1: Find faux amis words in French, German, Italian, Spanish, and English
			Exercise 5-2: FCA
			Exercise 5-3: The semantics of Lojban
		References
	Chapter 7 Controlled Natural Languages
		7.1 Simplifications of English: Basic English, Simple English, and Caterpillar English
		7.2 Formalizable Controlled Languages: Attempto Controlled English, PENG
		7.3 A CNL for Mathematics: ForTheL
		7.4 Exercises
			Exercise 6-1: Discovery of Attempto Controlled English
			Exercise 6-2: How simple is Simple English Wikipedia?
			Exercise 6-3: Write haikus inspired by themes by Emily Dickinson in Python
			Exercise 6-4: Do Daleks use a controlled language?
		References
Part II Mathematical Tools
	Chapter 8 Graphs
		8.1 Definitions
			8.1.1 Trees
		8.2 Basic Graph Algorithms
			8.2.1 Search in a Graph
			8.2.2 Shortest Paths
			8.2.3 An Example: Word Ladders
			8.2.4 Processing WordNet as a Graph
		8.3 Vertex Centrality
			8.3.1 Degree Centrality
				Degree Centrality in WordNet
			8.3.2 Closeness Centrality
				Closeness Centrality in WordNet
			8.3.3 Betweenness Centrality
				Betweenness Centrality in WordNet
		8.4 Community Detection
			8.4.1 Two Examples Based on Shakespeare\'s Night’s Dream
				8.4.1.1 The Co-presence on Stage Graph
				8.4.1.2 Doubling in MND
				8.4.1.3 Centralities of MND Characters in the Co-presence on Stage Graph
				8.4.1.4 Communities of MND Characters in the Co-presence on Stage Graph
				8.4.1.5 The Vocative Graph
				8.4.1.6 Centralities of MND Characters in the Vocative Graph
				8.4.1.7 Communities of MND Characters in the Vocative Graph
				8.4.1.8 Possible Improvements
		8.5 Further Reading
			8.5.1 Literature
			8.5.2 LATEX
			8.5.3 Science Fiction
		8.6 Exercises
			Exercise 7-1: Using WordNet for disambiguation
			Exercise 7-2: Find the most central word of the Quran and the Bible
			Exercise 7-3: Assortativity of Chinese Hyperonyms
			Exercise 7-4: Productivity of sinographic component base
		References
	Chapter 9 Formal Languages
		9.1 Background
		9.2 Basic Definitions
		9.3 Formal Grammars
			9.3.1 The Chomsky Hierarchy
		9.4 Regular Languages
			9.4.1 Regular Expressions
				9.4.1.1 Abstract Syntax
				9.4.1.2 POSIX Syntax
				9.4.1.3 Lazy Quantifiers
				9.4.1.4 Regular Expressions in Python
				9.4.1.5 Regular Expressions and ELIZA
				9.4.1.6 Regular Expressions, Gender-Neutral Methods, and Poetry
			9.4.2 Finite-State Automata and Transducers
				9.4.2.1 Finite-State Automata in Python
				9.4.2.2 Transducers
				9.4.2.3 Transducers in Python
		9.5 Context-Free Grammars
			9.5.1 Context-Free Grammars in Python
			9.5.2 Feature-Based Context-Free Grammars in Python
		9.6 Grammatical Inference
		9.7 Further Reading
			9.7.1 Literature
			9.7.2 LATEX
		9.8 Exercises
			Exercise 8-1: