Markov Models for Pattern Recognition: From Theory to Applications

Preface......Page 7
Contents......Page 9
1 Introduction......Page 14
1.2 Functional Principles of Markov Models......Page 16
1.3 Goal and Structure of the Book......Page 18
2.1 Speech......Page 20
2.2 Writing......Page 27
2.3 Biological Sequences......Page 35
2.4 Outlook......Page 39
Part I Theory......Page 42
Introductory Remarks......Page 44
3.1 Random Experiment, Event, and Probability......Page 46
3.2 Random Variables and Probability Distributions......Page 48
3.3 Parameters of Probability Distributions......Page 50
3.4 Normal Distributions and Mixture Models......Page 51
3.5 Stochastic Processes and Markov Chains......Page 53
3.6 Principles of Parameter Estimation......Page 54
3.7 Bibliographical Remarks......Page 57
4 Vector Quantization......Page 58
4.1 Definition......Page 59
4.2 Optimality......Page 60
4.3 Algorithms for Vector Quantizer Design......Page 63
4.4 Estimation of Mixture Density Models......Page 68
4.5 Bibliographical Remarks......Page 72
5.1 Definition......Page 74
5.2 Modeling Emissions......Page 76
5.3 Use Cases......Page 78
5.4 Notation......Page 80
5.5 Evaluation......Page 81
5.6 Decoding......Page 87
5.7 Parameter Estimation......Page 89
5.8 Model Variants......Page 104
5.9 Bibliographical Remarks......Page 105
nition......Page 108
6.2 Use Cases......Page 109
6.3 Notation......Page 110
6.4 Evaluation......Page 111
6.5 Parameter Estimation......Page 113
6.6 Model Variants......Page 122
6.7 Bibliographical Remarks......Page 125
Part II Practice......Page 128
Introductory Remarks......Page 130
7 Computations with Probabilities......Page 132
7.1 Logarithmic Probability Representation......Page 133
7.2 Lower Bounds for Probabilities......Page 135
7.3 Codebook Evaluation for Semi-Continuous HMMs......Page 136
7.4 Probability Ratios......Page 137
8.1 Model Topologies......Page 140
8.2 Modularization......Page 142
8.3 Compound Models......Page 144
8.4 Profile HMMs......Page 146
8.5 Modeling Emissions......Page 149
9 Robust Parameter Estimation......Page 150
9.1 Feature Optimization......Page 152
9.2 Tying......Page 165
9.3 Initialization of Parameters......Page 176
10 Efficient Model Evaluation......Page 178
10.1 Efficient Evaluation of Mixture Densities......Page 179
10.2 Beam Search......Page 180
10.3 Efficient Parameter Estimation......Page 183
10.4 Tree-like Model Organization......Page 187
11.1 Basic Principles......Page 194
11.2 Adaptation of Hidden Markov Models......Page 195
11.3 Adaptation of n-Gram Models......Page 199
12 Integrated Search Methods......Page 202
12.1 HMM Networks......Page 205
12.2 Multi-Pass Search......Page 206
12.3 Search Space Copies......Page 207
12.4 Time-Synchronous Parallel Model Decoding......Page 211
Part III Systems......Page 216
Introductory Remarks......Page 218
13.1 Recognition System of RWTH Aachen University......Page 220
13.2 BBN Speech Recognizer BYBLOS......Page 222
13.3 ESMERALDA......Page 223
14.1 OCR System by BBN......Page 228
14.2 Online Handwriting Recognition System of the University of Duisburg......Page 230
14.3 ESMERALDA Offline Recognition System......Page 232
15 Analysis of Biological Sequences......Page 234
15.1 HMMER......Page 235
15.2 SAM......Page 236
15.3 ESMERALDA......Page 237
References......Page 240
Index......Page 258