Handbook of Automated Scoring-Theory into Practice

Contents
Foreword……………………………………………………………………………………………………xi
1 The Past, Present, and Future of Automated Scoring…………………………1
Peter W. Foltz, Duanli Yan, and André A. Rupp
Part I: Theoretical Foundations
2 Cognitive Foundations of Automated Scoring…………………………………. 13
Malcolm I. Bauer and Diego Zapata-Rivera
3 Assessment Design with Automated Scoring in Mind…………………….29
Kristen DiCerbo, Emily Lai, and Matthew Ventura
4 Human Scoring with Automated Scoring in Mind…………………………..49
Edward W. Wolfe
5 Natural Language Processing for Writing and Speaking………………… 69
Aoife Cahill and Keelan Evanini
6 Multimodal Analytics for Automated Assessment…………………………..93
Sidney K. D’Mello
7 International Applications of Automated Essay Scoring……………….. 113
Mark D. Shermis
8 Public Perception and Communication around Automated
Essay Scoring…………………………………………………………………………………… 133
Scott W. Wood
9 An Evidentiary-Reasoning Perspective on Automated Scoring:
Commentary on Part I…………………………………………………………………….. 151
Robert J. Mislevy
Part II: Operational Methodologies
10 Operational Human Scoring at Scale……………………………………………… 171
Kathryn L. Ricker-Pedley, Susan Hines, and Carolyn Connelly
11 System Architecture Design for Scoring and Delivery………………….. 195
Sue Lottridge and Nick Hoefer
12 Design and Implementation for Automated
Scoring Systems………………………………………………………………………………. 217
Christina Schneider and Michelle Boyer
13 Quality Control for Automated Scoring in Large-Scale
Assessment………………………………………………………………………………………. 241
Dan Shaw, Brad Bolender, and Rick Meisner
14 A Seamless Integration of Human and Automated Scoring…………..263
Kyle Habermehl, Aditya Nagarajan, and Scott Dooley
15 Deep Learning Networks for Automated Scoring Applications…….283
Saad M. Khan and Yuchi Huang
16 Validation of Automated Scoring Systems…………………………………….. 297
Duanli Yan and Brent Bridgeman
17 Operational Considerations for Automated Scoring Systems:
Commentary on Part II……………………………………………………………………. 319
David M. Williamson
Part III: Practical Illustrations
18 Expanding Automated Writing Evaluation…………………………………….. 329
Jill Burstein, Brian Riordan, and Daniel McCaffrey
19 Automated Writing Process Analysis……………………………………………..347
Paul Deane and Mo Zhang
20 Automated Scoring of Extended Spontaneous Speech…………………..365
Klaus Zechner and Anastassia Loukina
21 Conversation-Based Learning and Assessment Environments………383
Arthur C. Graesser, Xiangen Hu, Vasile Rus, and Zhiqiang Cai
22 Automated Scoring in Intelligent Tutoring Systems………………………403
Robert J. Mislevy, Duanli Yan, Janice Gobert, and Michael Sao Pedro
23 Scoring of Streaming Data in Game-Based Assessments……………….423
Russell G. Almond
24 Automated Scoring in Medical Licensing……………………………………….445
Melissa J. Margolis and Brian E. Clauser
25 At the Birth of the Future: Commentary on Part III……………………….. 469
John T. Behrens
26 Theory into Practice: Reflections on the Handbook……………………….. 475
Andre A. Rupp, Peter W. Foltz, and Duanli Yan
Glossary………………………………………………………………………………………………….489
References……………………………………………………………………………………………… 501
Index……………………………………………………………………………………………………….553