The nature of problem solving. Using research to inspire 21st century learning

Foreword......Page 5
Acknowledgements......Page 7
Table of contents......Page 9
Executive summary......Page 17
Problem solving: Overview of the domain......Page 19
The development and assessment of problem solving in 21st-century schools......Page 21
Introduction......Page 22
Educational methods aimed at improving the quality of knowledge......Page 24
Developing the scope of international assessment programmes......Page 27
Conclusions for further research and development......Page 29
References......Page 30
Analytical problem solving: Potentials and manifestations......Page 35
Introduction......Page 36
Analytical problem solving as a competence......Page 38
Training in analytical problem-solving competence......Page 42
Summary and discussion......Page 44
References......Page 45
Problem solving: Understanding complexity as uncertainty......Page 49
Introduction......Page 50
Complex problem solving: Everyday examples......Page 51
Complex problem solving: Empirical examples......Page 52
Complexity by any other name: Uncertainty......Page 53
Table 3.1 The four possible outcomes of decisions based upon the controllability of a situation......Page 55
Practical solutions to practical problems......Page 57
Figures......Page 58
Problem solving from a mathematical standpoint......Page 61
Mathematicians’ views of problem solving......Page 62
What is a problem?......Page 63
The role of problem solving in the development of mathematics......Page 65
Students’ problem solving as viewed through PISA......Page 66
The role of metacognition in mathematical problem solving......Page 68
References......Page 72
Dynamic problem solving as a new perspective......Page 75
The PISA 2012 assessment of problem solving......Page 77
Major issues identified......Page 78
Evolution of the new problem-solving framework......Page 80
The key features of the PISA 2012 problem-solving framework......Page 83
Developing the consistency......Page 84
Figure 5.3. MP3 player: Item 2......Page 87
Figure 5.5. MP3 player: Item 4......Page 88
References......Page 89
Annex 5.A1. The PISA 2012 Problem Solving Expert Group......Page 92
Annex 5.A2. Countries and partner economies participating in the OECD PISA 2012 problem-solving assessment......Page 93
Interactive problem solving: Exploring the potential of minimal complex systems......Page 95
Introduction......Page 96
Interactive problem solving......Page 97
Measuring interactive problem solving......Page 98
The philosophy behind minimal complex systems......Page 99
The basic elements of minimal complex systems......Page 100
Table 6.1 State transition matrix of a fictitious finite state automaton......Page 102
Discussion......Page 103
References......Page 105
The history of complex problem solving......Page 109
Introduction......Page 110
Human failures and strategies......Page 111
Figure 7.1 Screenshot of a simulation of the complex Moro problem......Page 112
Assessment of complex problem solving......Page 114
Discussion......Page 116
Notes......Page 118
References......Page 119
Empirical results......Page 125
Empirical study of computer-based assessment of domain-general complex problem-solving skills......Page 127
Technology-based assessment and new areas of educational assessment......Page 128
From static to dynamic problem solving with reference to reasoning skills......Page 129
Methods......Page 130
Results......Page 132
Discussion......Page 136
References......Page 139
Factors that influence the difficulty of problem-solving items......Page 143
Characteristics that might influence task difficulty......Page 144
Described levels for task characteristics......Page 148
Sample items with ratings......Page 151
Analysis and results......Page 153
Discussion......Page 157
Notes......Page 158
References......Page 159
Assessing complex problem solving in the classroom: Meeting challenges and opportunities......Page 161
The Genetics Lab: A microworld especially (PS) developed for the classroom......Page 162
Challenge 1 – Digital natives......Page 165
Challenge 2 – Scoring......Page 169
Table 10.2. Means, standard deviations, reliability and intercorrelations 
of the Genetics Lab’s performance scores......Page 172
References......Page 173
New indicators......Page 177
Log-file data as indicators for problem‑solving processes......Page 179
What are log files?......Page 180
Figure 11.2. Sample log-file record from a text file recorded by the interactive Laughing Clowns task......Page 181
Table 11.1. A possible partial credit framework for scoring an indicator describing the quality of exploration......Page 185
Discussion......Page 189
References......Page 190
Educational process mining: New possibilities for understanding students’ problem-solving skills......Page 195
Background: From logs to knowledge......Page 196
Applying process mining to problem-solving behaviour data......Page 199
Figure 12.9. Decision tree for a complex problem-solving task......Page 208
References......Page 209
EcoSphere: A new paradigm for problem solving in complex systems......Page 213
The EcoSphere......Page 214
The EcoSphere program: BioSphere scenario......Page 218
Outlook and conclusion......Page 222
References......Page 223
Future issues: Collaborative problem solving......Page 227
Assessment of collaborative problem‑solving processes......Page 229
Introduction......Page 230
Collaborative problem-solving skills: A framework for understanding......Page 231
Use of the collaborative problem-solving framework......Page 234
Figure 14.4. Asymmetric task: “Olive Oil”......Page 238
Analysis......Page 240
Table 14.2. Correlation matrices for Student A and Student B across the social and cognitive strands and their components......Page 242
References......Page 243
Assessing conversation quality, reasoning, and problem solving with computer agents......Page 247
Introduction......Page 248
Learning environments with conversational agents......Page 249
Conversational dialogues with AutoTutor......Page 251
Trialogues......Page 254
Closing comments......Page 258
References......Page 259
Finale......Page 265
Epilogue......Page 267
Figure 4.1 Problem Situation......Page 64
Figure 4.3 A mathematician‘s problem-solving activities over time......Page 70
Figure 4.4 A pair of students’ activity-time allocation after a problem-solving course......Page 71
Figure 6.2 A simple finite state automaton......Page 101
Figure 7.2 Screenshot of the Tailorshop problem......Page 115
Figure 7.3 Screenshot of the finite state machine HEIFI......Page 117
Figure 8.2 Example of tasks in the inductive reasoning test......Page 131
Figure 8.4 Developmental curve of dynamic problem solving by school type......Page 134
Figure 8.5 Correlations between inductive reasoning, intelligence, domain-specific 
and domain-general problem solving......Page 135
Table 9.2 Ratings for MP3 Player Item 2 and Birthday Party Item 1......Page 152
Figure 9.3. Dendrogram showing clustering of 10 item characteristics......Page 155
Table 10.1. Sample characteristics of the Genetics Lab studies......Page 163
Figure 10.3. Genetics Lab Task 3: Changing the characteristics......Page 164
Figure 10.4. Start screen for a creature......Page 166
Figure 10.6. Adapted test development process of the Genetics Lab......Page 167
Figure 10.8. Acceptance of the Genetics Lab among students......Page 168
Figure 11.3. Temporal evidence map segment illustrating hypothetico-deductive reasoning from the single-player Laughing Clowns task......Page 186
Table 11.2. Example of a tuneable scoring rule for the “time to first action” indicator......Page 187
Figure 11.6. Temporal evidence map segment illustrating uncertainty about the problem from the single-player Laughing Clowns task......Page 188
Figure 12.1. Web search item......Page 197
Figure 12.3. The knowledge discovery process......Page 198
Figure 12.4. Aggregated test-taking processes of students on Item 9 and Item 11 (directed graph)......Page 201
Figure 12.5. Example of a MicroDYN item......Page 202
Figure 12.6. Aggregated problem solving behaviour on seven tasks, activities in the first three executions......Page 203
Figure 12.8. Decision tree for the job search task......Page 207
Figure 13.1. Screenshot of a scenario introduction......Page 219
Figure 13.2. The interface for drawing computer-based causal diagrams......Page 220
Figure 13.3. The first BioSphere scenario consists of two exogenous and three endogenous variables......Page 221
Figure 14.1. Framework for collaborative problem solving......Page 232
Figure 14.2. Illustration of an assessment task and of a stimulus-response-to-code structure......Page 236
Figure 14.3. Symmetric task: “Laughing Clowns”......Page 237
Tables......Page 241
Table 2.1 Results of expert rating of PISA 2003 test items......Page 40
Table 2.2 Intraclass correlation and relations with external variables for the three‑dimensional model of analytical problem-solving competence: Mean differences and correlations......Page 41
Table 8.2 Goodness of fit indices for measurement invariance of DPS in the MicroDYN approach......Page 133
Table 9.1. Proposed task characteristics affecting item difficulty......Page 146
Table 9.3. Standardised regression coefficients......Page 154
Table 9.4. Rotated component matrix......Page 156
Table 12.2. Clustering test takers based on problem-solving behaviour in an online environment......Page 204
Table 12.4. Clustering students based on online behaviour on the MicroDYN item......Page 205
Table 13.1. Information processing and interaction with the system......Page 217
Table 14.1. Components, strand elements and indicative behaviour needed for collaborative problem solving......Page 233