ASSESSMENT FRAMEWORKS FOR CYCLE 2 OF THE PROGRAMME FOR THE INTERNATIONAL ASSESSMENT OF ADULT... COMPETENCIES.

Foreword
Acknowledgements
Executive summary
1 The assessment frameworks for Cycle 2 of PIAAC: An introduction and overview
	Introduction
	The PIAAC assessment
		Instrumentation
	Assessment frameworks
		Coverage of the constructs
		Factors affecting the difficulty of assessment tasks
		Authenticity of tasks
		Content appropriate to the entire adult population
		Assessment at low skill levels: Reading and numeracy components
	The evolution of assessment frameworks in international adult assessments
		Developments in literacy
		Developments in numeracy
		Developments in problem solving
	Relationship of the PIAAC and PISA assessments
	The framework documents
	References
		Annex 1.A. Summary of the evolution of assessment frameworks – from IALS to PIAAC Cycle 2
	Notes
2 PIAAC Cycle 2 assessment framework: Literacy
	Introduction
		The second PIAAC study in the context of past international literacy studies
		Developments between IALS and PIAAC
		Information technology and the changing nature of literacy
		Evolution of the PIAAC Cycle 2 Literacy domain in comparison with previous frameworks
	Definition of literacy
	Core dimensions of the literacy domain
		Cognitive task demands
			Accessing text
			Understanding
			Evaluating
		Texts
			Text types
			Text format: Continuous, non-continuous and mixed texts
			Text organisation: Layout, content representation and access devices
			Source: Single vs. multiple texts
		Social contexts
	Assessing literacy
		General organisation of literacy tasks
		Response formats
		Adaptive testing design
	Recommended distribution of items
		Recommended distribution by cognitive task demands and number of sources
		Recommended distribution by context
		Distribution across other relevant dimensions
	The role of fluent reading, engagement and metacognition
	Factors driving task difficulty
	References
	Note
3 PIAAC Cycle 2 assessment framework: Numeracy
	Introduction
		Structure of the chapter
		Why have an assessment framework and construct for PIAAC?
	The assessment of numeracy in PIAAC
		Review report
		Some new issues
			Big Ideas in mathematics
			Number sense
			Embeddedness, authenticity, numerate behaviour and practices
		Rationale for assessing numeracy in PIAAC
	Conceptual and theoretical foundations
		Developing perspectives on adult numeracy
			Common use of the term numeracy in the 2000s
		Competence versus skill
		Numerate behaviour and practices
		Theoretical developments and foundations
			Contexts and demands for numeracy
				Implications of 21st century skills and demands on numeracy needs
			The role of numeracy in adults’ lives
			Numeracy in the workplace
				Work in the 21st century
				School mathematics versus everyday or workplace mathematics
			School-based perspectives on numeracy and informed civic participation
			A perspective on 21st century digital and technological implications
			Further research issues arising from the review paper
				Disposition to use mathematics
				Seeing mathematics in a numeracy situation
				Critical reflection and action
				Degree of accuracy and tolerances
				Authenticity, embeddedness and text-related reading demands
			Big Ideas in mathematics
		Towards a definition and description of numeracy for PIAAC Cycle 2
			Full range of numeracy capabilities
	Numeracy assessment construct in PIAAC Cycle 2
		Numeracy in the ALL survey
		Numeracy in PIAAC Cycle 1
		Definition of numeracy for PIAAC Cycle 2
			The major changes
		Elaboration of numerate behaviour and practices
			Enabling factors and processes
				Context/world knowledge and familiarity
				Literacy skills
				Disposition, beliefs and attitudes
				Numeracy-related practices and experiences
				Summary
		The dimensions in PIAAC Cycle 2
		Cognitive processes
			Description
			Access and assess situations mathematically (assess, identify, access and represent)
				Key questions that drive the complexity of this process:
			Act on and use mathematics (order, count, estimate, compute, measure, graph and draw)
				Key questions that drive the complexity of this process:
			Evaluate, critically reflect, make judgements (evaluate, reflect, justify and explain)
				Key questions that drive the complexity of this process:
		Mathematical content
			Description
			Quantity and number
				Illustrative examples:
			Space and shape
				Illustrative examples:
			Change and relationships
				Illustrative examples:
			Data and chance
				Illustrative examples:
		Context
			Description
			Personal
			Work
			Societal/community
		Representations
			Description
			Text or symbols
			Images of physical objects
			Structured information
			Dynamic applications
	Operationalisation of the PIAAC numeracy assessment
		Assessment developments
			Computer-based assessment of mathematics and numeracy
			Enhancing the assessment of numeracy in PIAAC Cycle 2
				21st century representation and interactivity
			A dimension for reviewing assessment possibilities
			Outcomes
		Constraints and challenges to enhancing the assessment of numeracy in PIAAC Cycle 2
			Constraints of the assessment design, platform and certain response types
			Money/currency issues
			Different measurement units
		The PIAAC approach to assessment
			Item pools and scale scores
			Usage of calculators and other tools or objects
		Basis for assessing numeracy in PIAAC Cycle 2
		Blueprint for assessing numeracy in PIAAC Cycle 2
			Cognitive processes
			Content
			Representations
			Context
		Factors explaining item/task complexity
	Relationship between PIAAC and PISA
		The commonalities between PIAAC and PISA
			Mathematical content
			Contexts
			Responses/actions
			Item formats
		General comments
			Drivers/indicators of mathematical proficiency
			Alignment of the two scales
	Numeracy components
		Introduction
			Reading component skills
		Rationale for the numeracy components assessment
		Numeracy component skills – conceptual issues
			Numeracy component skills - prerequisites or fundamentals?
			Delivery and other constraints
				Representations and reading demands
				Time
				Oral instructions and support
				Using money
				Item formats
		Numeracy component skills – possible content
			National and international frameworks
			Number sense
			Summary and where to next?
		Numeracy component skills – the scope
		The proposed numeracy components for PIAAC Cycle 2
			Linguistic issues
			Feedback from pilots
			The questions and items being asked—How many ...?
			The questions and items being asked—The biggest?
			Timing
			Automaticity/fluency
	References
	Notes
4 PIAAC Cycle 2 assessment framework: Adaptive problem solving
	Introduction
		Adapting to dynamically changing situations: The importance of adaptive problem solving
		The cognitive and metacognitive components of adaptive problem solving
	Definition of adaptive problem solving
		Explanation of the definition of adaptive problem solving
		Example tasks “Dinner Preparation” and “Stock Market”
			Example task: Dinner Preparation
			Example task: “Stock Market”
	Core dimensions of the APS domain
		Task dimensions
		Cognitive and metacognitive processes in adaptive problem solving
			Cognitive processes
				Problem definition: Mental model construction
				Search solution: Identifying effective operators
				Apply solution: Applying plans and executing operators
			Metacognitive processes
				Problem definition: Setting goals and monitoring problem comprehension
				Search for solution: Evaluating operators with respect to whether they can be executed
				Apply solution: Monitoring progress and regulating the problem-solving process
			Conclusions
		Reporting proficiency in adaptive problem solving
	Assessing adaptive problem solving
		Anchoring the APS assessment in the task dimensions
		Test design
			Test administration
			Design elements
			Demands on literacy and ICT skills
			Drivers of item difficulty
			Assessing metacognitive processes
		Item scoring and data capturing
			General scoring principles
			Log-file data
	Adaptive problem solving in the nexus of related constructs and implications for PIAAC Cycle 2
		Adaptive problem solving and literacy
		Adaptive problem solving and proficient use of information and communications technology (ICT)
		Adaptive problem solving and problem solving in technology-rich environments
		Summary and conclusion
	References
		Annex 4.A. Description of difficulty drivers