PISA 2018 Assessment and Analytical Framework

Foreword
Table of contents
1.  What is PISA?
	What makes PISA unique
	The PISA 2018 test
	An overview of what is assessed in each domain
	The evolution of reporting student performance in PISA
	The context questionnaires
	A collaborative project
2.  PISA 2018 Reading Framework
	Introduction
		Reading as the major domain
		Changes in the nature of reading literacy
		Continuity and change in the framework from 2000 to 2015
		Revising the framework for PISA 2018
		The importance of digital reading literacy
		Reading motivation, practices and metacognition
		The structure of the reading literacy framework
	Defining reading literacy
		Reading literacy...
		... is understanding, using, evaluating, reflecting on...
		...and engaging with...
		...texts...
		...in order to achieve one’s goals, to develop one’s knowledge and potential and to participate in society.
	Organising the domain
		Processes
			Text processing
				Reading fluently
				Locating information
				Understanding
				Evaluating and reflecting
			Task management processes
			Summary of reading processes
		Texts
			Source
			Organisational and navigational structure
			Text format
				Continuous texts
				Non-continuous texts
				Mixed texts
	Assessing reading literacy
		Scenarios
		Tasks
			Distribution of tasks
		Factors affecting item difficulty
		Factors improving the coverage of the domain
			Situations
			Text types
		Response formats
		Assessing ease and efficiency
		Assessing students' reading motivation, reading practices and awareness of reading strategies
			Intrinsic motivation and interest in reading
			Reading practices
			Awareness of reading strategies
			Teaching practices and classroom support for reading growth and engagement
		Considerations for adaptive testing
	Reporting proficiency in reading
		Reporting scales
		Interpreting and using the scales
		Defining levels of reading literacy proficiency
	Notes
	References
		Annex 2.A. Main changes in the reading framework, 2000-2015
		Annex 2.B. Sample tasks
	Task 1: Sample reading ease and efficiency task.
	Tasks 2-4: Sample scenario with three embedded tasks
3.  PISA 2018 Mathematics Framework
	Introduction
	Defining mathematical literacy
	Organising the domain of mathematics
		Mathematical processes and the underlying mathematical capabilities
			Mathematical processes
			Formulating situations mathematically
			Employing mathematical concepts, facts, procedures and reasoning
			Interpreting, applying and evaluating mathematical outcomes
			Desired distribution of items by mathematical process
			Fundamental mathematical capabilities underlying the mathematical processes
			Mathematical content knowledge
			Change and relationships
			Space and shape
			Quantity
			Uncertainty and data
			Desired distribution of items by content category
			Content topics for guiding the assessment of mathematical literacy
			Contexts
			Desired distribution of items by context category
	Assessing mathematical literacy
		Structure of the survey instrument
		Response formats
		Item scoring
		Computer-based assessment of mathematics
		Reporting proficiency in mathematics
	Notes
	References
4.  PISA 2018 Science Framework
	Introduction: Scientific literacy and why it matters
	Scientific literacy: Towards a definition
		The competencies required for scientific literacy
			Competency 1: Explaining phenomena scientifically
			Competency 2: Evaluating and designing scientific enquiry
			Competency 3: Interpreting data and evidence scientifically
	Organisation of the domain
		Contexts for assessment items
		Scientific competencies
		Scientific knowledge
			Content knowledge
			Procedural knowledge
			Epistemic Knowledge
	Assessment of the Domain
		Cognitive Demand
		Test Characteristics
			Item response formats
		Assessment Structure
		Reporting scales
	References
5.  PISA 2018 Financial Literacy Framework
	Introduction
		Policy interest in financial literacy
			Demographic and cultural shifts
			Risk shift and increased individual responsibility
			Increased supply of a wide range of financial products and services
			Increased demand for financial products and services
			New risks from financial products and services
		Expected benefits of financial education and improved levels of financial literacy
			Focus on youth
			Efficiency of providing financial education in schools
		The need for data
		The measurement of financial literacy in PISA
	Defining financial literacy
		Financial literacy…
		…is knowledge and understanding of financial concepts and risks…
		…and the skills…
		…motivation and confidence…
		…to apply such knowledge and understanding in order to make effective decisions…
		…across a range of financial contexts…
		…to improve the financial well-being of individuals and society…
		…and to enable participation in economic life.
	Organising the domain
		Content
			Money and transactions
			Planning and managing finances
			Risk and reward
			Financial landscape
		Processes
			Identify financial information
			Analyse information in a financial context
			Evaluate financial issues
			Apply financial knowledge and understanding
		Contexts
			Education and work
			Home and family
			Individual
			Societal
		Non-cognitive factors
			Access to information and education
			Access to money and financial products
			Attitudes towards and confidence about financial matters
			Spending and saving behaviour
	Assessing financial literacy
		The structure of the assessment
		Response formats and coding
		Distribution of score points
	The impact of knowledge and skills in other domains on financial literacy
	Reporting financial literacy
	Notes
	References
6.  PISA 2018 Global Competence Framework
	Introduction: The importance of an international global competence assessment
		What is global competence?
		Can schools promote global competence?
		Why do we need global competence?
			To live harmoniously in multicultural communities
			To thrive in a changing labour market
			To use media platforms effectively and responsibly
			To support the Sustainable Development Goals
		Should we assess global competence?
		How do we assess global competence?
		The building blocks of global competence – knowledge, skills, attitudes and values
			Knowledge about the world and other cultures
			Skills to understand the world and to take action
			Attitudes of openness, respect for people from different cultural backgrounds and global mindedness
			Valuing human dignity and diversity
	The assessment of global competence in PISA
		The assessment strategy
		The cognitive test on global understanding
			A short review of cognitive assessments in this area
			Defining the construct of global understanding
			The cognitive processes that support global understanding
			Content of the test units
			Complexity of the test units
			Format of the scenarios
			Response format
			Moderators of performance: reading comprehension, attitudes and values
		Self-reported information in the student questionnaire
			Self-reported knowledge and skills
				Self-reported knowledge of global and intercultural issues
				Self-reported ability to communicate in multicultural contexts
				Self-reported adaptability
				Self-reported perspective taking
			Self-reported attitudes
				Self-reported openness toward people from other cultural backgrounds
				Self-reported respect for people from other cultural backgrounds
				Self-reported global mindedness
			Questionnaire items on strategies, pedagogies and attitudes to teach global competence
	Conclusions
	Notes
	References
		Annex 6.A. Illustrative examples of scenarios for the cognitive assessment of global understanding
	Are global temperatures rising?
	A talented player
	A song in Quechua
		Annex 6.B. Description of possible topics for the scenarios of the cognitive test
	Culture and intercultural relations
		Identity formation in multi-cultural societies
		Cultural expressions and cultural exchanges
		Intercultural communication
		Perspective taking, stereotypes, discrimination and intolerance
	Socio-economic development and interdependence
		Economic interactions and interdependence
		Human capital, development and inequality
	Environmental sustainability
		Natural resources and environmental risks
		Policies, practices and behaviours for environmental sustainability
	Institutions, conflicts and human rights
		Prevention of conflicts and hate crimes
		Universal human rights and local traditions
		Political participation and global engagement
7.  PISA 2018 Questionnaire Framework
	Introduction
		Outline of the framework
	Defining the questionnaire core in PISA 2018
		Outline of the content covered in the questionnaires
			Non-cognitive and metacognitive constructs
			Student background
			Teaching and learning
			School policies and governance
		Previous use of contextual data from PISA: Measures that have been and will be important for analysis and reporting
			Comparability and trend items
			The PISA 2009 report
			Research publications
	Coverage of policy issues in PISA 2018
		Assessing non-cognitive and metacognitive constructs
			Reading-related outcomes (module 4)
			Dispositional and school-focused variables (module 9)
			Dispositions for global competence (module 10)
		Assessing student background
		Assessing teaching and learning processes
			Teaching practices for reading (module 2) and learning time and curriculum (module 11)
			Out-of-school experience (module 5)
			Teacher qualifications and professional development (module 1)
		Assessing school policies and governance
			School policies and approaches to educational governance
			Assessment, evaluation, and accountability (module 16)
		Dealing with response bias
	Notes
	References
8.  PISA 2018 Well-being Framework
	Executive summary
	Introduction
	Well-being as a multi-dimensional construct
	Addressing measurement challenges
		Balance single-item measures with multi-item indices
		Use a meaningful number of scale points and clearly differentiated scale labels
		Select measures that maximise cross-cultural comparability
		Consider item formats beyond traditional self-reports
		Consider alternative questionnaire designs to reduce respondent burden
	Suggested quality of life indicators
		Quality of Life as a Whole
			Life Evaluation and Life Satisfaction
			Affect/Emotional Well-being
		Self-Related Well-Being
			Health
				Objective Indicators
				Subjective Indicators
			Education and Skills
				Objective Indicators
				Subjective Indicators
			Psychological Functioning
		School-Related Well-Being
			Social Connections at School
				Objective Indicators
				Subjective Indicators
			Schoolwork
				Objective Indicators
				Subjective Indicators
			Other Potential Indicators
		Well-Being Outside of School
			Social Connections Outside of School
				Objective Indicators
				Subjective Indicators
			Material Living Conditions
				Objective Indicators
				Subjective Indicators
			Leisure Time
				Objective Indicators
				Subjective Indicators
			Other Potential Indicators
		Possible Composite Indicators
	Notes
	References
		Annex A. PISA 2018 Background questionnaires
Annex A. PISA 2018 Background questionnaires
Annex B. PISA 2018 Expert Groups