OECD - PISA 2018 Assessment and Analytical Framework

The OECD Programme for International Student Assessment (PISA) assesses the extent to which 15 year old students near the end of their compulsory education have acquired the knowledge and skills that are essential for full participation in modern societies. The assessment does not just ascertain whether students can reproduce knowledge; it also examines how well students can extrapolate from what they have learned and can apply that knowledge in unfamiliar settings, both in and outside of school. This approach reflects the fact that modern economies reward individuals not for what they know, but for what they can do with what they know.

The triennial assessment, launched in 1997, focuses on the core school subjects of reading, mathematics and science. Students proficiency in an innovative domain is also assessed; in 2018, this domain was global competence. This publication presents the theory underlying the PISA 2018 assessment the seventh since the programmes inception. It includes frameworks for assessing the three core subjects of reading, mathematics and science, the framework for the third assessment of students financial literacy, and the framework for assessing the innovative domain, global competence. These chapters outline the content knowledge that students need to acquire in each domain, the processes that students need to be able to perform, and the contexts in which this knowledge and these skills are applied. The publication also discusses how each domain is assessed. It concludes with the frameworks for the various questionnaires distributed to students, school principals, parents and teachers, and the framework for the new well-being questionnaire distributed to students.

In PISA 2018, reading was the major domain of assessment, as it was in 2000 and 2009. The three assessment subscales used in 2000 and 2009 were renamed locating information, understanding and evaluating and reflecting for 2018. Two new subscales were also used to describe students literacy with single-source and multiple-source texts. The reading scale was also extended by adding Level 1c, which better describes the proficiency of the lowest-performing students.

PISA is the product of a collaborative effort between OECD and the governments of both OECD countries and its partner countries/economies. The assessments are developed co-operatively, agreed by participating countries/economies, and implemented by national organisations. The co-operation of students, teachers and principals in participating schools has been crucial to the success of PISA during all stages of development and implementation.

The reading framework was developed by the reading expert group with the guidance of John de Jong and Peter Foltz from Pearson. The reading expert group was chaired by Jean-Franois Rouet (University of Poitiers, France). Other experts who contributed to the reading framework are Paul van den Broek (Universiteit Leiden, the Netherlands), Kevin Chung (University of Hong Kong, China), Sascha Schroeder (Max Planck Institute for Human Development, Berlin, Germany), Sari Sulkunen (University of Jyvskyl, Finland; also served as the liaison to the PISA global competence expert group), and Dominique Lafontaine (Universit de Lige, Belgium; also served as the liaison to the PISA questionnaire expert group).

The global competence framework was developed by Mario Piacentini of the OECD Secretariat with Martyn Barrett (University of Surrey, Guildford, UK), Veronica Boix Mansilla (Harvard University and Project Zero, Cambridge, USA), Darla Deardorff (Duke University, Durham, USA) and Hye Won Lee (Korea Institute for Curriculum and Evaluation, Jincheon, Korea), with additional help from Rose Bolognini and Natalie Foster (OECD Secretariat), Natasha Robinson (University of Oxford, UK) and Mattia Baiutti (Fondazione Intercultura, Colle di Val dElsa, Italy and the University of Udine, Italy). This framework built on earlier work from experts who led the first part of the development of the global competence assessment: Darla Deardorff (Duke University, Durham, USA), David Kerr (University of Reading, UK and YoungCitizens, London, UK), Peter Franklin (HTWG Konstanz University of Applied Sciences, Germany), Sarah Howie (University of Pretoria, South Africa), Wing On Lee (Open University of Hong Kong, China), Jasmine B Y Sim (National Institute of Education, Singapore), and Sari Sulkunen (University of Jyvskyl, Finland).

The framework for the PISA 2018 questionnaires was developed by the questionnaire expert group with the guidance of John de Jong and Christine Rozunick from Pearson. The questionnaire expert group was chaired by Fons van de Vijver (Tilburg University, the Netherlands; the North-West University, Potchefstroom, South Africa; and the University of Queensland, Brisbane, Australia). Other experts who contributed to the development of the questionnaire framework are Dominique Lafontaine (Universit de Lige, Belgium), Sarah Howie (University of Pretoria, South Africa), Andrew Elliot (University of Rochester, USA), Therese Hopfenbeck (University of Oxford, UK) and David Kaplan (University of Wisconsin-Madison, USA).

The framework for the well-being questionnaire was developed by Jonas Bertling (ETS). All other frameworks were based on versions developed for previous PISA cycles.

Pearson facilitated the development of the reading and questionnaire frameworks. The Educational Testing Service (ETS) was responsible for managing and overseeing this survey, developing the instruments, scaling, analysis, and developing the electronic platform. Other partners or subcontractors involved with ETS include the Department of Experimental and Theoretical Pedagogy at the Universit de Lige (aSPe) in Belgium and the Educational Measurement and Research Centre (EMACS) of the University of Luxembourg in Luxembourg. Westat assumed responsibility for survey operations and sampling with the subcontractor, the Australian Council for Educational Research (ACER). cApStAn Linguistic Quality Control assumed responsibility for ensuring the linguistic equivalence of all language versions.