The PISA ranking: a reliable indicator of students’ capabilities?
This month the triennial PISA ranking – the Programme for International Student Assessment – was published, the seventh round of assessments since the launch of the programme in 2000. Testing students’ knowledge and skills in reading, mathematics and science, the OECD describes PISA as “not only the world’s most comprehensive and reliable indicator of students’ capabilities” but also as “a powerful tool that countries and economies can use to fine-tune their education policies”.
PISA emphatically looks at the future in developing its vision and methodology. As individuals are increasingly rewarded not just for what they know, but for they can do with what they know, PISA explains that it tests very specific skills: “Students have to be able to extrapolate from what they know, think across the boundaries of subject-matter disciplines, apply their knowledge creatively in novel situations and demonstrate effective learning strategies”.
The PISA ranking is the result of tests conducted in 79 countries, in at least 150 schools per country, and with most countries assessing between 4,000 and 8,000 students. The 2018 ranking produced some remarkable results, making some countries proud and others disappointed or vexed. The students in four provinces in Eastern China outperformed their peers in all of the other participating countries by a wide margin. Even more remarkable is that the level of income in these regions is well below OECD average. Something similar can be said of Estonia, the top performing country amongst the OECD countries even though its expenditure per student remains about 30% lower than the OECD average. The PISA report is particularly clear about the relation between financial means and education: “History shows that countries with the determination to build a first-class education system can achieve this even in adverse economic circumstances, and their schools today will be their economy and society tomorrow. So it can be done”.
Some western countries are worried or annoyed about the latest ranking. The Netherlands is one of seven countries that have been scoring worse in all three domains since first participating in 2003, and reading scores even dropped below the OECD average. The PISA ranking set off some alarm bells, as 25% of Dutch students are said to not have the reading skills needed to participate in today’s society. Although this is attributed to social media and shorter attention spans, it does not explain the markedly worse results compared to France and Germany. PISA itself also acknowledges that the data does not really say much about cause and effect: “The results offer a snapshot of education systems at a certain moment in time; but they do not – they cannot – show how the school systems got to that point, or the institutions and organisations that might have helped or hindered progress”.
Luxembourg, too, is disappointed but its commentators are also highly critical of PISA, arguing not only that its standards and regulations are so vague that countries can easily manipulate the process but also that the test itself is poorly drafted. It is unclear whether the growth of pupils from immigrant background (55% now compared to 40% in 2015) is part of the explanation why Luxembourg dropped below the OECD average. Germany does find a correlation between the PISA results and immigration. The President of the Federation for Teachers notes that 200,000 refugee children have had to be accommodated in the German school system since 2015. More generally, if children with a migration background were excluded from the test, Germany would move from place 20 to place 8. It is therefore open for debate whether some of the top scoring countries have a serious advantage for being very homogenous societies, and whether this is properly reflected in the test outcomes.
Finally, some commentators have pointed to the possible role of austerity measures. The countries where results dropped most rapidly seem to be Eurozone countries whose education systems were heavily affected by budget cuts, including the Netherlands, Greece and Cyprus. This shows that an area where having a long-term vision is key can be seriously impacted by a focus on short-term gains.
Edited by: Dr. Olivier Vonk