Do school league tables add any value?Posted: March 1, 2011
Some new research suggests they do!
One of the arguments against My School is that information on the performance of different schools will be interpreted incorrectly by the press, the public and parents.
The concerns usually relate to fears that schools will ‘teach to the test’; high socioeconomic status parents will abandon under-performing schools; or due regard will not be given to the special conditions that apply to some schools.
While I acknowledge there are risks with performance data, I’m suspicious of anyone who tells me that parents can’t be trusted – supposedly for both their own good and the good of their children – with full information about their child’s school.
I’ve written about My School before in the context of the ICSEA Index (here and here). With the revised My School web site going live on Friday, it’s timely to look at some recently published research on school league tables in Wales and The Netherlands (H/T Alex Tabarrok).
The research on Wales takes advantage of a ‘natural experiment’ – secondary school performance tables that have been published in both England and Wales since 1992 were abolished in Wales in 2001 but maintained in England. The research examines the effects over the subsequent years in both jurisdictions.
The University of Bristol researchers found there is “systematic, significant and robust evidence that abolishing school league tables markedly reduced school effectiveness in Wales”. They estimate the negative impact on schools is equivalent to increasing class size from 30 pupils to 38.
They also find that the negative effect is “concentrated in the schools in the lower 75% of the distribution of ability and poverty” and argue that the “results show that the policy reform in Wales reduced average performance and raised educational inequality”.
The researchers also investigate:
whether the removal of public performance information changed the sorting of students to schools, testing the leading hypothesis that the league tables facilitate segregation. In fact, we find no evidence that the policy change has had a significant impact on either sorting by ability or by socioeconomic status.
They conclude that:
school accountability policies hold promise for raising school performance, particularly for students in disadvantaged schools and neighbourhoods. If uniform national test results exist, publishing these in a locally comparative format appears to be an extremely cost-effective policy for raising attainment and reducing inequalities in attainment.
While much of the criticism of league tables relates to the fear of ‘gaming’ the system (e.g. not letting low performing students sit the test), the study finds that it only has a minor role in explaining the different outcomes in England and Wales.
Researchers from the CPB Netherlands Bureau for Economic Policy Analysis tracked 3,032 schools over the period 1996-2006 following the publication of league tables by a daily newspaper, Trouw, starting in 1997. The researchers conclude that:
The general picture that emerges from our analysis is that schools do respond to quality information by changing their quality outcomes in the short and in the long run….. Our results indicate that schools that receive a negative score are triggered to improve their long-term outcomes, with accountability incentives that can be qualified as ‘ex post’ – that is, after the occurrence of receiving a low ranking.
They also find that the influence of ‘gaming’ by schools is relatively small.
The media and others will inevitably use the My School data to assemble league tables, but these two pieces of research suggest that the benefits might be well and truly worth it. Those benefits derive from making openly available information that gives schools, parents and perhaps even students an incentive to do better.
It’s nevertheless important to mention that, like these researchers, I think schools have broader aims than preparing students for the Year 12 exam. The success or otherwise of these broader endeavours is not captured by these sorts of studies or by the NAPLAN results posted to the My School web site.