Do school league tables add any value?

Click to generate Malcolm's next bestseller!

Some new research suggests they do!

One of the arguments against My School is that information on the performance of different schools will be interpreted incorrectly by the press, the public and parents.

The concerns usually relate to fears that schools will ‘teach to the test’; high socioeconomic status parents will abandon under-performing schools; or due regard will not be given to the special conditions that apply to some schools.

While I acknowledge there are risks with performance data, I’m suspicious of anyone who tells me that parents can’t be trusted – supposedly for both their own good and the good of their children – with full information about their child’s school.

I’ve written about My School before in the context of the ICSEA Index (here and here). With the revised My School web site going live on Friday, it’s timely to look at some recently published research on school league tables in Wales and The Netherlands (H/T Alex Tabarrok).

The research on Wales takes advantage of a ‘natural experiment’ – secondary school performance tables that have been published in both England and Wales since 1992 were abolished in Wales in 2001 but maintained in England. The research examines the effects over the subsequent years in both jurisdictions.

The University of Bristol researchers found there is “systematic, significant and robust evidence that abolishing school league tables markedly reduced school effectiveness in Wales”. They estimate the negative impact on schools is equivalent to increasing class size from 30 pupils to 38.

They also find that the negative effect is “concentrated in the schools in the lower 75% of the distribution of ability and poverty” and argue that the “results show that the policy reform in Wales reduced average performance and raised educational inequality”.  Read the rest of this entry »

Does My School have lessons for planners?

The brouhaha over My School is an important lesson for those of us interested in geography because it demonstrates how misleading a reliance on average values can be.

In a previous post, Limitations of My School, I pointed to some problems with the Index of Community Socio-educational Advantage (ICSEA) used on the My School web site. In particular, I was surprised to find that prestigious Melbourne Grammar scored much the same on ICSEA as nearby State school, Camberwell High.

This unlikely result was because each school’s ICSEA rating is calculated from the social and economic characteristics of the Census Collection Districts in which its students live.

Camberwell High School

The problem is that although the average socioeconomic status of Camberwell and surrounds is very high, there are nevertheless significant numbers of residents whose incomes don’t extend to finding upwards of $20,000 per annum in post tax income to fund a private school education for each of their children.

In other words, Camberwell High ranks highly on ICSEA because its students live in ritzy areas, not because the students themselves come from relatively well-off families (although some former students, like Kylie Minogue, have gone on to considerable financial success).

What this tells us is that not all households within a cluster of contiguous suburbs earn the average income – there’s considerable variability even though the suburbs might, on average, be quite affluent.

To throw further light on this issue, I looked at the incomes of families in Ivanhoe East at the 2006 Census to try and get a handle on the degree of variability in income within a small geographical area. Ivanhoe East is around the same distance from the CBD as Camberwell and has 984 families. It has about six times as many dwellings as a Census Collector’s District (Camberwell has 5,011 families). Read the rest of this entry »

More on My School

Following up on my comments about My School on 5 March, Limitations of My School, there is great post by Sam Wylie on Core Economics today, School performance data. While he doesn’t address my issue of concern (the accuracy of the social profiling methodology used for comparison purposes), he makes a number of other good points. In particular, I thought his closing, where he addresses the issue of league tables, was well put:

“In the matter of school testing there is always the presentation of an argument that parents and the press will not interpret the data correctly.  The people can’t be trusted.  How often does that idea come up in different spheres?  We shouldn’t devolve power or information to individual households because they cannot be trusted not to abuse it.  Those types of arguments are always based on the protection of private interest (teaching unions) or elitism”.

Limitations of My School

The My School web site launched by the Deputy Prime Minister earlier this year has some important lessons for any disciplines that rely heavily on spatial measures to capture social and economic information.

With two school age children, I’m sympathetic to Julia Gillard’s push for more information to be made available on schools’ performance.  I think providing the NAPLAN information is a good start and I’m looking forward to seeing time-based data next year. Read the rest of this entry »