Is it healthy to assume correlation means causation?

Association of obesity trend in the US with vehicles miles travelled (left) and with Justin Wolfer's age (right)

The link between the physical environment and health outcomes like obesity is fraught. The Victorian Legislative Council’s Environment and Planning References Committee should bear this in mind as it goes about its new inquiry into the contribution of environmental design to public health.

The Committee might want to start with the first chart in the accompanying exhibit, which comes from a recent issue of The Economist and purports to show that obesity has increased in the US in line with the increase in miles driven over the last 15 years. The chart is based on work done by researchers at the University of Illinois who found “a striking correlation between these two variables – but with a large time lag……This near-perfect correlation (99.6%) permits predictions about obesity rates”.

Expatriate Australian economist and Wharton business school Assoc Professor, Justin Wolfers, points out the folly of this claim. It is, in his words, a “nonsensical correlation”:

When you see a variable that follows a simple trend, almost any other trending variable will fit it: miles driven, my age, the Canadian population, total deaths, food prices, cumulative rainfall, whatever.

To demonstrate his point, Professor Wolfers prepared the second chart showing an even better correlation between changes in obesity over the period and changes in his age – he didn’t even need to resort to a time lag to get such a good fit! He acknowledges The Economist offered the customary caveat that correlation does not equal causation but this chart, he says, is so completely unconvincing as to warrant a different warning: “Not persuasive enough that you should bother reading this article” (in the interests of balance, here’s The Economist’s subsequent response to Professor Wolfer’s charge).

This exchange highlights a problem with much of the research that purports to show the physical environment — particularly density and/or public transport access —  has a strong effect on health-related variables like obesity. There’s plenty of evidence of correlation but not much evidence of causation. There’s no doubt obesity is inversely related to both density and access to public transport, but if it turns out these aren’t the underlying drivers of obesity then the economic cost of misdirected policies could potentially be significant.

There are special reasons why it’s hard to establish causation when dealing with real life infrastructure projects and transport/land use programs. These British epidemiologists reviewed 77 international studies examining the effectiveness of policy interventions to reduce car use. They concluded the evidence base is weak, finding only 12 were methodologically strong – and they mainly involved relatively small-scale initiatives like providing better information about travel options or direct financial incentives to reduce driving (incidentally, only half of those 12 actually worked i.e. reduced car use).

The key problem is it’s enormously difficult with infrastructure projects and real-world operational programs to set up the sort of rigorous randomised trials that epidemiologists are accustomed to doing in relation to new drugs. It’s too hard to set up a double blind trial to evaluate the impact of a new rail station or a new fare structure. In most cases built environment and transport researchers have to settle for less conclusive ‘before and after’ and ‘cross-sectional’ studies.

So policy-makers have to be very careful about what they infer from the correlations they see between physical and behavioural variables. This applies to research on topics like obesity, as I’ve previously pointed out here, here, here and here. I’ll look at this issue again shortly, but it’s very likely plausible planning and transport policies can’t really do much about obesity at a social level and would in any event probably be extraordinarily inefficient ways of tackling this serious social issue.

What’s disappointing though is far too many of the interventions the epidemiologists found wanting actually would be suited, albeit with difficulty, to a more rigorous methodology — many of them involve nothing more complex than improved information (not that medical researchers haven’t had a difficult history themselves in introducing more rigorous trials — see, for example, The Emperor of all Maladies, a fascinating account of the development of cancer research). I’ll also look at some of these sorts of small-scale interventions shortly.