Why do the worst infrastructure projects get built?

Inaccuracy of transportation project cost and benefit estimates by type of project, in constant prices (Flyvbjerg)

Under-estimating the cost of major infrastructure projects and over-estimating the demand is so chronic that forecasters deserve some harsh medicine, according to Professor Bent Flyvbjerg from Oxford University’s Said Business School. He says “some forecasts are so grossly misrepresented that we need to consider not only firing the forecasters but suing them too – perhaps even having a few serve time”.

Australians have plenty of experience with underperforming infrastructure projects. For starters, just in transport alone, there’s Brisbane’s Clem 7 road tunnel, Sydney’s Lane Cove and Cross City tunnels, the Brisbane and Sydney airport trains, Melbourne’s Myki ticketing fiasco, and the 2,250 km Freightlink rail line connecting Adelaide and Darwin. And they’re just the ones we know about!

Professor Flyvbjerg says cost overruns in the order of 50% in real terms are common for major infrastructure projects and overruns above 100% are not uncommon. Writing in the Oxford Review of Economic Policy, he argues that demand and benefit forecasts that are wrong by 20–70% compared with the actual outcome are also common.

Transport projects are among the worst performers (see exhibit). Professor Flyvbjerg examined 258 transport projects in 20 nations over a 70 year time frame. He found the average cost overrun for rail projects is 44.7% measured in constant prices from the build decision. For bridges and tunnels, the equivalent figure is 33.8%, and for roads 20.4%. The difference in cost overrun between the three project types is statistically significant and the size of the standard deviations shown in the first exhibit demonstrate the high degree of uncertainty and risk associated with these sorts of projects.

He also found that nine out of 10 projects have cost overruns; they happen in all nations; they’ve been a constant over the last 70 years; and cost estimates have not improved over time.

And it’s not just under-estimation of costs. Errors in forecasts of travel demand for rail and road infrastructure are also endemic. He found that actual passenger traffic for rail projects is on average 51.4% lower than forecast traffic. He says:

This is equivalent to an average overestimate in rail passenger forecasts of no less than 105.6 per cent. The result is large benefit shortfalls for rail. For roads, actual vehicle traffic is on average 9.5 per cent higher than forecasted traffic. We see that rail passenger forecasts are biased, whereas this is less the case for road traffic forecasts.

He also found that nine out of ten rail projects over-estimate traffic; 84% are wrong by over ±20%; it occurs in all countries studied; and has not improved over time.

Thus the risk associated with rail projects in particular is extraordinary. They face both an average cost overrun of 44.7% and an average traffic shortfall of 51.4%.

Professor Flyvbjerg identifies three possible causes for these errors. One is technical error – this relates to factors like inadequate data and the inherent difficulty of forecasting the future. It is the customary defence when projects fail. Another is psychological error, like optimism bias. Finally, there’re political and economic explanations, where promoters, investors and politicians deliberately under-estimate costs and over-estimate benefits.

Based on the various projects he examined, Professor Flyvbjerg rejects both technical and psychological theories as the primary cause of errors. He points the finger at political explanations – “strategic misrepresentation”, as he describes it. Promoters and forecasters have an incentive to intentionally under-estimate costs and over-estimate benefits to obtain approval and funding for their projects. This results in:

an inverted Darwinism i.e. survival of the unfittest. It is not the best projects that get implemented, but the projects that look best on paper…..Forecasting is mainly another kind of rent-seeking behaviour, resulting in a make-believe world of misrepresentation which makes it extremely difficult to decide which projects deserve undertaking and which do not.

Apart from “having a few forecasters serve time”, the key way to address the problem, he recommends, is via use of ‘reference class forecasting’. In essence, it involves comparing the project against a large sample of similar past projects to determine the range of cost and benefit miscalculations.

Professor Flyvbjerg illustrates this proposal by relating a story about a group of Israeli teachers and academics who were tasked with developing a curriculum for high schools. The team members were each asked to give their estimate of the time the project would take – their estimates ranged from 18 months to 30 months. They then asked a distinguished curriculum expert how long similar projects he’d been involved with had taken. After some consideration he replied that about 40% never finished; of the remainder, he could think of none that completed the task within seven years.

Virginia Postrel inteviewed Professor Flyvbjerg for Bloomberg. He told her when ‘reference class forecasting’ was introduced in the UK it stopped a number of projects dead in their tracks. “This has never happened before”, he told her. However the world’s biggest infrastructure projects, like HSR lines in China, are not subject to such testing. Data on these projects is simply not reliable – “if the party says there’s no cost overrun, there’s no cost overrun”. No wonder promoters look so longingly at China, says Postrel, where infrastructure glamour is the law.

Rail-ridership predictions are especially over-optimistic in the U.S. Postrel reports the average gap between expectations and reality is 60 percent, compared with 23 percent in Europe. Using Professor Flyvbjerg’s findings, her back-of-the-envelope calculation suggests California High-Speed Rail can expect to carry only 15.6 million passengers a year by 2035, rather than the 39 million projected.

Professor Flyvbjerg focuses on transport projects in this paper, but he says the same problems apply to other project types including ICT systems, buildings, aerospace projects, defence, mega-events such as the Olympics and the World Cup, water projects, dams, power plants, oil and gas extraction projects, mining, large-scale manufacturing, big science, and urban and regional development projects.

While things are bad enough with transport infrastructure, they’re even worse with ICT:

if a major project is not already messed up, injecting a good dose of ICT will do the job….As if it were not difficult enough to develop, say, a major new airport, we are now developing airports that depend on major new ICT for their operations, and we pay the price…… if you are doing ICT as part of a major project (or as a major project in itself) be sure to get ICT that has been developed and debugged elsewhere and that has a proven track record in daily use.

Hmmm….I can think of a few governments in Australia that wished they’d taken that advice.

21 Comments on “Why do the worst infrastructure projects get built?”

  1. wizofaus says:

    And yet I do wonder…in how many of these projects with big cost overruns and underutilisation (passenger rail services in particular) has it become blatantly obvious that building it never made sense in the first place? And how much account is taken of the indirect savings that come from building such infrastructure?

  2. wizofaus says:

    Well sure, there’s plenty of stupid promises…but I’m asking about projects that have been completed and have had sufficient time (e.g. a decade at least) to get meaningful utilisation numbers on.

    It just goes against everything I’ve read and come to understand over the last few years to suddenly hear that rail projects are less likely to be economically justifiable in the long run than equivalent road-based alternatives.

    • Alan Davies says:

      Have a look at his paper – Professor Flybvjerg mentions a number of projects from around the world.

      There’s lots of politics around heavy and light rail projects. Just look at the last Fed and recent State elections in Australia. So we might be getting more rail projects but we’re not always getting the best.

      • James says:

        What did you have in mind as “best” Alan?
        I skimmed over most of Professor Flyvbjerg’s article and may have missed a few things but could find no mention of negative externalities nor the benefit cost ratio of projects over the useful life of a given project (which I understand it taken to be ~50 years). Did I miss something?

        • Alan Davies says:

          I was responding there to wizofaus’s 2nd para and talking about Australian rail projects. It seems to me some of the projects touted in Melbourne – like the Avalon, Doncaster, Rowville rail lines – are likely to have low benefits and are only really on the table because of the sorts of political pressures Flyvbjerg refers to. I’d expect something like the Melbourne Metro to stack up better.

          • I suspect some biases with your assessment here Alan.

            The main benefit of the “Melbourne Metro” is the extra capacity it will supposedly provide the network through the CBD. Extra capacity that the City Loop was meant to provide 30 odd years ago!

            The City Loop ran well over cost and didn’t produce the benefits it was meant to provide. Do you really believe the Melbourne Metro would not suffer the same fate?

            Of course some people argue that the loop does in fact have the capacity to run more trains, providing some operational changes. Personally I’m in this camp. It doesn’t take an expert to see that there are some major bottlenecks that come primarily from operational errors.

          • Alan Davies says:

            Julian, based on Professor Flyvbjerg’s findings there is a very good chance the Metro project would go well over budget! I was thinking though, in response to James’ question about “best” projects, that it would have much higher benefits than something like Doncaster rail.

            I’m not (yet) conversant enough with the alternatives to the Metro to take a firm view on which approach should be adopted, but whichever it is, I expect it would still come out streets ahead of Doncaster in terms of the ratio of benefits to costs (as probably would some other alternative projects, like eliminating the worst level crossings).

          • “like eliminating the worst level crossings”

            Now that sounds like something well worth doing!

            I know I preach his work a heck of a lot, but if you haven’t read Paul Mees report regarding the rail tunnel its worth reading. The Metro idea may not be likely any time soon because of the change in Government, but he does discuss the current operating inefficiencies, and offers much cheaper alternatives to building any major new infrastructure.

            It’s 36 pages long, and like most of his writing, it is easy to follow, even when discussing technical details.

            Does Melbourne need another central city rail tunnel?.

          • krammer56 says:

            Agree re: the rail projects you list – at least Metro One would also have significant city-shaping benefits!

  3. RED says:

    The comment about ICT is apropos to the Myki situation in Victoria. If better research had been done on the risk of developing a new system from scratch, maybe the government would have done the intelligent thing and dropped transport zones so that it could use an existing ticketing system.

  4. Michael says:

    Does Professor Flyvbjerg comment on scope creep? Sorry I’m too tied up to do the research myself. It occurs to me that where a lot ICT projects go awry is when additional features get added or increased in complexity after the project is initially started. In fact this seems to be so common that projects are often deliberately underquoted by vendors safe in the knowledge that they will make fat profits on providing all the additional features once the project gets started and everyone and his or her dog starts chipping in with “wouldn’t it be good to have xyz… in the sytem.

  5. wizofaus says:

    I skimmed through the paper and spent a bit time reading about the Channel Tunnel (the first example he gives), and I have to say it’s hard not to avoid the conclusion than the money could have been far better spent in other ways (though I gather it is at least now operating at a profit).
    However I’d be very wary about assuming that just because there have been a few spectacular failures of forecasting with particular rail projects that all rail project forecasts in general should be viewed especially skeptically.

    • Alan Davies says:

      His figures are averages, so definitely it can’t be assumed that all rail cost/benefit forecasts are dodgy. However the levels of inaccuracy he finds are so high it’d be sensible to look very warily at any rail forecasts.

      • wizofaus says:

        I’d say very warily at ALL forecasts, and extremely wairly at all forecasts for the largest projects, especially those including very long tunnels underwater! It guess it’ll take a bit more to convince me out of the ‘rail needs all the good publicity it can get’ camp 🙂

        One thing I can’t seem to find with regard to the chunnel though is whether the CTRL (High Speed 1) project has had any impact on its viability since opening.

        • wizofaus says:

          I did find passenger numbers (in millions) on the Eurostar wikipedia page.

          2005 7.45
          2006 7.85
          2007 8.26
          2008 9.1
          2009 9.2
          2010 9.5

          That’s a fairly significant step up since completion of the CTRL, all during a period of ‘economic uncertainity’ as we like to euphemistically call it. If it really were a complete boondoggle there would be people predicting its eventual shutdown. In reality most observers seem to feel it will only become more important as fast rail networks through Europe are expanded.
          Plus I will say…almost certainly it will still be there in centuries (in some form or another), as a testament to the technological ingenuity of our age…that’s not something you can really put a dollar value on.

  6. Moss says:

    Alan, did he look at the impact of ownership? ie do private infrastructure projects (admittedly few and far between) tend to be better at estimating costs and utilisation than public projects? I imagine that if a company were to be at risk that would modify the estimates to be more conservative? Probably difficult to tease the complexities out, as every infrastructure project involves some level of politicisation.

  7. krammer56 says:

    Flyvbjerg’s work has been around for a while, but cost overruns persist.

    In my experience with a range of infrastructure projects in different places, there are common elements to the issue:
    * projects are announced by politicians wanting a headline to be seen to be doing something long before sufficient (or any) work has been done;
    * insufficient time and money is made available to really plan the project properly;
    * once the project is announced, the scope gets fiddled with – usually with some more gold plate;
    * there is no doubt optimism bias towards low costs and high benefits is alive and well – but it exists at all levels from the project team, manager and sponsor (especially including politicians);
    * large complex projects such as major bridges or tunnels and big rail projects are done rarely, so continuity of expertise (both in planning and construction) is not always available;
    * the understanding and pricing (and fudning) of risk, especially early in a project, is generally abysmal, with approaches generally limited to “stick a 20% contingency on”;
    * funding is often end-loaded, causing additional costs in completing most of the project in a rush to meet deadlines.

    Fortunately, Infrastructure Australia has at least taken a more cautious approach and funded detailed planning for really risky projects (e.g. Brisbane Inner City Rail and Melbourne’s Metro One projects) so hopefully both the costs and benefits are more realistic.

    One way of improving this may be to require an independent project audit, say by the Auditor General, before funding can be committed to projects over a certain value. The AG would be required to sign off on costs and benefits – the politicians would still need to decide whether the benefits outweigh the costs for society.

    This would require better and more comprehensive planning to occur and at least install a safety valve in the process.

    It would be much better than the AG doing an audit afterwards on a project business case that was a belated attempt by the bureaucrats to justify a silly political promise for a project that nobody really thought was a goer!

  8. Johnyboy says:

    I agree about the need to have continiuty of expertise to get these things right. Id like there to be federal government department that costs projects for the states. I also want them to cos the projects for the military. This departments policy is to get the forcasts right. If they are wrong ,there needs to be corrective action. We need something they can aim for. Say +-20%.

    I want to see a process available and transparent to the public. The statistical modelling and other things on a website. We need to see some process to get improvements on it. That there sole directive. Id like it to be independent like the reserve bank. It might even be part of the reserve bank. Id like it to do costings for all the public works in australia. So we at least know the truth about things. I want this department to have a time for submissions from the public.

    I think an earilier post has hit the nail on the head about the need to keep the expertise. I think the continiuity of expertise is critical. Its like the forcasting for the weather. Maybe the same people who forcast the weather could do this.

    I also want there to be restrictions on where they can work after they work there and also restrictions on where they can work before they work there. I also want restrictions on the type of work they can work on after they retire. The aim is to make corruption difficult if not impossible. It might be restricted to academics.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s