Is the Transport Department coming out on outcomes?

Posted: September 19, 2011 | Author: | Filed under: Management, Public transport | Tags: , , , , , , , , , | 2 Comments

xkcd on the value of time (the corresponding figure in Australia is circa 4 minutes)

I don’t ordinarily read annual reports but this story in Melbourne’s Herald-Sun acknowledged its source was the Victorian Department of Transport’s 2010-11 annual report. The newspaper noted a 3.4% increase in reported crime on Victoria’s public transport system in 2009-10. Much to my surprise, the paper acknowledged up front that patronage had also increased over the period, by 1.6%. That’s a pertinent qualification but it surprised me that a News-owned tabloid bothered to make it. Well done to the Hun and reporter Peter Rolfe – a big tick!

Looking at the performance outcomes section of the annual report, I see the Department’s been clever – its performance indicator for reported crime against the person is effectively the change in offences minus the change in patronage growth. Bit hard to ignore that and it certainly helps to put the crime figure in perspective.

This got me wondering what progress the Department is making in dealing with crime on public transport. What, I wondered, is the trend? How does the latest figure compare with the previous year? This was when I discovered some serious deficiencies in the way the Department reports on outcomes.

A key shortcoming is the annual report fails to give the previous year’s outcome figure. So, for example, the report tells us that the total volume of freight carried by rail in Victoria increased by 8.31 billion tonne-kilometres in the most recent year for which data is available, but provides no indication of whether that is an improvement on the trend or a deterioration. What is the reader to make of this number?

Similarly, bicycle path use increased 2.8% and rail capacity utilisation in the morning peak grew by 9.7%, but in neither case is there a figure on the previous year’s outcome. This is in stark contrast with the financials, which use the convention of showing 2011’s numbers against the previous years. The examples I’ve cited aren’t isolated cases – none of the performance outcome results show the previous year.

The importance of context is shown by the Herald-Sun’s report. Peter Rolfe did some legwork and found reported crime dropped 10% in the previous year, while patronage grew 6%. That’s much better than the latest year’s 3.4% vs 1.6%, suggesting a marked deterioration in performance. Although he failed to acknowledge the difference between the two years might in part be due to increased policing (a cross!), he nevertheless illustrates the value of having something against which to compare performance measures.

Unfortunately, the shortcomings of the Department’s annual report get worse. While each indicator has a target “increase” or “reduction”, most of the “results” actually provide a snapshot or balance statistic – they don’t show the change. For example, one outcome indicator purports to show the increase in the proportion of trips starting or ending in Central Activities Areas (CAA). But what we’re told is that 3% of trips started or ended in CAAs in 2009-10. That’s valuable information in its own right, but it’s not telling us if the Department is succeeding on this indicator. It’s not telling us if there was an increase. It’s not telling us what the indicator says it tells us!

Another example is the target to reduce fuel consumption of petrol vehicles – the result is given as 11.4 litres per vehicle kilometre, but again there’s no indication whether things are getting better or getting worse. Likewise, the result for the car occupancy rate indicator is 1.2 persons (it’s meant to reduce); the greenhouse emissions of the vehicle fleet is 384 g/km (it’s meant to reduce); the average weight carried per freight vehicles is 4.19 tonnes (it’s meant to increase); and so on – but do these represent an improvement or a deterioration on the trend?

This is not a minor issue. Of the 34 indicators that claim to show an “increase” in good things or a “reduction” in bad things, 20 do not even attempt to show how they’ve changed. They’re all snapshots. Experts will in many cases know the context but annual reports are prepared for parliamentarians and for the people. Read the rest of this entry »

Why do the worst infrastructure projects get built?

Posted: July 19, 2011 | Author: | Filed under: Infrastructure, Management | Tags: , , , , , , , , , | 21 Comments

Inaccuracy of transportation project cost and benefit estimates by type of project, in constant prices (Flyvbjerg)

Under-estimating the cost of major infrastructure projects and over-estimating the demand is so chronic that forecasters deserve some harsh medicine, according to Professor Bent Flyvbjerg from Oxford University’s Said Business School. He says “some forecasts are so grossly misrepresented that we need to consider not only firing the forecasters but suing them too – perhaps even having a few serve time”.

Australians have plenty of experience with underperforming infrastructure projects. For starters, just in transport alone, there’s Brisbane’s Clem 7 road tunnel, Sydney’s Lane Cove and Cross City tunnels, the Brisbane and Sydney airport trains, Melbourne’s Myki ticketing fiasco, and the 2,250 km Freightlink rail line connecting Adelaide and Darwin. And they’re just the ones we know about!

Professor Flyvbjerg says cost overruns in the order of 50% in real terms are common for major infrastructure projects and overruns above 100% are not uncommon. Writing in the Oxford Review of Economic Policy, he argues that demand and benefit forecasts that are wrong by 20–70% compared with the actual outcome are also common.

Transport projects are among the worst performers (see exhibit). Professor Flyvbjerg examined 258 transport projects in 20 nations over a 70 year time frame. He found the average cost overrun for rail projects is 44.7% measured in constant prices from the build decision. For bridges and tunnels, the equivalent figure is 33.8%, and for roads 20.4%. The difference in cost overrun between the three project types is statistically significant and the size of the standard deviations shown in the first exhibit demonstrate the high degree of uncertainty and risk associated with these sorts of projects.

He also found that nine out of 10 projects have cost overruns; they happen in all nations; they’ve been a constant over the last 70 years; and cost estimates have not improved over time.

And it’s not just under-estimation of costs. Errors in forecasts of travel demand for rail and road infrastructure are also endemic. He found that actual passenger traffic for rail projects is on average 51.4% lower than forecast traffic. He says:

This is equivalent to an average overestimate in rail passenger forecasts of no less than 105.6 per cent. The result is large benefit shortfalls for rail. For roads, actual vehicle traffic is on average 9.5 per cent higher than forecasted traffic. We see that rail passenger forecasts are biased, whereas this is less the case for road traffic forecasts.

He also found that nine out of ten rail projects over-estimate traffic; 84% are wrong by over ±20%; it occurs in all countries studied; and has not improved over time.

Thus the risk associated with rail projects in particular is extraordinary. They face both an average cost overrun of 44.7% and an average traffic shortfall of 51.4%. Read the rest of this entry »

-Are there really limits to what planning can do?

Posted: March 7, 2011 | Author: | Filed under: Management, Planning | Tags: , , , , , , , | 7 Comments

States' and Territories responses to 23 land use planning 'challenges'

There are, but in Victoria those limits appear to be very elastic.

Because it controls the use of land, the whole complex edifice of planning regulation touches to a greater or lesser extent a lot of the things we do.

In a newly released report commissioned by COAG, the Productivity Commission gives us an insight into how the nation’s planning agencies think the land use control system influences our lives.

The report, Performance Benchmarking of Australian Business Regulation: Planning, Zoning and Development Assessments, examines the regulatory frameworks of each jurisdiction, the processes for supply of land, the bases for assessing developer contributions, compliance costs for business, and competition issues arising from planning decision-making.

As part of its investigations, the Commission asked each State and Territory to answer this question: “To what extent can government use the planning, zoning and DA system to positively influence the following challenges”?

The answers each jurisdiction provided to 23 “challenges”, graded from “no effect” through to “major effect”, are shown in the accompanying chart (copied from the report). The survey was completed between October and November 2010, prior to the Victorian State election.

Bear in mind that the survey relates specifically to the powers of land use planning agencies i.e. not transport or other agencies. Also, the planners were specifically asked about the scope to positively influence each of the challenges. There are some interesting claims here and some equally interesting comparisons between States and Territories.

There’s a consensus that, given (presumably) the right policies, land use planning can have a major positive influence on managing greenfield development, accommodating population growth, managing the transition to higher population densities, providing diverse/appropriate housing, and protecting biodiversity.

By and large I’d agree with that. My only caveat would be the understanding that some of the benefits will come from reducing rather than increasing the degree of planning intervention. A prime example is the many restrictions on constructing higher density housing within established urban areas.

Where the survey gets really interesting is outside these five key areas. Victoria in particular stands out from its peers. Read the rest of this entry »

Will changing management arrangements give us better cities?

Posted: November 7, 2010 | Author: | Filed under: Management, Planning, Public transport | Tags: , , , , , , , , , | 7 Comments

Counting votes - Federal election 2010

Almost everybody, it seems, from political parties to academics, think tanks and planning experts, reckons the key priority for improving planning and public transport in Melbourne is to reform the way they’re managed.

The clamour for revised governance arrangements in order to effect reform has been increasing in Melbourne, with groups like the Committee for Melbourne, the Greens, the Public Transport Users Association and the Committee for Economic Development of Australia agitating for change.

It’s therefore sobering to see the Grattan Institute pointing out that reformed governance arrangements are not a silver bullet. The Director of the Institute’s Cities program, Jane-Frances Kelly, makes the point that:

the evidence from successful overseas cities does not support the idea that changing governance structures will help. In the successful cities we examined, no one type of governance was dominant. Unnecessary changes to governance structures can also be a distraction from the things that are vital. In short, changing structures is no cure-all. Read the rest of this entry »