What does ‘random’ look like?

Which plot is random?

The exhibit shows two seemingly similar patterns – but one of them is random and one isn’t. Can you tell which is which? More in a moment.

I’ve taken these plots from Steven Pinker’s new book, The better angels of our nature: why violence has declined. It’s in a Chapter titled The statistics of deadly quarrels where he discusses the statistical patterning of wars and takes a small detour into “a paradox of utility”, specifically our tendency to see randomness as regularity with little clustering.

This cognitive illusion has relevance to all disciplines but is of particular interest to anyone interested in spatial issues, as a couple of these examples show.

Professor Pinker, who’s a psychologist at Harvard, cites the example of the London blitz, when Londoners noticed a few sections of the city were hit by German V-2 rockets many times, while other parts were not hit at all:

They were convinced that the rockets were targeting particular kinds of neighborhoods. But when statisticians divided a map of London into small squares and counted the bomb strikes, they found that the strikes followed the distribution of a Poisson process—the bombs, in other words, were falling at random. The episode is depicted in Thomas Pynchon’s 1973 novel Gravity’s Rainbow, in which statistician Roger Mexico has correctly predicted the distribution of bomb strikes, though not their exact locations. Mexico has to deny that he is a psychic and fend off desperate demands for advice on where to hide

Another example is The Gambler’s Fallacy – the belief that after a long run of (say) heads, the next toss will be tails:

Tversky and Kahneman showed that people think that genuine sequences of coin flips (like TTHHTHTTTT) are fixed, because they have more long runs of heads or of tails than their intuitions allow, and they think that sequences that were jiggered to avoid long runs (like HTHTTHTHHT) are fair

The exhibit above shows a simulated plot of the stars on the left. On the right it shows the pattern made by glow worms on the ceiling of the famous Waitomo caves, New Zealand. The stars show constellation-like forms but the virtual planetarium produced by the glow worms is relatively uniform.

That’s because glow worms are gluttonous and inclined to eat anything that comes within snatching distance, so they keep their distance from each other and end up relatively evenly spaced i.e. non-randomly. Says Pinker:

The one on the left, with the clumps, strands, voids, and filaments (and perhaps, depending on your obsessions, animals, nudes, or Virgin Marys) is the array that was plotted at random, like stars. The one on the right, which seems to be haphazard, is the array whose positions were nudged apart, like glowworms

Thus random events will occur in clusters, because “it would take a non-random process to space them out. The human mind has great difficulty appreciating this law of probability”. Read the rest of this entry »


Why did travellers stop using horses?

Yes, it’s Melbourne Cup time, “the richest two mile handicap in the world” since 1861. So, it’s obligatory to find a horsey topic.

In SuperFreakonomics, their follow-up to the highly successful Freakonomics, Steven Levitt and Stephen Dubner relate the Parable of Horseshit, or how the car and public transport saved New York from asphyxiating under tonnes of dung.

The authors’ purpose was to show that trends don’t go on forever and that technological innovation can have a key role in addressing severe problems (although this got them into a huge controversy when they applied this way of thinking to climate change!), but I want to stick to the equine angle.

Eric Morris provides a knowledgeable and informed account of the enormous problems posed by animals in urban areas, most especially horses, in From horse power to horsepower. It’s only nine pages, however there’s a much briefer account by Elizabeth Kolbert, published in the New Yorker. It’s from her review of SuperFreakonomics. Even if you already know the outline of the story, she brings some interesting detail:

In the eighteen-sixties, the quickest, or at least the most popular, way to get around New York was in a horse-drawn streetcar. The horsecars, which operated on iron rails, offered a smoother ride than the horse-drawn omnibuses they replaced. (The Herald described the experience of travelling by omnibus as a form of “modern martyrdom.”) New Yorkers made some thirty-five million horsecar trips a year at the start of the decade. By 1870, that figure had tripled.

The standard horsecar, which seated twenty, was drawn by a pair of roans and ran sixteen hours a day. Each horse could work only a four-hour shift, so operating a single car required at least eight animals. Additional horses were needed if the route ran up a grade, or if the weather was hot. Horses were also employed to transport goods; as the amount of freight arriving at the city’s railroad terminals increased, so, too, did the number of horses needed to distribute it along local streets. By 1880, there were at least a hundred and fifty thousand horses living in New York, and probably a great many more. Each one relieved itself of, on average, twenty-two pounds of manure a day, meaning that the city’s production of horse droppings ran to at least forty-five thousand tons a month. George Waring, Jr., who served as the city’s Street Cleaning Commissioner, described Manhattan as stinking “with the emanations of putrefying organic matter.” Another observer wrote that the streets were “literally carpeted with a warm, brown matting . . . smelling to heaven.” In the early part of the century, farmers in the surrounding counties had been happy to pay for the city’s manure, which could be converted into rich fertilizer, but by the later part the market was so glutted that stable owners had to pay to have the stuff removed, with the result that it often accumulated in vacant lots, providing breeding grounds for flies.

The problem just kept piling up until, in the eighteen-nineties, it seemed virtually insurmountable. One commentator predicted that by 1930 horse manure would reach the level of Manhattan’s third-story windows. New York’s troubles were not New York’s alone; in 1894, the Times of London forecast that by the middle of the following century every street in the city would be buried under nine feet of manure. It was understood that flies were a transmission vector for disease, and a public-health crisis seemed imminent. When the world’s first international urban-planning conference was held, in 1898, it was dominated by discussion of the manure situation. Unable to agree upon any solutions—or to imagine cities without horses—the delegates broke up the meeting, which had been scheduled to last a week and a half, after just three days. Read the rest of this entry »


A short break in transmission

The Melbourne Urbanist will be travelling for the next couple of weeks and is unlikely to be in a position to put up any new posts while on the road. You can still read the extensive archive and leave comments. Transmission will resume around the middle of October.


Links for urbanists – No. 4

Click to compare Manhattan's Highline before and after it was converted from an elevated rail line to an elevated park. Drag the slider to see more of either (thanks to Per Square Mile)

Assorted links to some of the useful, the informative, the interesting, and sometimes even the slightly weird sources I stumble across from time-to-time:

  1. Ghosts of Manhattan’s Highline (and other traces of the past). Some commentary on historical opposition to elevated rail lines in NY
  2. Some people are very serious about No Far King Par King across their driveways!
  3. A brief economic explanation of peak oil
  4. Some links on the decline in car travel – The end of motoring and The road less travelled
  5. Plea to abolish train zones – from the Dandenong Leader
  6. US rail construction costs compared to some other places
  7. The magic formula for increasing transit ridership – make it hard and make it expensive to use cars
  8. The agglomeration benefits of Britain’s Crossrail project
  9. Loud s*x is a billion dollar problem
  10. Is Charles Darwin the real founder of economics?
  11. Trends in car occupancy in Australia
  12. Life in the fast lane – Melbourne’s laneways
  13. Psychogeography and the end of planning (this is about Reyner Banham and LA)
  14. Cities as hotels – would privately owned cities work?
  15. Understanding liveable city rankings
  16. The Bolt case is actually not the perfect opportunity to argue bravely for freedom of expression. The judgement by Bromberg J, with summary, is here.
  17. Higher density creates jobs and increases productivity. This report in the NY Times is adapted from a chapter in Ryan Avent’s new e-book, The Gated City. Here’s another extract, this time from The Atlantic. You can buy a copy of this excellent book to read on your computer, iPad, iPhone or Kindle for less than $2 from Amazon. I’m reading it on my iPhone in those idle moments (you don’t need a Kindle).
  18. A collection of reviews, comments and reactions to The Gated City, from Alex Tabarrok, Lloyd Alter (and again), Yonah Freemark, Chris Bradford, Peter Gordon, David Levinson, Jim Gleeson, and Randall O’Toole. Some follow-up comments by Ryan Avent: Urbanists and evidence; Jobs and density; Moving towards stagnation (audio); and Expensive real estate drives away people and jobs (video)
  19. Ryan Avent yet again, but this time reviewing the volume that started the intellectual e-book fashion, Tyler Cowen’s The great stagnation (I sought to download it for $4 from Amazon at the start of the year but was refused access – eventually got an ePub, but it was a torrent of trouble. Note that I had no territorial problems with Amazon when I bought The Gated City). Some further analysis by Noah Smith.
  20. Stephen Rowley reports on his visit to the village that inspired the New Urbanism, Seaside, Florida
  21. Architecture for developing countries – leave the ‘design’ attitude at home please
  22. More on the importance of density (or is it city size?) for innovation and productivity
  23. The perils of opening a store in Manhattan. “She also realized too late that people wouldn’t be inclined to buy a dozen Glassybabys, the way they do in Seattle, because they would have to carry them home. “In Seattle, everyone had cars so we never thought about it,” Ms. Rhodes said”

Can selection bias shoot down an argument?

Manhattan in Motion

Urban policy is rich in opportunities for fallacious thinking – for example, surveys that purport to show huge latent demand for a particular mode of transport, but sample only users of that mode. So I’m always interested in new examples of where we can so easily go wrong.

Alex Tabarrok at Marginal Revolution recently provided a pointer to this great historical example by John D. Cook of the importance of selection bias. It isn’t specifically related to urban policy, but nevertheless provides a valuable lesson. Cook says:

During WWII, statistician Abraham Wald was asked to help the British decide where to add armour to their bombers. After analysing the records, he recommended adding more armour to the places where there was no damage!

According to Cook, while it seems backward at first, Wald realised his data came from bombers that hadn’t been shot down:

That is, the British were only able to analyse the bombers that returned to England; those that were shot down over enemy territory were not part of their sample. These (surviving) bombers’ wounds showed where they could afford to be hit. Said another way, the undamaged areas on the survivors showed where the lost planes must have been hit because the planes hit in those areas did not return from their missions.

Wald assumed that the bullets were fired randomly, that no one could accurately aim for a particular part of the bomber. Instead they aimed in the general direction of the plane and sometimes got lucky. So, for example, if Wald saw that more bombers in his sample had bullet holes in the middle of the wings, he did not conclude that Nazis liked to aim for the middle of wings. He assumed that there must have been about as many bombers with bullet holes in every other part of the plane but that those with holes elsewhere were not part of his sample because they had been shot down.

Here’s another example via Marc Gawley, who noted a BBC story implying that three quarters of those committing crimes during the London riots had previous convictions or cautions. Is that really true, he wonders, or:

is it that those with previous convictions have their details on a police database and it was therefore possible to identify (and find) those people based on images from photographs and recordings made last month? Read the rest of this entry »


9/11: Should America move on?

South Pool, 9/11 memorial, Ground Zero

It’s a very brave or very insensitive American who would publicly question his nation’s entire 9/11 memorial project on the day of the tenth anniversary. Yet just a few days ago, on the morning of 11 September, US economist Robin Hanson began his internationally popular blog, Overcoming Bias, by complaining that half the comics in his Sunday paper “are not-funny 9/11 memorials”.

I did my own mawkish 9/11 ‘memorialising’ last time (the video On the transmigration of souls), so I too was guilty of dishing out largely uninformative and non-analytical content on this occasion. But Professor Hanson has a much bigger message (the links are his):

In the decade since 9/11 over half a billion people have died worldwide. A great many choices could have delayed such deaths, including personal choices to smoke less or exercise more, and collective choices like allowing more immigration….

Yet, to show solidarity with these three thousand victims, we have pissed away three trillion dollars ($1 billion per victim), and trashed long-standing legal principles. And now we’ll waste a day remembering them, instead of thinking seriously about how to save billions of others. I would rather we just forgot 9/11.

Do I sound insensitive? If so, good — 9/11 deaths were less than one part in a hundred thousand of deaths since then, and don’t deserve to be sensed much more than that fraction. If your feelings say otherwise, that just shows how full fricking far your mind has gone.

Just to really drive home his point, Professor Hanson updated his post with a link to news agency Aljazeera, which featured an article Let’s forget 9/11 by American Tom Engelhardt, author of  The American way of war: how Bush’s wars became Obama’s and The end of victory culture.

Not surprisingly, given this was 11 September, some comments from readers were pretty direct:

This will, sadly, be the last time I read your blog.

You, Mr. Hanson, are an idiot. You have no conception of mythos or national dignity. Good bye, sir.

Robin, you’re “fricking” out of your mind.

A co-worker used to have a little sign in her office: Most people know how to remain silent. Few people know when.

Some other comments took issue with his basic argument:

Insensitivity will not stop people from being affected by some tragedies more than others.

Human nature is to attach greater significance to things that are close to us and have evoked personal feelings, rather than basing it on magnitude alone. And there is nothing wrong with that. Your moralizing is no better than the ridiculous moralizing that has gone on in the past ten years in reaction to 9/11. Everyone has their story to tell; yours involves looking down on the natural human reaction to a dramatic event. That is all.

Defence against terrorists, not solidarity with victims, explains the “pissing away” of three trillion dollars.

You cannot reasonably expect this sort of brazen and demented contrarianism to induce readers in search of a new moral framework to think, “hrm, maybe I’ll give this utilitarianism thing a whirl and see where it takes me!”

Might as well forget about the Holocaust too. In the bigger scheme of things, what were only 6M Jews?

But quite a few comments were supportive of Professor Hanson’s view:

I think what Robin is saying is that the response to 9/11 didn’t do any of the things that the “leaders” who pushed those responses said those responses were going to do. What those responses did do was piss away $3 Trillion (and counting), kill a bunch of US soldiers, and squander the “good will” of the rest of the world while committing great evils; the killing of many Iraqi civilians, the strengthening of al Qaeda and the bankrupting of the US economy.

If only one 1/100 of the social capital spent on 9/11 remembrance was spent on organ markets… I would expect more than 100x as many lives would have been saved compared to any anti-terrorist measures implemented because of 9/11. Read the rest of this entry »


Links for urbanists – No. 3

Villa Verticale - Cat Poljski (Etching, aquatint)

Assorted links to some of the useful, the informative, the interesting, and sometimes even the slightly weird sources I stumble across from time-to-time:

  1. The wonderful world of Cat Poljski.
  2. A “brief, wondrous” history of brutalist architecture. Back in the olden days when I was a student, we were weaned on photos of this sort of stuff. Like religion, if they get you young enough you’re a goner – I still love the pile of Mars Bars Moshe Safdie confected for Habitat 67 (although this photograph is its least flattering angle).
  3. The Naipaul test – can you tell the sex of the authors of these novels just by reading a brief extract?  V.S. Naipaul once famously claimed: “I read a piece of writing and within a paragraph or two I know whether it is by a woman or not”.
  4. Better pedestrian infrastructure is essential for safer roads and neighbourhoods – the tragic case of Raquel Nelson and her son AJ, in Atlanta.
  5. Have we already hit and passed ‘peak electricity’ in Australia?
  6. A simple economic model of fare evasion and non-compliance.
  7. Intercity buses are much cheaper than trains in the US (if you ignore the cost of building the roads).
  8. There’s lots of competition between PC makers but they’re all competing on price. Only one company makes the iMac but it has the freedom to make something really good.
  9. A lighter bicycle doesn’t make you any faster.
  10. Mark Twain on Taming the Bicycle (this is the real Mark Twain).
  11. The California Cycleway, opened in 1900, was an elevated tollway built specially for bicycle traffic through the Arroyo Seco, intended to connect the cities of Pasadena and Los Angeles.
  12. The Great Transition claims to provide the first comprehensive blueprint for building an economy based on stability, sustainability and equality.
  13. The parallel between climate change and teaching evolution in schools in Tennessee in 1925.
  14. Cost Overruns: How I Learned to Stop Worrying and Hate Bent Flyvbjerg.
  15. GPS and the end of the road – place and placelessness in America.
  16. 19th century cyclists paved the way for modern motorists’ roads.
  17. All aboard Tokyo’s last surviving streetcar.
  18. Melbourne is No. 8 in Askmen.com’s list of the 25 cities for blokes to visit in 2011.
  19. The top-10 least-polluting cities in the US – LA is No. 8 and Miami is No. 5. Having a temperate climate helps heaps.
  20. Video flythrough of Westfield’s new Stratford shopping centre in London.
  21. A new look at the industrial revolution – how it broke free from the shackles of land.
  22. Stockholm is the only city in the world where electors voted for congestion pricing.
__________________________________________________________________
BOOK GIVEAWAYfollow this link to be in the running for one of two copies of Meg Mundell’s Black Glass. Entries close midday Thursday 15 September 2011.