What does ‘random’ look like?

Which plot is random?

The exhibit shows two seemingly similar patterns – but one of them is random and one isn’t. Can you tell which is which? More in a moment.

I’ve taken these plots from Steven Pinker’s new book, The better angels of our nature: why violence has declined. It’s in a Chapter titled The statistics of deadly quarrels where he discusses the statistical patterning of wars and takes a small detour into “a paradox of utility”, specifically our tendency to see randomness as regularity with little clustering.

This cognitive illusion has relevance to all disciplines but is of particular interest to anyone interested in spatial issues, as a couple of these examples show.

Professor Pinker, who’s a psychologist at Harvard, cites the example of the London blitz, when Londoners noticed a few sections of the city were hit by German V-2 rockets many times, while other parts were not hit at all:

They were convinced that the rockets were targeting particular kinds of neighborhoods. But when statisticians divided a map of London into small squares and counted the bomb strikes, they found that the strikes followed the distribution of a Poisson process—the bombs, in other words, were falling at random. The episode is depicted in Thomas Pynchon’s 1973 novel Gravity’s Rainbow, in which statistician Roger Mexico has correctly predicted the distribution of bomb strikes, though not their exact locations. Mexico has to deny that he is a psychic and fend off desperate demands for advice on where to hide

Another example is The Gambler’s Fallacy – the belief that after a long run of (say) heads, the next toss will be tails:

Tversky and Kahneman showed that people think that genuine sequences of coin flips (like TTHHTHTTTT) are fixed, because they have more long runs of heads or of tails than their intuitions allow, and they think that sequences that were jiggered to avoid long runs (like HTHTTHTHHT) are fair

The exhibit above shows a simulated plot of the stars on the left. On the right it shows the pattern made by glow worms on the ceiling of the famous Waitomo caves, New Zealand. The stars show constellation-like forms but the virtual planetarium produced by the glow worms is relatively uniform.

That’s because glow worms are gluttonous and inclined to eat anything that comes within snatching distance, so they keep their distance from each other and end up relatively evenly spaced i.e. non-randomly. Says Pinker:

The one on the left, with the clumps, strands, voids, and filaments (and perhaps, depending on your obsessions, animals, nudes, or Virgin Marys) is the array that was plotted at random, like stars. The one on the right, which seems to be haphazard, is the array whose positions were nudged apart, like glowworms

Thus random events will occur in clusters, because “it would take a non-random process to space them out. The human mind has great difficulty appreciating this law of probability”.

By the way, this is an interesting book. As summarised by Peter Singer in his review for the New York Times, “the central thesis of “Better Angels” is that our era is less violent, less cruel and more peaceful than any previous period of human existence:

The decline in violence holds for violence in the family, in neighborhoods, between tribes and between states. People living now are less likely to meet a violent death, or to suffer from violence or cruelty at the hands of others, than people living in any previous century.

Even if you don’t buy his big thesis, this is a fascinating journey through the history of civilisation, philosophy, psychology, and much more.

To finish, here’s a puzzle Professor Pinker poses at the start of the chapter (he reckons almost no one gets it right. The answer is in the comments):

Suppose you live in a place that has a constant chance of being struck by lightning at any time throughout the year. Suppose that the strikes are random: every day the chance of a strike is the same, and the rate works out to one strike a month. Your house is hit by lightning today, Monday. What is the most likely day for the next bolt to strike your house?

I’ve looked at interesting statistical “stories” before –e.g. see Can selection bias shoot down an argument.

Advertisements

8 Comments on “What does ‘random’ look like?”

  1. Michael says:

    Interesting article. Presumably your chance of a lightening strike is the same as before since the lightening would have no memory, or is it more likely to be tomorrow due to clustering?
    Does he discuss the changes in the expectations of violence in the book?

    You might also be interested in Daniel Kahneman’s Thinking, Fast and Slow. I haven’t read it but from interviews of him I have heard it sounds well worth a read – or at least listen to the LSE podcast of him talking about it. Unfortunately I already have a large stack of books waiting to be read.

    • Alan Davies says:

      That’s pretty good. From Pinker:

      The answer is “tomorrow,” Tuesday. That probability, to be sure, is not very high; let’s approximate it at 0.03 (about once a month). Now think about the chance that the next strike will be the day after tomorrow, Wednesday. For that to happen, two things have to take place. First lightning has to strike on Wednesday, a probability of 0.03. Second, lightning can’t have struck on Tuesday, or else Tuesday would have been the day of the next strike, not Wednesday. To calculate that probability, you have to multiply the chance that lightning will not strike on Tuesday (0.97, or 1 minus 0.03) by the chance that lightning will strike on Wednesday (0.03), which is 0.0291, a bit lower than Tuesday’s chances.

      There’s a coincidence. I got Kahneman’s new book on Tuesday (the marvel of e-books!) and will have a look at it on the weekend (I use a Sony Reader).

      • Russ says:

        Of course, in most scientific applications you’ll have the opposite information: a sequence of events with varying durations between them and (potentially) some change in frequency compared to previous observations. What the scientist would really like to know is whether there is in fact a change in the expected probability, or whether the trend is merely random noise.

        Or to put it another way, would Pinker recommend dropping Ricky Ponting?

        • Dudley Horscroft says:

          Slightly OT – the same applies to extremes of Climate Change. For extremes, we have a sequence of events, varying times between them, we don’t know if they occur more often or less often, we don’t know if they are worse or not and we don’t know if they reflect real change in climate or are just part of a totally random sequence. (Also depends on our definition of ‘extreme’!)

          But when we get a ‘heat wave’ or a long ‘drought’ and the news says that it is the worst since “X” (a date sometime in the past) we wonder.

          What we do know is that disasters are worse because (a) the population in subject areas has increased – so more people are affected, (b) inflation means the nominal cost has increased – although the real cost per person may be constant, and (c) neglect of proper precautions may make what was a survivable disaster when the infrastructure was built unsurvivable now, when the infrastructure has been allowed to deteriorate (think New Orleans and Katrina).

          Apologies if this is too off topic, but disasters are – one hopes – random events. Aren’t they?

      • wizofaus says:

        Hmm, I thought the answer was “today” – i.e. straight away, in the very next quantum unit of time…

        BTW, don’t suppose you can do a giveaway for this book!

        • Russ says:

          wizofaus, the highest probability occurs in the next specific unit of time, but because the units are days, that becomes Tuesday. Assuming the lightning strike was randomly distributed within a day, the probability of it occurring again that day (Monday) is 0.015. If the units of time were months then there’d be an average 63% of chance of getting out of the current month without a strike (0.97^15), and 25% chance of the following month as well (0.97^15). The probability of going an entire year (365 days) without lightning would be 0.0014%, you’d need to be a week into December before you’d expect your next lightning strike the following year.

  2. Urt says:

    My guess:
    Most likely day for the *next* lightning strike is tomorrow. Days after that have the same chance of a lighting strike, but a higher chance of one in the interim.


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s