Terry Pratchett - The Science of Discworld I

A case in point. It is said that 84% of the children of Israeli fighter pilots are girls. What is it about the life of a fighter pilot that produces such a predominance of daughters? Could an answer lead to a breakthrough in choosing the sex of your children? Or is it just a statistical freak? It's not so easy to decide. Gut feelings are worse than useless, because human beings have a rather poor intuition for random events. Many people believe that lottery numbers that have so far been neglected are more likely to come up in future. But the lottery machine has no 'memory', its future is independent of its past. Those coloured plastic balls do not know how often they have come up in previous draws, and they have no tendency to compen­sate for past imbalances.

Our intuition goes even further astray when it comes to coinci­dences. You go to the swimming baths, and the guy behind the counter pulls a key at random from a drawer. You arrive in the changing room and are relieved to find that very few lockers are in use ... and then it turns out that three people have been given lock­ers next to yours, and it's all 'sorry!' and banging locker doors together Or you are in Hawaii, for the only time in your life ... and you bump into the Hungarian you worked with at Harvard. Or you're on honeymoon camping in a remote part of Ireland ... and you and your new wife meet your Head of Department and his new wife, walking the other way along an otherwise deserted beach. All of these have happened to Jack.

Why do we find coincidences so striking? Because we expect ran­dom events to be evenly distributed, so statistical clumps surprise us. We think that a 'typical' lottery draw is something like 5, 14, 27, 36, 39,45, but that 1,2, 3,19,20,21 is far less likely. Actually, these two sets of numbers have exactly the same probability: 1 in 13,983,816. A typical lottery draw often includes several numbers close together, because sequences of six random numbers between 1 and 49 are more likely to be clumpy than not.

How do we know this? Probability theorists tackle such ques­tions using 'sample spaces', their name for what we earlier called a 'phase space', a conceptual 'space' that organizes all the possibil­ities. A sample space contains not just the event that concerns us, but all possible alternatives. If we are rolling a die, for instance, then the sample space is 1, 2, 3, 4, 5, 6. For the lottery, the sample space is the set of all sequences of six different numbers between 1 and 49. A numerical value is assigned to each event in the sample space, called its 'probability', and this corresponds to how likely that event is to happen. For fair dice each value is equally likely, with a proba­bility of 1/6. Ditto for the lottery, but now with a probability of 1/13,983,816.

We can use a sample space approach to get a ball-park estimate of how amazing the Formula One coincidence was. Top drivers all lap at very nearly the same speed, so the three fastest times can eas­ily fall inside the same tenth-of-a-second period. At intervals of a thousandth of a second, there are one hundred possible lap times for each to 'choose' from: this list determines the sample space, The probability of the coincidence turns out to be one chance in ten thousand. Unlikely enough to be striking, but not so unlikely that we ought to feel amazed.

Estimates like this help to explain astounding coincidences reported in newspapers, such as a bridge player getting a 'perfect hand', all thirteen cards in one suit. The number of games of bridge played every week worldwide is huge, so huge that every few weeks the actual events explore the entire sample space. So occa­sionally a perfect hand actually does turn up, with the frequency that its small but non-zero probability predicts. The probability of all four players getting a perfect hand at the same time, though, is so micoscopic that even if every planet in the galaxy had a billion inhabitants, all playing bridge every day for a billion years, you wouldn't expect it to happen.

Nevertheless, every so often the newspapers report a four-way perfect hand. The sensible conclusion is not that a miracle hap­pened, but that something changed the odds. Possibly the players got close to a four-way perfect hand, and the tale grew in the telling, so that when the journalist arrived with a photographer, another kind of narrative imperative ensured that their story fitted what the jour­nalist had been told. Possibly they deliberately cheated to get their names in the papers. Scientists, especially, tend to underestimate the propensity of people to lie. More than one scientist has been fooled into accepting apparent evidence of extrasensory perception or other 'supernatural' events, which can actually be traced to delib­erate trickery.

Many other apparent coincidences, on close investigation, slither into a grey area in which trickery is strongly suspected, but may never be proved, either because sufficient evidence is unob­tainable, or because it's not worth the trouble. Another way to be fooled about a coincidence is to be unaware of hidden constraints that limit the sample space. That 'perfect hand' could perhaps be explained by the way bridge players often shuffle cards for the next deal, which can be summed up as: poorly. If a pack of cards is arranged so that the top four cards consist of one from each suit, and thereafter every fourth card is in the same suit, then you can cut (but not shuffle, admittedly) the pack as many times as you like, and it will deal out a four-way perfect hand. At the end of a game, the cards lie on the table in a fairly ordered manner, not a random one, so it's not so surprising if they possess a degree of structure after they've been picked up.

So even with a mathematically tidy example like bridge, the choice of the 'right' sample space is not entirely straightforward. The actual sample space is 'packs of cards of the kind that bridge players habitually assemble after concluding a game', not 'all possi­ble packs of cards'. That changes the odds.

Unfortunately, statisticians tend to work with the 'obvious' sam­ple space. For that question about Israeli fighter pilots, for instance, they would naturally take the sample space to be all children of Israeli fighter pilots. But that might well be the wrong choice, as the next tale illustrates.

According to Scandinavian folklore, King Olaf of Norway was in dispute with the King of Sweden about ownership of an island, and they agreed to throw dice for it: two dice, highest total wins. The Swedish king threw a double-six. 'You may as well give up now,' he declared in triumph. Undeterred, Olaf threw the dice ... One turned up six ... the other split in half, one face showing a six and the other a one. 'Thirteen, I win,' said Olaf. [45] Possibly he was holding a large axe at the time.

Something similar occurs in The Colour of Magic, where several gods are playing dice to decide certain events on the Discworld:

The Lady nodded slightly. She picked up the dice-cup and held it steady as a rock, yet all the Gods could hear the three cubes rattling about inside. And then she sent them bouncing across the table.

A six. A three. A five.

Something was happening to the five, however. Battered by the chance collision of several billion molecules, the die flipped onto a point, spun gently and came down a seven.

Blind lo picked up the cube and counted the sides.

'Come on ,' he said wearily. 'Play fair.'

Nature's sample space is often bigger than a conventional statisti­cian would expect. Sample spaces are a human way to model reality: they do not capture all of it. And when it comes to estimating sig­nificance, a different choice of sample space can completely change our estimates of probabilities. The reason for this is an extremely important factor, 'selective reporting', which is a type of narra-tivium in action. This factor tends to be ignored in most conventional statistics. That perfect hand at bridge, for instance, is far more likely to make it to the local or even national press than an imperfect one. How often do you see the headline BRIDGE PLAYER GETS ENTIRELY ORDINARY HAND, for instance? The human brain is an irrepressible pattern-seeking device, and it seizes on certain events that it considers significant, whether or not they really are. In so doing, it ignores all the 'neighbouring' events that would help it judge how likely or unlikely the perceived coincidence actually is.