Fooled By Randomness and the Black Swan

There is a very interesting series of books by Nassim Taleb, more information is available at his site. Nassim works on Wall Street, and many of the examples are taken from that space, but his ideas have much wider application to the way we live our lives and see the world.

The Fooled By Randomness concept is really about the self-selection bias in statistics. If we are successful in business, we tend to believe that it is because we are doing something well, while in many cases the success is within the bounds of normal random variation. The example Nassim gives is stock market traders who get a run of successful trades, then crash spectacularly. It isn't that they were clever for a while and then did something stupid, they were just riding a series of random events. There are many examples in politics and religion that follow the same pattern.

"Pride comes before a fall" is another example, and the entire concept of becoming "over-confident" in ourselves and the outcomes of our actions is a foundation of religious belief.

Nassim's most recent book is called "The Black Swan - The Impact of the Highly Improbable". For hundreds of years the only breen of swan known in the western world was the white swan. If you asserted that black swans might exist you would be met with derision and incomprehension by all the experts in this subject, since there was no evidence, it was highly improbable. However there is nothing preventing black swans from existing, and when Australia was explored an entire breed of black swans was discovered, and all the experts were found to be wrong. Nassim says "The world is too ambiguous --the Black Swan comes from believing in crisp and neat texture of reality, and from the overestimation of our skills in mapping the world."

Many "believers" are uncomfortable with ambiguity, and base their beliefs on claims that things could not have happened by accident or randomness. They invoke paranormal or religious explanations for events that were certain to happen, but which are infrequent enough to be surprising. There are always going to be trees and cheese sandwiches etc. that randomly take on shapes that look something like a religious figure, so when one occurs it should not be regarded as surprising or miraculous.

People also find it hard to understand relative probabilities. For example, the probability that you will be killed in a car crash is relatively high, government figures show that in both 2003 and 2004 about 42,000 people per year died in the USA from this cause. However, these deaths occur a few at a time, spread over a large area, and are treated as background noise. There is very little media coverage of these deaths compared to gun massacres and terrorist attacks that are perhaps a thousand times less deadly when averaged over the years. The media attention and the effect on people's ability to feel safe is out of proportion to the actual risk of death.

On a more positive note, life on earth is not an accident. Its not unlikely, it was certain to occur given the initial conditions of a planet containing a mixture of elements that was about the right distance from the Sun to have liquid water on its surface. It took a long time, but any individual probability multiplied by a long enough time becomes a certainty.