Prediction - Taleb vs Silver

Can we predict the future?

• Silver suggests we can’t in a large number of domains after testing a range of past predictions.

• The accuracy of weather forecasting has significantly improved over the last 30 years.

• Earthquake predictors have generally failed. • Disease epidemic predictors have generally failed. • Most stock market predictors generally fail in the

long run.

The Signal and the Noise

• US elections • Baseball performance • Poker • Chess

The Signal and the Noise

• Weather forecasting is a (limited) success story. • The science of earthquake forecasting seems barely

to have evolved since the ninth century. • Economic forecasts have often failed to “predict”

recessions even once they were under way.

Prediction, forecasting and modelling

• A prediction is a definitive and specific statement about when and where an event (e.g. earthquake) will strike (Silver 2012)

• A forecast is a probabilistic statement usually over longer time scale e.g. 60% chance of earthquake over next 30 years. (Silver 2012)

• A model is a simplification of a real world system which attempts to represent relationships between variables.

Criticisms of Silver

• Perhaps understandably he does not highlight the uncertainty around his perfect predictions for 2012 US election. He was lucky!

• He talks about the importance of probabilistic statements yet describes prediction “failures” in forecasting election results, financial crisis.

• He has too much confidence in the ability of forecasting models. They cannot incorporate typically unique, non repeatable extreme events which are best studied separately.

The Black Swan

• The world is hideously more complex than a game of poker, baseball or political elections (ludic fallacy)

• A black swan event is an unexpected outlier that carries an extreme impact.

Introducing the turkey

A picture of the turkey on Christmas Eve

Did the turkey make a statistical miscalculation?

• According to Bayes’ theorem, every episode of the turkey being fed by a generous owner made it more confident of its safety.

• Think stock market crashes (Lehman Brothers, Enron etc) where incorporating past data makes you more confident of the validity of a positive trend and more likely to be harmed by the eventual crash.

Anti-Fragility

• Build in redundancies (Fukushima, flood defences) • Do not suppress randomness with tight controls. • Make sure participants have “skin in the game” • Expose yourself to positive black swans • Take risks with a known limited downside and unbounded

upside.

Extremistan and Mediocristan

• In Mediocristan, large samples contain no single instance that will significantly change the aggregate or total. The largest observation is insignificant to the sum (weights, heights, calorie intake, mortality rates)

• In Extremistan, inequalities are such that one single observation can disproportionately impact the aggregate or total (wealth, income, book sales, deaths in war, inflation rates, economic data)

The normal (Gaussian) distribution

Why can’t I predict in Extremistan?

• Imperfect information • Feedback loops • Chain reactions (positive/negative spirals) • Past data is taken from the world in a very

different state to the present (or future) and extrapolated

• Models are simplifications of the real world. • Models typically rely on tractable statistical

distributions (e.g. normal distribution) with thin tails.

What can we predict?

• The relative fragility, robustness and anti-fragility between two systems/objects.

• Mediocristan (sports, poker, elections) • Short term trends in Extremistan but not the

extreme event (MUCH more important). • Do not fall for the ludic fallacy of perceiving

the world as a simple, well organised game subject to consistent rules.

“First, do no harm”

• Predicting the weather, making small bets on sports or political elections is not going to harm or threaten society (minimal downside)

• Predicting budget deficits, interest rates, inflation rates, stock prices many years into the future can threaten the fabric of society if they are taken seriously and no contingencies are prepared (potentially large downside)

Conclusion

• We generally cannot predict the timing of black swan events without insider information.

• We cannot predict the severity of black swan events.

• We can assess the comparative fragility, robustness or anti-fragility of systems rather than trying to predict the exact timing, severity of the rare event.

• Make yourself robust or anti-fragile (if possible) to uncertain future events.