How accurate are your business predictions?

The behavioural science of forecasting and decision

making

Needle Partners• International full service law firm– Corporate & commercial focus

• Offices in Leeds, London & Kuala Lumpur

What is BE?• The study of how we make decisions

• The study of the biases that affect decision making

Content• The importance of predictions for your

business• The problems with current forecasting• The heuristics and cognitive biases that

affect predictions• Super-forecasters• Strategies for decision making

Step Game• Pick a number between 0 and 100• To win, your number must be two-thirds of

the average of all the numbers chosen by everyone else

Results

Mental Process• If everyone chooses a random

number– Average will be 50– 2/3 of 50 = 33– 2/3 of 33 = 22– 2/3 of 22 = 15–…….– 0 = Nash’s Equilibrium = Hyper rational

“the goal in the stock market is not necessarily to pick the best company. It is to pick the company that others think will be the best company. In fact, it is to pick the company that others think that others think that others think — and so on — is the best company.” John Maynard Keynes

Why predictions are important

• Business Planning

• Controlling Stock supply

• Counterfactuals

• Predicting competitor response

Forecasting Experts• Qualitative

• Time series analysis

• Causal models

Forecasting Experts

Forecasting Experts

Forecasting Experts• Self-Perceived Expertise Predicts Claims of

Impossible Knowledge – Atir, Rosenzweig & Dunning 2015

Expert love• Why do we love experts?– Fear of uncertainty

• Study shows that forecasters who give positive predictions are perceived better–More confident–More in depth analysis–More trustworthy

Rule Game• Try to figure out the rule for the

sequence• 2, 4, 8

Rule Game• The answer is simply that the next

number is larger than the previous number

• Confirmation bias: We seek evidence that supports our initial belief

Cognitive Bias• A cognitive bias refers to a systematic

pattern of deviation from norm or rationality in judgment, whereby inferences about other people and situations may be drawn in an illogical fashion.

Optical Illusion

Gambler’s Fallacy

Hot hand fallacy

Illusion of Control

False Consensus Effect• The tendency for people to assume

that their own opinions, beliefs, preferences, values, and habits are normal and that others also think the same way that they do.

Which group of companies (A or B) had the larger total sales revenues in 2003?

Group A: Reebok International, Hilton Hotels, Starbucks, Radioshack, Hershey Foods

Group B: CoconoPhillips, American International Group, McKesson, AmerisourceBergen, The Altria Group

• Recognition Heuristic: People tend to perceive information that is recognisable to be of greater significance.

• Availability Heuristic: mental shortcut that involves basing judgements on information and examples that immediately spring to mind.

Heuristics

Super Forecasters• Phillip Tetlock & IARPA• The Good Judgment Project (2011-2013)• Open source tournament style forecasting

competition

Super Forecasters• Intelligence• Experience• Domain expertise

Super ForecastersOpen mindednessOpen minded forecasters aren’t afraid to change their minds, are happy to seek out conflicting views and are comfortable with the notion that fresh evidence might force them to abandon an old view of the world and embrace something new.

Super ForecasterGrowth Mindset

• Determination• Self reflection• Willingness to learn

Super ForecasterTeams are better than individuals

• Performed 10% better• The average of everyone’s guesses are

usually best• Noise is cancelled out by averaging

Super ForecasterProbabilistic reasoning

• Problem-solving techniques based on the use of probability theory for weighing evidence and inferring conclusions.

• Bayesian statistics

Super ForecasterCHAMP• Comparisons are important• Historical trends can help• Average opinions• Mathematical models• Predictable biases exist and can be

allowed for.

Prediction Markets• Stocks are tied to events• Stock pay-outs depend on outcomes• Stock prices correlate with probabilities

Pre-Mortem

Pre-MortemGuy Kawasaki:“A pre-mortem is the best way that I know to increase the probability of a project’s success. It means you tell the team to assume the project failed and then come up with reasons why it failed.Then you eliminate those reasons.”

Final Lessons• Don’t always believe experts• Be aware of and avoid biases• Include others in predictions• Learn from previous predictions

Thank you!• Questions?