Facts, Figures & Fictions

33
Numis Securities Media Conferenc e 2011 Facts, Figures & Fictions - Johnbillett.com 1 Facts, Figures & Fictions Presentation to the Numis Securities Media Conference, London April 14 th 2011 by John Billett

description

Presentation on the uses & misues of data, embracing illustrations & examples, as presented to the Numis Securities Media Conference in London April 2011

Transcript of Facts, Figures & Fictions

Page 1: Facts, Figures & Fictions

Numis Securities Media Conference 2011

Facts, Figures & Fictions - Johnbillett.com

1

Facts, Figures & Fictions

Presentation to the Numis Securities Media Conference, London April 14th 2011 by John Billett

Page 2: Facts, Figures & Fictions

Numis Securities Media Conference 2011

Facts, Figures & Fictions - Johnbillett.com

2

7 * 7 = 48

(!)

Page 3: Facts, Figures & Fictions

Numis Securities Media Conference 2011

Facts, Figures & Fictions - Johnbillett.com

3

Data has been around for ages, but there’s much that’s “new”• Scale:

• “We are moving from samples to audits”

• Speed:• “Last year to yesterday”

• Interactive: • “Output becomes input”

• Intensity: • “More pressure for immediate results”

Page 4: Facts, Figures & Fictions

Numis Securities Media Conference 2011

Facts, Figures & Fictions - Johnbillett.com

4

Loads of data

1. We struggle with the unforeseen dilemma of what to do with the unimaginable quantities of data produced & received every year

2. 2011 will generate data requiring the storage of 2,700 billion iPads

3. A single USB stick holds the equivalent of 20 tonnes of printed paper

Page 5: Facts, Figures & Fictions

Numis Securities Media Conference 2011

Facts, Figures & Fictions - Johnbillett.com

5

Loads of views & loads of data • “Data are becoming a new type of

raw material that’s on a par with capital & labour” (Bain & Company)

• “We shall soon be collecting data about each & every movement, transaction and more. There will be no excuse for not living lives of organised moderation every hour of every day” (Karen Canty: Enders Analysis)

Page 6: Facts, Figures & Fictions

Numis Securities Media Conference 2011

Facts, Figures & Fictions - Johnbillett.com

6

Our freedom to get it wrong• These changes do not affect our

ability to misinterpret data • Indeed the apparent speed and

alleged accuracy of data could even increase that misinterpretation

• The fundamental “rules” of data have not changed

Page 7: Facts, Figures & Fictions

Numis Securities Media Conference 2011

Facts, Figures & Fictions - Johnbillett.com

7

4 + 4 = ?Depends if you’re buying or selling

Page 8: Facts, Figures & Fictions

Numis Securities Media Conference 2011

Facts, Figures & Fictions - Johnbillett.com

8

Buying or selling?

Gatwick Currency Exchange

Buying Selling

Euro 1.31 1.06

US 1.82 1.49

Japan 153 121

Page 9: Facts, Figures & Fictions

Numis Securities Media Conference 2011

Facts, Figures & Fictions - Johnbillett.com

9

Buying or selling?

Gatwick Currency Exchange

Buying Selling

Euro 1.31 1.06

US $ 1.82 1.49

Japan Yen 153 121

Spread

23%-26%!!The “average” doesn’t exist!

Page 10: Facts, Figures & Fictions

Numis Securities Media Conference 2011

Facts, Figures & Fictions - Johnbillett.com

10

Beware the Average

The parable of the statistician who drowned in a pond 5cms deep - on average

Page 11: Facts, Figures & Fictions

Numis Securities Media Conference 2011

Facts, Figures & Fictions - Johnbillett.com

11

Beware the Average

Data Set #1• Average = 50. Spread/dispersion 49-51• Average Useful

• Data Set #2• Average = 50. Spread/dispersion 25-75• Average Crap

Page 12: Facts, Figures & Fictions

Numis Securities Media Conference 2011

Facts, Figures & Fictions - Johnbillett.com

12

Confusion of Cause & Effect• Advertising Sales

• Massive FMCG advertiser uses “case rate” budget setting, with sales last year setting ad budget this year

• Advertising Sales

Page 13: Facts, Figures & Fictions

Numis Securities Media Conference 2011

Facts, Figures & Fictions - Johnbillett.com

13

Confusion of Cause & Effect• Price of TV Advertiser demand

• Our work for ISBA & ITV proves that the better value of TV the less advertisers spend

• Company profitability• Cash Flow• Consumer spending • Confidence

• Price of TV Advertiser demand

Page 14: Facts, Figures & Fictions

Numis Securities Media Conference 2011

Facts, Figures & Fictions - Johnbillett.com

14

False Accuracy

• There are no more than two significant figures

• 87.6 & 88.4 are both 88

• % calculation is meaningless on a data base smaller than 100 observations

Page 15: Facts, Figures & Fictions

Numis Securities Media Conference 2011

Facts, Figures & Fictions - Johnbillett.com

15

Journalists love data fiction• “Tesco’s pre Xmas 2010 sales have

declined”• “The rate of growth fell from 3.6%

to 2.9%”• (Source: BBC Today Programme - Radio 4)

• Tesco’s growth of 2.9% (lower than competitors’ growth rates) on significantly high sales the previous year meant Tesco had increased its market share

Page 16: Facts, Figures & Fictions

Numis Securities Media Conference 2011

Facts, Figures & Fictions - Johnbillett.com

16

The Direct Marketing Data Fallacy – Certainty in Fiction #1• “For the same money ad. X generated

twice the response of ad. Y so its twice as cost effective”

• This classic flawed data analysis overlooks:-

• Previous exposure patterns• Competitive activity• Other influences• Consumer experiences

Page 17: Facts, Figures & Fictions

Numis Securities Media Conference 2011

Facts, Figures & Fictions - Johnbillett.com

17

The Direct Marketing Data Fallacy – Certainty in Fiction #2• “The Google -charging for last click-

proposition, equates to cost per sales creation”

• This classic flawed data analysis overlooks:-

• Previous exposure patterns• Contacts “lost” before last click• Value of non- internet exposure• Whole journey approach

Page 18: Facts, Figures & Fictions

Numis Securities Media Conference 2011

Facts, Figures & Fictions - Johnbillett.com

18

The most common flaw – “The Ecological Fallacy”• “The drawing of inferences about

individual consumers based on aggregate level data analysis, results in the correct conclusions only by accident” (Robinson, Clancy, Berger et all)

• Market segmentation; targeting; clusters; mapping; promotions; etc.

• 16-34; ABC1; High Income: etc

Page 19: Facts, Figures & Fictions

Numis Securities Media Conference 2011

Facts, Figures & Fictions - Johnbillett.com

19

Conspiracy Theory, Common Sense, Coincidence & Cock Up• The Spillers Flour Graders ads

never appeared in the key baking season from Nov to Jan!

• Effectiveness? • Ad Stock? • Decay?• Financial Year Starts February

Page 20: Facts, Figures & Fictions

Numis Securities Media Conference 2011

Facts, Figures & Fictions - Johnbillett.com

20

Conspiracy Theory, Common Sense, Coincidence & Cock Up• P&G grew market share &

advertised in London only Monday-Thursday with Unilever only at Weekends

• Consumer receptivity?• Cheaper ad costs?• Econometric model insight?• Insufficient airtime

Page 21: Facts, Figures & Fictions

Numis Securities Media Conference 2011

Facts, Figures & Fictions - Johnbillett.com

21

Attribution lies in the eye of the beholder• Chicago• 5 year time series (Republican claim)• Police numbers increased

Teenage crime reduced • 20 year time series (Democrat claim)• New abortion laws High take up

in poorer families Fewer Criminals

Teenage crime reduced

Page 22: Facts, Figures & Fictions

Numis Securities Media Conference 2011

Facts, Figures & Fictions - Johnbillett.com

22

Words implying “precision” whose reality can be subject to large variation

• AVERAGE• BENCHMARK• DATA BASE• INDEX• NORM• STANDARD• SCORING• TARGET

Page 23: Facts, Figures & Fictions

Numis Securities Media Conference 2011

Facts, Figures & Fictions - Johnbillett.com

23

Measurement & Importance• You can “measure” what you can

measure, but that may not be important

• Consumers’ claims about behaviour• Consumers’ intentions to act

• What is important may not be “measurable” in conventional terms

• Mixed media exposure & influence

Page 24: Facts, Figures & Fictions

Numis Securities Media Conference 2011

Facts, Figures & Fictions - Johnbillett.com

24

Reject consumers’ claims & accept consumers’ behaviour• “We assume what we are being told

is not the truth” (Amelia Torode: VCCP)

• “Data based on claimed use of digital boxes indicated the end of live viewing and death of TV advertising. Reality is viewing +15% & more ads viewed” (Jeremy Tester BSkyB)

Page 25: Facts, Figures & Fictions

Numis Securities Media Conference 2011

Facts, Figures & Fictions - Johnbillett.com

25

Four widespread data facts1. Chance events are often conspicuously

misinterpreted as accomplishments or failure

2. The connection between actions & results is not as direct as we might like to believe

3. We benefit from looking beyond superficial explanations

4. Never have so many data been taken out of files & left un analysed or misread

Page 26: Facts, Figures & Fictions

Numis Securities Media Conference 2011

Facts, Figures & Fictions - Johnbillett.com

26

An arduous route from data to action

• “Data & insights achieve nothing unless they reach the decision-makers and are then acted upon. Fear, politics, denial etc in most companies make this a non-trivial issue” (Patrick Barwise London Business School)

Page 27: Facts, Figures & Fictions

Numis Securities Media Conference 2011

Facts, Figures & Fictions - Johnbillett.com

27

Facts, Figures & Fictions Check List• #1 Averages are misleading unless

you know the range, spread & dispersion of observations around that average

• #2 Demand to know the number of observations in any data base plus the scale of any claimed “norms” or “benchmarks”

Page 28: Facts, Figures & Fictions

Numis Securities Media Conference 2011

Facts, Figures & Fictions - Johnbillett.com

28

Facts, Figures & Fictions Check List• #3 When examining any data,

always appreciate who commissioned & paid for the study & adjust accordingly

• #4 Know the issues & challenges of the business whose data is under examination

Page 29: Facts, Figures & Fictions

Numis Securities Media Conference 2011

Facts, Figures & Fictions - Johnbillett.com

29

Facts, Figures & Fictions Check List• #5 Investigate & encourage time

series data, not isolated one offs

• #6 Check the data you have is complete, representative & meaningful. Avoid false attribution.

Page 30: Facts, Figures & Fictions

Numis Securities Media Conference 2011

Facts, Figures & Fictions - Johnbillett.com

30

Facts, Figures & Fictions Check List• #7 Are the observations fit for your

purpose? Is there unexplained variability? Randomness, bias, error & chance are alive & well

• #8 If it ain’t simple to understand – don’t believe it. Reject Google-de-goop.

Page 31: Facts, Figures & Fictions

Numis Securities Media Conference 2011

Facts, Figures & Fictions - Johnbillett.com

31

Facts, Figures & Fictions

Presentation to the Numis Securities Media Conference, London April 14th 2011 by John Billett

Page 32: Facts, Figures & Fictions

Numis Securities Media Conference 2011

Facts, Figures & Fictions - Johnbillett.com

32

References & Acknowledgements

• Patrick Barwise : London Business School• “Freakonomics”: Steven Levitt & Stephen Dubner• “The Drunkards Walk”: Leonard Mlodinow• “The Ecological Fallacy”: Clancy, Berger & Magliozzi. Journal

of Advertising Research• “Facts from Figures”: Michael Moroney• Graeme Hutton: Universal McCann NY• “Market Segmentation: Fallacies & Faults”: Tonks & Farr

University of Lancaster. Marketing Review • Media Tel UK : MediaPost NY• Irene Pantazis: US Association of National Advertisers• Jeremy Tester: BSkyB• Andrew Walmsley

Page 33: Facts, Figures & Fictions

Numis Securities Media Conference 2011

Facts, Figures & Fictions - Johnbillett.com

33

• Johnbillett.com• 22A Vincent House• Vincent Square• London SW1P 2NB• [email protected]• 07836 200 321• www.johnbillett.com