OECD Short-Term Economic Statistics Working PartyJune 26-28 2006 Analysis of revisions for...

OECD Short-Term Economic Statistics Working PartyJune 26-28 2006

Analysis of revisions for short-term economic statistics

Richard McKenzie OECD

OECD Short Term Economic Statistics Working Party


Analysis of revisions for short-term economic statistics

• Main results from OECD revisions analysis study for the index of industrial production

– Country comments and reactions

– Preliminary results from a similar OECD study on Retail Trade Volume

• OECD Main Economic Indicators Original Release Data and Revisions Database

– Providing the tools for countries to do their own analysis


Revisions analysis for IIP

• An attempt to analyse 3 dimensions of statistical quality and make comparisons across countries

– Timeliness

– Accuracy (assessed through revisions)

– Coherency (between IIP and GDP in industry)

• Also able to study the interactions between these quality measures

• Monthly snapshots of data published in MEI CD-Rom from February 1999 – March 2006


Main reasons for revisions in IIP

• Late data or revised data from respondents; later correction of errors or imputations

• Different estimation methodologies for early estimates and lower sample sizes

• Updating of seasonal factors

• Benchmarking to other sources (e.g. National Accounts or annual structural surveys)

• Changes to base period and changes in statistical methodology, classifications etc.

0

0.5

1

1.5

2

2.5

3

3.5

Mea

n Ab

solu

te R

evis

ion(

%)

3 Months Later 1 Year Later 2 Years Later

Mean absolute revision to first estimates of year-on-year growth rates


0

0.5

1

1.5

2

2.5

3

3.5

Mea

n Ab

solu

te R

evis

ion(

%)



Are revisions statistically significant?

• Mean absolute size of revisions is important

– Revisions analysis quantifies this and provides a basis to review our compilation methods

• Ideally revisions should centre around zero over time (i.e. equally likely to be + or - )

– If this is not true then reasons causing this tendency in the compilation process must be found

-0.8-0.6-0.4-0.2

00.20.40.60.8

11.21.41.61.8

Mea

n Re

visi

on(%

)

Month on Month Year on Year

Mean revision between first estimates and published one year later

-0.8

-0.6

-0.4

-0.2

0

0.2

0.4

0.6

0.8

1

1.2

1.4

1.6

1.8

Mea

n R

evis

ion(

%)




Which short term measures should we advise users to focus on?

• Are first estimates of month-on-previous-month growth rates from the IIP reliable enough to enable informed decision making?

– Not in most countries! First estimates of MoM growth rates revised by 2/3 initial value on average after one year (95% countries revised by more than 2/5)

– Shows the value that revisions analysis can provide to users

• How to extract the latest signals in estimates without being misleading?

40

50

60

70

80

90

100

Perc

enta

ge

% Sign(Later) = Sign(Earlier)

Reliability of MoM growth rates to signal expansion or contraction(sign of first estimate compared to that one year later)


Do revisions affect identification of turning points?

• Very difficult to study

– As future turning points interact with past revisions

• OECD Composite Leading Indicators dates turning points in IIP using the Bry-Boschan routine

– Compared series of first estimates of YoY growth rates with those published two years later, at the BB turning point spots

– Where turning points were evident in the YoY growth rates series, preliminary estimates and those 2 years later usually coincided or were very close


Timeliness and accuracy

• Is there a trade-off between timeliness and size of revisions for the index of industrial production?

• Might expect those countries which publish earlier to have higher revisions – at least in the following two months

– E.g. due to estimation based on lower response rates, less editing of errors, flash estimation techniques

IIP mean absolute revision to first estimate of monthly growth rate after two months

SMFFF

SF

S

S

0

0.2

0.4

0.6

0.8

1

1.2

1.4

Perc

ent

0

0.2

0.4

0.6

0.8

1

1.2

1.4

1.6

2 months 1 year 2 months 1 year 2 months 1 year 2 months 1 year

Finland Mexico Portugal Spain

Mea

n Ab

solu

te R

evis

ion(

%)

Before

After

Mean absolute revision to year-on-year growth rates before and after an improvement in timeliness.


Coherence analysis

• In concept the Index of industrial production should have a strong relationship to Value added in industry from the National Accounts

• Coherency test: compare annual growth rates of IIP (most recent series) to annual growth rates in value added in industry from the national accounts

– Maybe the countries with high revisions are more coherent (e.g. due to revisions caused by benchmarking?)

-0.2

0

0.2

0.4

0.6

0.8

1

1.2

Correlation between annual growth rates for the IIP and GDP in industry from the annual national accounts


Review of country comments to IIP revisions analysis study

• Comparisons with similar studies done by the national institute (or comments on results / methods used)

– Generally similar results for countries that had done their own studies (some exceptions)

– Need to look at other time windows and remove outliers

– Sign change analyse of MoM revisions

– Impact of seasonal effects should be separated

– Few comments on the validity of the timeliness analysis



• Comments on reasons for revisions and national revisions analysis policies

– About 1/3 countries reported aspects of their revisions analysis policy (some with recent changes and efforts to be clear to users in this area)

– Need to be wary of revisions analysis as definitive measure of quality (one user emphasised the preference for transparency on statistical methods)

– Reasons for revisions covered earlier, all in line with those presented in the paper



• Comments on the importance of performing revisions analysis for Short term statistics

– 2/3 countries stressed the importance of performing revisions analysis

– Many countries welcomed the provision of tools and a standardised approach which can save their resources


Preliminary results from Retail Trade Volume revisions analysis

• Very similar results to analysis of IIP

– Mean absolute revisions increase the longer the revision interval

– Revision size similar across countries – with some countries noticeably higher and with mean revision statistically significant

– First estimate of MoM growth rates subject to large revision within one year


0

0.5

1

1.5

2

2.5

AUS AUT BEL BRA CAN CHN CZE DEU DNK EMU ESP FIN GBR HUN

Mea

n A

bsol

ute

Rev

isio

n(%

)



0

0.5

1

1.5

2

2.5

IRL ITA J PN KOR MEX NLD NOR NZL POL PRT RUS SWE USA ZAF

Mea

n Ab

solu

te R

evis

ion(

%)


-0.8-0.6-0.4-0.2

00.20.40.60.8

1

AUS AUT BEL BRA CAN CHN CZE DEU DNK EMU ESP FIN GBR GRC HUN

Mea

n Re

visi

on(%

)




-0.4-0.3-0.2-0.1

00.10.20.30.40.50.6

IRL ITA JPN KOR MEX NLD NOR NZL POL PRT RUS SWE USA ZAF

Mea

n Re

visi

on(%

)



OECD Main Economic Indicators Original Release Data and Revisions Database

• Full time series of data published every month starting from the February 1999 edition of the Main Economic Indicators for 21 key economic variables

• Access OECD revisions analysis studies for GDP, Index of Industrial Production and Retail Trade Volume

• Automated programs and detailed user guide allowing users to perform there own revisions analysis for any country / variable combination available in the database

• Information on reasons for revisions and references to international recommendations for establishing revisions analysis policies (IMF SDDS and OECD)

Live version

• http://stats.oecd.org/mei/default.asp?rev=1

http://stats.oecd.org/mei/default.asp?rev=1


Conclusions / Future work

• Engaging users on results of revisions analysis studies

– Especially the expected reliability of different measures from first estimates – should we do more work to promote more appropriate measures?

• How to promote the facilities developed by the OECD

– Large saving of resources for NSIs to perform revisions analysis, with OECD or their own data

• How to use the results from revisions analysis to improve compilation methods?


THE END

OECD Short-Term Economic Statistics Working PartyJune 26-28 2006 Analysis of revisions for...

Documents

Transcript of OECD Short-Term Economic Statistics Working PartyJune 26-28 2006 Analysis of revisions for...