Post on 12-Sep-2021
Exploiting Big Data for Statistics:Exploiting Big Data for Statistics:Some Implications for Policy
Andrew Wyckoff*
Directorate for Science, Technology & IndustryOrganisation for Economic Co-operation and Development
F b 25 February 2013
* The views expressed in this presentation are those of the authors anddo not necessarily reflect the opinions of the OECD or its Membership.
Overview
• Some applications today
• Implications for public policy:– Economic & social policies;Economic & social policies;– Statistical policy
• Discussion
S & C
Mesh Networks / Ambient Computing
Supply Chain Management Security & Access Control
Work In Process Tracking Consumer ApplicationsWork In Process Tracking Consumer Applications
Asset Management Environmental Applications
3
Internet traffic flows continue to grow
90,000
100,000
MOBILEDATA
Monthly global IP traffic (Petabytes), 2005-2014
70,000
80,000MOBILE DATA
Consumer Internet video
Consumer Voice over IP (VoIP)
50,000
60,000Consumer Online gaming
Consumer Video calling
20,000
30,000
40,000Consumer Web, email, and data
Consumer file sharing
CONSUMERVOD
0
10,000
20,000 CONSUMER VOD
BUSINESS INTERNET / INTRANET
2010 2011 2012 2013 2014 2015
Source: CISCO VNI 2011
A lot of big data buzz
• “Data is the new oil.” Andreas Weigend, Stanford (ex Amazon)
• “The future belongs to companies and people that turn data into products”, Mike Loukides, O’Reilly Media
“The challenge–“Ten reasons why Big Data will
“Why big data is a big deal”InfoWorld 9/1/11
The challenge–and opportunity–of big data”McKinsey Quarterly 5/11
Big Data will change the travel industry”Tnooz 8/15/11
“Keeping Afloat in a Sea of 'Big
InfoWorld – 9/1/11 McKinsey Quarterly—5/11
“Getting a Handle on Big Data with
Tnooz -8/15/11
“The promise ofin a Sea of Big Data”ITBusinessEdge – 9/6/11
on Big Data with Hadoop”Businessweek-9/7/11
The promise of Big Data”Intelligent Utility-8/28/11
IT has always had an impact on Statistics
see www.abs.gov.au
Price Statistics MIT Billion Price ProjectMIT Billion Price Project
Demand for Jobs / SkillsHelp Wanted Statistics from the Conference BoardHelp Wanted Statistics from the Conference Board
New Job Starts / Job ChangesLinkedInLinkedIn
OECD Output ForecastsSWIFTSWIFT
Some economic policy implications
BenefitsBenefits
• Timeliness & now casting• Timeliness & now-casting• Robustness & granularity• Affordability & access• Democratisation & creativityDemocratisation & creativity
Unknown properties of Web Data
Source: www nature comSource: www.nature.com
Some economic policy implications
Ch llChallenges
•Unknown bias•Potential Instability•Quality
Some implications for NSOs: Will they get by-passed by “Big Data” ?Will they get by passed by Big Data ?
Some policy implications for NSOs
Good news
• Big Data tools are becoming widely availablebecoming widely available
• The Cloud can address infrastructure needs
• The “statistical commons” • The statistical commons grows
Some policy implications for NSOs
Challenges to AddressC a e ges o dd ess
A &O hi•Access &OwnershipP i•Privacy
•Liability•Liability•Skills•Skills
Access to and ownership of proprietary data
Privacy Issues
• image from http://wwwalthdatainnovation.com/sites/datawork.drupalgardens.com/files/styles/large/public/target.jpg
Liability
Skills “…the sexy job in the next 10 years
will be statisticians.”
Source: NYT, 5 August 2009
Possible new NSO roles
• Take on a new mission as a trusted 3rd party whose role would be to certify the statistical quality y q yof these new sources?
• Issue statistical “best practices” in the use of non-traditional sources and the mining of “big data”?g g
• Use non-traditional sources to augment (and g (perhaps replace) their official series?
Going ForwardG “A bl d d d t ld” b ildi • Groves: “A blended data world” – building on-top of existing surveys
lib i b d d– Calibrating web data to survey data
• Use of relative vs. absolute measures, Use of relative vs. absolute measures, now-casting
• Develop new methodologies
• Active Experimentation extracting • Active Experimentation, extracting lessons, devising “best practices”