Post on 12-Nov-2014
description
Richard Akerman
NRC-CISTI
Presented at Access 2009, Oct. 1, 2009
Will We Command Our Data?From the Petascale to the Personal
Overview
• Definitions / Assumptions• How Big is Data?• Four Sources of Data
• Drivers• Activities
Definitions / Assumptions
• Petabyte = 1000 Terabytes• data = datasets• “data is”
How Big is Data?
http://www.instructables.com/file/FA9N61CF54HJ6GG/
How Big is Data?
http://www.flickr.com/photos/doctorow/2731870631/
How Big is Data?
http://en.wikipedia.org/wiki/File:Postduif.jpg
Four Sources of Data
• Research data• Government data• Library data• Personal data
General Drivers
• Since 2000, a convergence of factors:–Value of sharing–Ease of sharing–Level of sharing (machine level)
Specific Drivers: Research Data
• OECD Principles and Guidelines for Access to Research Data from Public Funding (April 2007)
• The Toronto Statement on prepublication data sharing (September 2009)
OECD Principles
• “Open access to research data from public funding should be easy, timely, user-friendly and preferably Internet-based.”
http://www.flickr.com/photos/ben-zvan-photography/468487548/
Specific Drivers: Open Government Data
• US Memorandum on Transparency and Open Government (January 2009)
• US Memorandum on the Freedom of Information Act (January 2009)
Specific Drivers: Open Government Data
• UK Power of Information Task Force Report (March 2009)– Modernise data publishing and reuse
http://poit.cabinetoffice.gov.uk/poit/category/data-final/– “public information held by for example the police,
health bodies and local authorities is often not available. This is bad for democratic expression, the economy and citizen customers.”
• Data.gov (May 2009)• UK PM Brown meets with Sir Berners-Lee (Sept. 2009)
Specific Drivers: Library Data
• ILS Customer Bill-of-Rights, John Blyberg (November 2005)
• “Berkeley Accord” (March 2008)
Specific Drivers: Personal Data
• Wired cover feature “Living by numbers” (July 2009)– “
Know Thyself: Tracking Every Facet of Life, from Sleep to Mood to Pain, 24/7/365”
– “Numbers are making their way into the smallest crevices of our lives. We have pedometers in the soles of our shoes and phones that can post our location as we move around town. We can tweet what we eat into a database and subscribe to Web services that track our finances. There are sites and programs for monitoring mood, pain, blood sugar, blood pressure, heart rate, … and prayers.”
Why Libraries
• Advocates• Exemplars• Experts
Research Data:DataCite
• http://www.datacite.org/• “DOIs for data”• “The long term vision of the partnership is to
support researchers by providing methods for them to locate, identify, and cite research datasets with confidence.”
Research Data: Gateway to Data Sets
• NRC-CISTI, Gateway to (Canadian) Scientific Data Sets
• http://cisti-icist.nrc-cnrc.gc.ca/eng/services/cisti/scientific-data/data-sets/– e.g. Canadian Astronomy Data Centre (CADC),
Large Synoptic Survey Telescope (LSST)
Government Data: Canada - Federal
• http://geogratis.cgdi.gc.ca/• StatsCan Data Liberation Initiative (DLI)• Ontario Data Documentation, Extraction Service and Infras
tructure Initiative (ODESI)– “The project will target Statistics Canada datasets...
The files will be marked-up using DDI, an international, XML-based metadata tagging system which allows data resource discovery, distributed access, extraction and analysis.”
Library Data
• A million free covers from LibraryThing• Open Library
http://openlibrary.org/dev/docs/data• Talis Connected Commons• MESUR – Services• http://id.loc.gov/ (LCSH)
APIs vs raw data
• APIs– Always serve up latest data– Control over access– Tracking/stats– Advanced/complex functionality on top of the data
• Raw data– Unconstrained / can do things never imagined by API– Hard to track / version– Can lose metadata– Allows choice of computing
Personal Data:Total Recall
• http://totalrecallbook.com/ (Sept. 2009)
Richard Akerman
© 2009 Government of Canada
Licensed in the Creative Commons
Thank You
http://creativecommons.org/licenses/by-nc-sa/2.5/ca/