Post on 26-Jan-2015
description
Ubiquitous, affordable devices and connectivity
Big data: The next frontier for innovation, competition, and productivity
May 2011 | byJames Manyika, Michael Chui, Brad Brown, Jacques Bughin, Richard
Dobbs, Charles Roxburgh, Angela Hung Byers, McKinsey Global Institute (MGI)
CloudBig data
Aggregation
MachineLearning Analytics
“Snow White Portraits”
http://www.mypersonality.org/wiki
http://www.pnas.org/cgi/doi/10.1073/pnas.1218772110
http://youarewhatyoulike.com/Kosinski, M., Stillwell D.J., Graepel T (2013) Private traits and
attributes are predictable from digital records of human behavior.
Proceedings of the National Academy of Sciences (PNAS).)
Data
Acquisition &
modelling
Collaboration
and
visualisation
Analysis &
data mining
Dissemination
& sharing
Archiving and
preserving
fourthparadigm.org
Data-intensive Research
X-Info
• Data ingest
• Managing a petabyte
• Common schema
• How to organize it
• How to reorganize it
• How to share with others
• Query and Vis tools
• Building and executing models
• Integrating data and Literature
• Documenting experiments
• Curation and long-term
preservation
The Generic Problems
Experiments &Instruments
Simulations
Literature
Other Archives
facts
facts
facts
facts
Questions
Answers
The Cloud
democratizes
access to scale &
economies of scale
Genomics Revolution
Genomics
Manual Measurement
Automated Measurement
Sample Collection
Historical Photographs
Counting
Ubiquitous
Motes
Aircraft SurveysModel Output
Typing
Monitoring
Collation
Quality assurance
Aggregation
Analysis
Reporting
Forecasting
Distribution
Done poorly,but a few notablecounter-examples
Done poorly to moderately,not easy to find
Sometimes done well,generally discoverable and available,
but could be improved
Integration
(I. Zaslavsky & CSIRO, BOM, WMO)
Web search:
“open weather
data azure”
World Wide Telescope
www.worldwidetelescope.org http://www.worldwidetelescope.org/planck.html
Datacenter
Edge point
Big Data Cloud Ecosystem
Windows Azure for Research Group
@azure4research
www.azure4research.com
Windows
Azure
Urban
Science
Machine
Learning
Environmental
Science
Life
Sciences
Computer
science
Engineering
British Library Labs cloud
analysis of digital catalogues,
including 19th Century books
scanned by Microsoft.@MechCuratorBot
mechanicalcurator.tumblr.com
Use laptops &
desktop computers
Overwhelmed by
data
Finding analysis
ever more difficult;
sharing even
harder
www.azure4research.com