Democratized Data Access for Sustainable Development · Democratized Data Access for Sustainable...
Transcript of Democratized Data Access for Sustainable Development · Democratized Data Access for Sustainable...
![Page 1: Democratized Data Access for Sustainable Development · Democratized Data Access for Sustainable Development ... “Cloud platform holders, ... Commercial EO data – WorldWind, ...](https://reader031.fdocuments.in/reader031/viewer/2022022605/5b69912b7f8b9acc608eac2b/html5/thumbnails/1.jpg)
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Democratized Data
Access for Sustainable
Development
Mark Ryland ([email protected])
Chief Architect, World-Wide Public Sector team
http://wwps.io
![Page 2: Democratized Data Access for Sustainable Development · Democratized Data Access for Sustainable Development ... “Cloud platform holders, ... Commercial EO data – WorldWind, ...](https://reader031.fdocuments.in/reader031/viewer/2022022605/5b69912b7f8b9acc608eac2b/html5/thumbnails/2.jpg)
???
… AWS: the global leader in
utility (aka cloud) computing
Amazon.com…
![Page 3: Democratized Data Access for Sustainable Development · Democratized Data Access for Sustainable Development ... “Cloud platform holders, ... Commercial EO data – WorldWind, ...](https://reader031.fdocuments.in/reader031/viewer/2022022605/5b69912b7f8b9acc608eac2b/html5/thumbnails/3.jpg)
Salim Ismail’s
Exponential Organizations: Why new organizations are ten times better, faster, and
cheaper than yours (and what to do about it)
![Page 4: Democratized Data Access for Sustainable Development · Democratized Data Access for Sustainable Development ... “Cloud platform holders, ... Commercial EO data – WorldWind, ...](https://reader031.fdocuments.in/reader031/viewer/2022022605/5b69912b7f8b9acc608eac2b/html5/thumbnails/4.jpg)
It’s hard to pin down exactly when this new
[exponential] organizational form emerged…
If we had to pick an official ExO origin date, it
would be March 2006, when Amazon launched
AWS and created the low-cost “Cloud” for
medium and small businesses. From that date
on, the cost of running a data center moved
from fixed CAPEX cost to variable cost. Today,
it is almost impossible to find a single startup
that doesn’t use AWS.
“ ” —Exponential Organizations, p.46
![Page 5: Democratized Data Access for Sustainable Development · Democratized Data Access for Sustainable Development ... “Cloud platform holders, ... Commercial EO data – WorldWind, ...](https://reader031.fdocuments.in/reader031/viewer/2022022605/5b69912b7f8b9acc608eac2b/html5/thumbnails/5.jpg)
“…data must be organized, well-
documented, consistently formatted, and
error free. Cleaning the data is often the
most taxing part of data science, and is
frequently 80% of the work.”
—Data Driven by DJ Patil and Hilary Mason
Cloud eliminates the
“undifferentiated heavy
lifting” behind data science
When (cleansed and curated) data is
shared in the cloud, anyone can
analyze any volume of data without
needing to download or store it
themselves.
Cloud democratizes
access to data, and the
computing power needed
to process it
![Page 6: Democratized Data Access for Sustainable Development · Democratized Data Access for Sustainable Development ... “Cloud platform holders, ... Commercial EO data – WorldWind, ...](https://reader031.fdocuments.in/reader031/viewer/2022022605/5b69912b7f8b9acc608eac2b/html5/thumbnails/6.jpg)
Public Datasets on AWS
Many high-value datasets are available for anyone to
access at zero cost on AWS
Examples include:
Landsat on AWS 3K Rice Genome NEXRAD on AWS
![Page 7: Democratized Data Access for Sustainable Development · Democratized Data Access for Sustainable Development ... “Cloud platform holders, ... Commercial EO data – WorldWind, ...](https://reader031.fdocuments.in/reader031/viewer/2022022605/5b69912b7f8b9acc608eac2b/html5/thumbnails/7.jpg)
RGB
Visible light
Infrared
Vegetation
Shortwave infrared
Urban areas
Wellington, New Zealand
![Page 8: Democratized Data Access for Sustainable Development · Democratized Data Access for Sustainable Development ... “Cloud platform holders, ... Commercial EO data – WorldWind, ...](https://reader031.fdocuments.in/reader031/viewer/2022022605/5b69912b7f8b9acc608eac2b/html5/thumbnails/8.jpg)
landsatonaws.com Serverless browser interface to Landsat on AWS
![Page 9: Democratized Data Access for Sustainable Development · Democratized Data Access for Sustainable Development ... “Cloud platform holders, ... Commercial EO data – WorldWind, ...](https://reader031.fdocuments.in/reader031/viewer/2022022605/5b69912b7f8b9acc608eac2b/html5/thumbnails/9.jpg)
![Page 10: Democratized Data Access for Sustainable Development · Democratized Data Access for Sustainable Development ... “Cloud platform holders, ... Commercial EO data – WorldWind, ...](https://reader031.fdocuments.in/reader031/viewer/2022022605/5b69912b7f8b9acc608eac2b/html5/thumbnails/10.jpg)
Landsat on AWS
Graph: Drew Bollinger (@drewbo19) at Development Seed
Old queries: 18,000
New queries: 72,000
Old time (avg sec): 375
New time (avg sec): 75
Difference (sec): 300
Time saved (sec): 21.6M
Time saved (days): 250
Some data about open data
![Page 11: Democratized Data Access for Sustainable Development · Democratized Data Access for Sustainable Development ... “Cloud platform holders, ... Commercial EO data – WorldWind, ...](https://reader031.fdocuments.in/reader031/viewer/2022022605/5b69912b7f8b9acc608eac2b/html5/thumbnails/11.jpg)
NOAA’s analysis of NEXRAD usage:
• Making it available on AWS led to a 230
percent increase in total usage
• Simultaneously decreased usage of
their servers by 50 percent
• Having the data enhanced and available
close to computing resources on the
cloud has made the data much more
usable
Some data…
![Page 12: Democratized Data Access for Sustainable Development · Democratized Data Access for Sustainable Development ... “Cloud platform holders, ... Commercial EO data – WorldWind, ...](https://reader031.fdocuments.in/reader031/viewer/2022022605/5b69912b7f8b9acc608eac2b/html5/thumbnails/12.jpg)
12
Estimated net economic benefit thru 2020: €6M+
“Cloud platform holders, providing storage and
easy access as well as cloud processing power
and tools for basic image processing, are
expected to play a major role in the EO data
market in the coming decade, as already
witnessed for instance with Amazon Web
Services and its open data pilot project based on
Landsat data.”
Report mentions impact of about AWS
customers/ partners Digital Globe,
Planet Labs; start-ups Plume Labs
(France), Building Radar (Germany),
Sinergise (Slovenia), and Vizzuality
(Spain/UK)
![Page 13: Democratized Data Access for Sustainable Development · Democratized Data Access for Sustainable Development ... “Cloud platform holders, ... Commercial EO data – WorldWind, ...](https://reader031.fdocuments.in/reader031/viewer/2022022605/5b69912b7f8b9acc608eac2b/html5/thumbnails/13.jpg)
Sinergise Sentinel Hub
http://sentinel-hub.com
WMS Commercial EO data – WorldWind, GeoEye,...
Aerial imagery (drone, plane)
Other raster data
Open EO data - Sentinel-1, Sentinel-2, Landsat, etc.
Cloud GIS
Web applications
Open source GIS (QGIS, OpenJump,..)
Commercial GIS applications
WMTS
Machine learning
API
![Page 14: Democratized Data Access for Sustainable Development · Democratized Data Access for Sustainable Development ... “Cloud platform holders, ... Commercial EO data – WorldWind, ...](https://reader031.fdocuments.in/reader031/viewer/2022022605/5b69912b7f8b9acc608eac2b/html5/thumbnails/14.jpg)
Building Radar
Building Radar is a Munich-based
start-up that combines web
scraping, Sentinel-2 data, natural
language processing, computer
vision, and machine learning to
identify and evaluate construction
projects—all on AWS
http://buildingradar.com
![Page 15: Democratized Data Access for Sustainable Development · Democratized Data Access for Sustainable Development ... “Cloud platform holders, ... Commercial EO data – WorldWind, ...](https://reader031.fdocuments.in/reader031/viewer/2022022605/5b69912b7f8b9acc608eac2b/html5/thumbnails/15.jpg)
Elevation
Models
Aerial
Imagery
Climate
Models
Satellite
Imagery
High-resolution
Radar
aws.amazon.com/earth
![Page 16: Democratized Data Access for Sustainable Development · Democratized Data Access for Sustainable Development ... “Cloud platform holders, ... Commercial EO data – WorldWind, ...](https://reader031.fdocuments.in/reader031/viewer/2022022605/5b69912b7f8b9acc608eac2b/html5/thumbnails/16.jpg)
UN Community already in the Cloud
• Int’l Civil Aviation Organization
has been using AWS for years
• Huge increase in functionality,
agility, and time to results
• Cost-effective, particular for
bursty workloads
• Public as well as private
shared data
![Page 17: Democratized Data Access for Sustainable Development · Democratized Data Access for Sustainable Development ... “Cloud platform holders, ... Commercial EO data – WorldWind, ...](https://reader031.fdocuments.in/reader031/viewer/2022022605/5b69912b7f8b9acc608eac2b/html5/thumbnails/17.jpg)
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Thank you!
Mark Ryland ([email protected])
Chief Architect, World-Wide Public Sector team
http://wwps.io