© 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

Jed Sundwall, Open Data Global Lead

November 29, 2016

Earth on AWSWorking with Planetary-scale Data in the Cloud

What to Expect from the Session

• About open data on AWS

• Advantages of Amazon S3 for sharing data

• How E&J Gallo and DigitalGlobe use AWS to work with

geospatial data

Why Does AWS Care About Open Data?

Sharing data on AWS makes it accessible

to a large and growing community of

researchers, entrepreneurs, and

enterprises who use the AWS cloud.

“…data must be organized, well-

documented, consistently formatted, and

error free. Cleaning the data is often the

most taxing part of data science, and is

frequently 80% of the work.”

- Data Driven by DJ Patil and Hilary Mason

Undifferentiated Heavy Lifting

Traditional Data Acquisition

Data Acquisition in the Cloud

When data is shared in the cloud, anyone

can analyze any volume of data without

needing to download or store it themselves.

Landsat on AWS

Landsat

Landsat 8 satellite Raster data

RGB

Visible light

Infrared

Vegetation

Shortwave infrared

Urban areas

Wellington, New Zealand

Landsat on AWS

Amazon

EC2

s3://landsat-pds

.tarUSGS

.tiff

Landsat on AWS

Graph by Drew Bollinger (@drewbo19) at Development Seed

landsatonaws.comServerless browser interface to Landsat on AWS

New open source tools

• GDAL – http://www.gdal.org/

• Rasterio – https://mapbox.github.io/rasterio/

• sat-utils suite – https://github.com/sat-utils

Elevation

Models

Aerial

Imagery

Climate

Models

Satellite

Imagery

High-resolution

Radar

aws.amazon.com/earth

AWS Cloud Credits for Research

provide promotional AWS cloud credits

for anyone to conduct research using

Earth Observation data.

aws.amazon.com/earth/research-credits

© 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

John Webb, Manager, Information Technology , E & J Gallo Winery

Wine and Grape Supply Data LakeE & J Gallo

E&J GALLO WINERY

Background

E&J Gallo Winery

Largest Winery in the world

• Established in 1933 and headquartered in Modesto,

California, E. & J. Gallo Winery remains a privately-held

and ever-growing company that employs 6,000 people

worldwide.

• Largest land owner in the State of California

Products and Distribution

• With products available in more than 90 countries, E. &

J. Gallo Winery is the largest exporter of California wine,

and imports wines from Argentina, Italy, New Zealand

and Spain

• WE hold 90 brands and include table and sparkling

wines, beverage products, dessert wines and distilled

spirits

Gallo Portfolio

Popular Premium Ultra

Premium

WINEGROWING

Perennial v. Row Crop

Wine and Grape Supply

Grower Relations

• Manage external grape purchases

• Inform growers on best management practices

Gallo Vineyards Inc.

• Gallo owned vineyards

• Irrigation management

• Yield and Quality estimation

• Best management practices for vineyard management

• Operations planning and scheduling

Viticulture Chemistry and Enology

• Internal research division

• Quality and yield enhancement

• Variable rate irrigation

Winemaking

Complexities of Wine Growing

Not a Biomass Product

• Manage more than just nitrogen and water

• A number of activities go against a given vine

Perennial Crop

• Average vineyard lifecycle of 24 years

• Vine management is an on-going activity

• Canopy Management

• Shoot Thinning

• Vine Balancing

• Trellis Management

• Nutrients

• Pesticides

High Value Crop Management

• Direct correlation between ranch management plan and

quality

• Yield impact associated with canopy management

activities

DATA DRIVEN INSIGHTS

Data Lake Modeling

Why Data Matters

We operate in an ‘Activity’ based agricultural environment

from which we can drive insights

• Events or Transactions help us manage our business

• These events are both structured and unstructured

Structured Events such as…

• Tons of grapes delivered to a specific winery

• Distribution of soil across a vineyard

• Ranch management transactions applied (i.e. shoot

thinning, leafing, dropping fruit etc.)

Unstructured Events such as…

• Imagery

• Soil Moisture Nodes

• Yield Monitors

• Mechanized Asset Control

Data Driven Value

Having a robust data and analytics platform allows us to

drive insights

• Grape to Bottle data management

• Improve Quality

• Increase Yield

• Detect and prevent early on-set disease

• Maintain a competitive supply position through predictive

yield estimation

Data Lake Model

EC2

Elastic Load Balancing

S3

Redshift

SQL on DB

Instance

EMR

Customer Facing

Product Layer

EDW

Reports

Data Processing,

Analysis, R&D Layer

Data Collection

Layer

Dashboard

Insights

D

a

t

a

L

a

y

e

r

© 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

Shay Har-Noy, VP & General Manager, Platform, DigitalGlobe

Getting Data Out of JailDigitalGlobe’s Geospatial Big Data Platform

@iheartcrowds

3,500,000 km2 collected EVERY DAY

13,200,000,000,000 pixels

Last 3 Days

Last 14 Days

Last 3 Months

With frequent coverage of high interest areas

Goal: Large scale information extraction

Three trends working in our favor

1.

Three trends working in our favor

2. hidden layer 1 hidden layer 2 hidden layer 3

Three trends working in our favor

3.

Open up Geo to world of innovators working on Deep Learning

Solve a new class of problems

to impact business, government, and people

strategic approach: Leverage the ecosystem

Crowdsourcing

– Data out of jail!

Storage Compute

GBD Partner Tools Tools

MapsAPI

Catalog API

Workflow API

Scale humans until machines get smarter

Use humans to create training data

Crowdsourcing

Training and testing set to allow benchmarking and iteration

We have common frameworks to accelerate prototyping

We need a common data set to evaluate performance

- 50cm WV-2 image w/8 spectral bands

- Each image 200m2

- Covering Rio de Janeiro, Brazil

- 7000 Geotiff images

Available for your computing pleasure on

Rio Olympics – Use Case

• Governments around the world were preparing for the upcoming Olympics in Rio De Janeiro

• Thorough understanding of the security and safety of the athletes is front of mind

• Knowledge of high risk areas, traffic patterns, line-of-sight and overall structure distribution is critical to ensure a secure games

What you can expect

1) Satellite imagery analytics will be way more common in your industry

2) Computer vision tasks are going to change the way we see the world 3) We are going to learn a lot

Try it yourself

aws.amazon.com/public-data-sets/spacenet/

developer.digitalglobe.com

Thank you!

Remember to complete

your evaluations!

Related Sessions