2016 asprs track: spatial analysis at the continental scale: a practical approach by dan getman

Spatial Analysis at the Continental Scale: A Practical Approach

Dan GetmanGeospatial Big Data SolutionsDigitalGlobe9/22/2016 GISITR

~ 7.6M sqkm~ 24M people

~ 13M structures

https://www.psma.com.au/sites/default/files/styles/image_800x534/public/shutterstock_138152432.jpg?itok=GLja4tlG

What are we delivering?

http://i1.wp.com/altigator.com/wp-content/uploads/Multispectral-vs-Hyperspectral1.png?w=916

http://missionscience.nasa.gov/ems/08_nearinfraredwaves.html

http://s3.amazonaws.com/content.satimagingcorp.com/media/cms_page_media/66/stereo-ikonos-data-collection-page_1.jpg

http://content.satimagingcorp.com/static/galleryimages/3m-dsm-combined-web.jpghttp://s3.amazonaws.com/content.satimagingcorp.com/media/cms_page_media/66/stereo-ikonos-data-collection-page_1.jpg

http://www.ltdwatch.com/jls-in-3d/

10To Update Footer, Find Header & Footer in Insert Tab and Apply to All

Zoom In

11To Update Footer, Find Header & Footer in Insert Tab and Apply to All

https://www.psma.com.au/geoscape

https://www.psma.com.au/geoscape

LULCLULCTree

Masked DSM

Tree Masked

DSM

Roof Materials Classification



Swimming Pools and PV Systems

LULC

VNIR Images SWIR ImagesPan Images

Digital Surface Model

VNIR Images SWIR Images

Land Use Land Cover

Tree Height Assessment

Tree MaskedDigital Surface Model

Pan Images



Land Use Land CoverBuilding Footprints

Tree Height AssessmentRooftop Height Assessment


Pan Images

Solar Panels and Pools






Pan Images

Solar Panels and Pools

Composite ImageDigital Surface Model



Roof Top Material Classification



Rooftop Material Assessment

Pan Images

Data Provider

Analysis Paradigms (a few anyway)

Cloud Based Organized

Image StoreCloud Based

Scalable Compute

Cloud Based WMS or Tile

Store Cloud Based

Scalable Compute

Analysis Paradigms: Scalable (GBDx)

• Analyst never touches or purchases imagery, just information

• Analyst can run their own algorithms or anyone else's• Leverages compute size needed for each process• Parallelized on nodes and through data distribution

across nodes

• Configured for processing at the state, national, continental scale

• Configured for processing all imagery that meets certain specifications as it is collected

Cloud Based Image Processing Framework

Image Catalog

User Defined

Processing

Scalable Compute

Provider Defined

Processing

Analysis Paradigms: Scalable (GBDx) Raw Image

Orthorectify

Compensate for

Atmosphere

Pan Sharpen

Other Standard Function

User Defined Function

Create Output

Rest Endpoint

User Defined Function

User Defined Process

Specified Through the

APIOutput can be

imagery, vector, tabular

data

Starts with

“Raw” Image

GBDx

AWS S3

100s of Source Datasets

EC2 Instance

Postgres and Python

Many EC2 Instances

Many Python Process

AWS S3

Result Datasets

GBDx

AWS S3


EC2 Instance

Postgres and Python

Many EC2 Instances

Many Python Process

AWS S3

Result Datasets

Main module reads in

metadata and creates

optimized number of runs

GBDx

AWS S3


EC2 Instance

Postgres and Python

Many EC2 Instances

Many Python Process

AWS S3

Result Datasets

Each worker pulls needed

data and distributes

across its cores

GBDx

AWS S3


EC2 Instance

Postgres and Python

Many EC2 Instances

Many Python Process

AWS S3

Result Datasets

Main module aggregates results and submits the final aggregation process on GBDx, which pushes the final

result to the customer S3 bucket

GBDx

AWS S3


EC2 Instance

Postgres and Python

Many EC2 Instances

Many Python Process

AWS S3

Result Datasets

• Imagery is never purchased or downloaded, just analyzed

• Processing is automated and event driven• Customer gets information, not imagery

We Get Captured at

Different Times

We All Run In Parallel

As Data Arrives

• Ground elevation is higher than eave or roof elevation

• Roof elevations have non-standard distribution

• Eave is below 2.5m (one story building)

Issues Impacting Building Height Accuracy

Issue 1: Ground elevation is higher than eave or roof elevationArea used to estimate

ground elevation(purple, not including

inside of polygon)

Area used to estimate eave elevation

(pink, not including inside of polygon)

Area used to estimate roof elevation

(green, including inside of polygon)

Issue 1: Ground elevation is higher than eave or roof elevation

In considering the area covered by this doughnut, note that

the ground is is almost completely

obscured by trees or other buildings

Vegetation is masked out of the DSM, which means that this point is likely to be our only

“ground” measurement…

Issue 1: Ground elevation is higher than eave or roof elevation

This “building” has the same issue with surrounding trees, but as it’s really an empty pool that just looks like a building, the ground

elevation actually should be higher than the “eaves”

Issue 2: Roof elevations have non-standard distribution

Trees are intentionally

masked from the elevation data


Unfortunately, mainly due to mixed pixels, some trees are

missed in the masking


Q3 to Q4 range is large

Looks like a building


Q3 to Q4 range is large

Looks like a tree


Trees are removed and the whole distribution looks like a building

• Types of occurrences • Eave is < 2.5m and Roof is < 2.5m• Lots of sheds and water storage

in this group• Is it an error or a correct

estimate?

Issue 3: Calculated eave heights are below a one story building (2.5m)

Issue 3: Calculated eave heights are below a one story building (2.5m)Eave is < 2.5m and Roof is < 2.5m:

• Anyone can create a process that operates on imagery and add it to the ecosystem

• Processes can operate on every square meter of the globe

• Processes can operate on imagery as it is collected for an AOI

• Low resolution imagery can be used to determine which high resolution imagery is needed to save costs

• This can be driven by other data science applications

• Examples• Detect when an airport is operating

abnormally• Detect when air/auto traffic patterns change• Detect when there is more or less activity in

certain areas

Enormous Opportunities

Questions and Comments?

2016 asprs track: spatial analysis at the continental scale: a practical approach by dan getman

Technology

Transcript of 2016 asprs track: spatial analysis at the continental scale: a practical approach by dan getman