@ajturnerCTO DC Dev Center

@seangorman Strategist DC Dev Center

The State of Big Data for Geo

Trends

Of the 318 million mobile handsets shipped in 2011 79.9% were GPS enabled

iSuppli 2010

Jess3 2011

Meeker 2011

In the United States 72.2% of users accessed social media sites and blogs through their mobile devices – up 37% from 2010

ComScore 2011

What is Big Data?

Volume

MrGeo

MapReduce Geo (MrGeo)

• DIA project initiated by Terry Busch to extend geoprocessing to very large data sets

• Built by SPADAC -> GeoEye -> Digital Globe• Uses HDFS and MapReduce to store, process, and index

geospatial imagery and vector data• Interoperable with:

– ArcGIS Desktop– COMET– Google Earth– WMS clients– Adobe Flex and Silverlight environments

• Listed on the Hadoop Apache page to be open sourced

Brian Levy 2010

Velocity

Esper

GCEP

• Geospatial complex event processing• Extends the to include the ability to use Geospatial

constructs in the rules for filtering events • The ability to utilize the OGC Geospatial Functions

within Esper's Event Processing Language (EPL).–Contains, within, disjoint, intersects, overlaps,

crosses, intersection, touches, buffer, relate, union, convex hull

Variety

Neo4j Spatial• Utilities for importing from ESRI Shapefile as well as

Open Street Map files• Support for all the common geometry types• An RTree index for fast searches on geometries• Support for topology operations during the search

(contains, within, intersects, covers, disjoint, etc.)• The possibility to enable spatial operations on any graph

of data, regardless of the way the spatial data is stored, as long as an adapter is provided to map from the graph to the geometries.

• Ability to split a single layer or dataset into multiple sub-layers or views with pre-configured filters

Peter Neubauer 2011

Peter Neubauer 2011

ESRI and Big Data

Why Big Data?

1. Connecting Big Data to GIS

2. Scaling Geoprocessing

Case Study:NYC Marathon

Case Study:Colorado Wildfires

3. Evolving GeoAnalysis

Are we doing it backwards?

May 2nd OBL Tweets6,454 Tweets with GPS 6,000,000 total Tweets

Why run our analysis with six thousand instead of six million?

Tracking the first

OBL Tweet

Extract Analyze

ExtractAnalyze

or

Statistical Mechanics

Community Detection

The Future

Real Time Pattern Analysis and Alerting